1*20a1d0f4STakashi Iwai===================== 2*20a1d0f4STakashi IwaiALSA PCM Timestamping 3*20a1d0f4STakashi Iwai===================== 4*20a1d0f4STakashi Iwai 5*20a1d0f4STakashi IwaiThe ALSA API can provide two different system timestamps: 6*20a1d0f4STakashi Iwai 7*20a1d0f4STakashi Iwai- Trigger_tstamp is the system time snapshot taken when the .trigger 8*20a1d0f4STakashi Iwai callback is invoked. This snapshot is taken by the ALSA core in the 9*20a1d0f4STakashi Iwai general case, but specific hardware may have synchronization 10*20a1d0f4STakashi Iwai capabilities or conversely may only be able to provide a correct 11*20a1d0f4STakashi Iwai estimate with a delay. In the latter two cases, the low-level driver 12*20a1d0f4STakashi Iwai is responsible for updating the trigger_tstamp at the most appropriate 13*20a1d0f4STakashi Iwai and precise moment. Applications should not rely solely on the first 14*20a1d0f4STakashi Iwai trigger_tstamp but update their internal calculations if the driver 15*20a1d0f4STakashi Iwai provides a refined estimate with a delay. 16*20a1d0f4STakashi Iwai 17*20a1d0f4STakashi Iwai- tstamp is the current system timestamp updated during the last 18*20a1d0f4STakashi Iwai event or application query. 19*20a1d0f4STakashi Iwai The difference (tstamp - trigger_tstamp) defines the elapsed time. 20*20a1d0f4STakashi Iwai 21*20a1d0f4STakashi IwaiThe ALSA API provides two basic pieces of information, avail 22*20a1d0f4STakashi Iwaiand delay, which combined with the trigger and current system 23*20a1d0f4STakashi Iwaitimestamps allow for applications to keep track of the 'fullness' of 24*20a1d0f4STakashi Iwaithe ring buffer and the amount of queued samples. 25*20a1d0f4STakashi Iwai 26*20a1d0f4STakashi IwaiThe use of these different pointers and time information depends on 27*20a1d0f4STakashi Iwaithe application needs: 28*20a1d0f4STakashi Iwai 29*20a1d0f4STakashi Iwai- ``avail`` reports how much can be written in the ring buffer 30*20a1d0f4STakashi Iwai- ``delay`` reports the time it will take to hear a new sample after all 31*20a1d0f4STakashi Iwai queued samples have been played out. 32*20a1d0f4STakashi Iwai 33*20a1d0f4STakashi IwaiWhen timestamps are enabled, the avail/delay information is reported 34*20a1d0f4STakashi Iwaialong with a snapshot of system time. Applications can select from 35*20a1d0f4STakashi Iwai``CLOCK_REALTIME`` (NTP corrections including going backwards), 36*20a1d0f4STakashi Iwai``CLOCK_MONOTONIC`` (NTP corrections but never going backwards), 37*20a1d0f4STakashi Iwai``CLOCK_MONOTIC_RAW`` (without NTP corrections) and change the mode 38*20a1d0f4STakashi Iwaidynamically with sw_params 39*20a1d0f4STakashi Iwai 40*20a1d0f4STakashi Iwai 41*20a1d0f4STakashi IwaiThe ALSA API also provide an audio_tstamp which reflects the passage 42*20a1d0f4STakashi Iwaiof time as measured by different components of audio hardware. In 43*20a1d0f4STakashi Iwaiascii-art, this could be represented as follows (for the playback 44*20a1d0f4STakashi Iwaicase): 45*20a1d0f4STakashi Iwai:: 46*20a1d0f4STakashi Iwai 47*20a1d0f4STakashi Iwai --------------------------------------------------------------> time 48*20a1d0f4STakashi Iwai ^ ^ ^ ^ ^ 49*20a1d0f4STakashi Iwai | | | | | 50*20a1d0f4STakashi Iwai analog link dma app FullBuffer 51*20a1d0f4STakashi Iwai time time time time time 52*20a1d0f4STakashi Iwai | | | | | 53*20a1d0f4STakashi Iwai |< codec delay >|<--hw delay-->|<queued samples>|<---avail->| 54*20a1d0f4STakashi Iwai |<----------------- delay---------------------->| | 55*20a1d0f4STakashi Iwai |<----ring buffer length---->| 56*20a1d0f4STakashi Iwai 57*20a1d0f4STakashi Iwai 58*20a1d0f4STakashi IwaiThe analog time is taken at the last stage of the playback, as close 59*20a1d0f4STakashi Iwaias possible to the actual transducer 60*20a1d0f4STakashi Iwai 61*20a1d0f4STakashi IwaiThe link time is taken at the output of the SoC/chipset as the samples 62*20a1d0f4STakashi Iwaiare pushed on a link. The link time can be directly measured if 63*20a1d0f4STakashi Iwaisupported in hardware by sample counters or wallclocks (e.g. with 64*20a1d0f4STakashi IwaiHDAudio 24MHz or PTP clock for networked solutions) or indirectly 65*20a1d0f4STakashi Iwaiestimated (e.g. with the frame counter in USB). 66*20a1d0f4STakashi Iwai 67*20a1d0f4STakashi IwaiThe DMA time is measured using counters - typically the least reliable 68*20a1d0f4STakashi Iwaiof all measurements due to the bursty nature of DMA transfers. 69*20a1d0f4STakashi Iwai 70*20a1d0f4STakashi IwaiThe app time corresponds to the time tracked by an application after 71*20a1d0f4STakashi Iwaiwriting in the ring buffer. 72*20a1d0f4STakashi Iwai 73*20a1d0f4STakashi IwaiThe application can query the hardware capabilities, define which 74*20a1d0f4STakashi Iwaiaudio time it wants reported by selecting the relevant settings in 75*20a1d0f4STakashi Iwaiaudio_tstamp_config fields, thus get an estimate of the timestamp 76*20a1d0f4STakashi Iwaiaccuracy. It can also request the delay-to-analog be included in the 77*20a1d0f4STakashi Iwaimeasurement. Direct access to the link time is very interesting on 78*20a1d0f4STakashi Iwaiplatforms that provide an embedded DSP; measuring directly the link 79*20a1d0f4STakashi Iwaitime with dedicated hardware, possibly synchronized with system time, 80*20a1d0f4STakashi Iwairemoves the need to keep track of internal DSP processing times and 81*20a1d0f4STakashi Iwailatency. 82*20a1d0f4STakashi Iwai 83*20a1d0f4STakashi IwaiIn case the application requests an audio tstamp that is not supported 84*20a1d0f4STakashi Iwaiin hardware/low-level driver, the type is overridden as DEFAULT and the 85*20a1d0f4STakashi Iwaitimestamp will report the DMA time based on the hw_pointer value. 86*20a1d0f4STakashi Iwai 87*20a1d0f4STakashi IwaiFor backwards compatibility with previous implementations that did not 88*20a1d0f4STakashi Iwaiprovide timestamp selection, with a zero-valued COMPAT timestamp type 89*20a1d0f4STakashi Iwaithe results will default to the HDAudio wall clock for playback 90*20a1d0f4STakashi Iwaistreams and to the DMA time (hw_ptr) in all other cases. 91*20a1d0f4STakashi Iwai 92*20a1d0f4STakashi IwaiThe audio timestamp accuracy can be returned to user-space, so that 93*20a1d0f4STakashi Iwaiappropriate decisions are made: 94*20a1d0f4STakashi Iwai 95*20a1d0f4STakashi Iwai- for dma time (default), the granularity of the transfers can be 96*20a1d0f4STakashi Iwai inferred from the steps between updates and in turn provide 97*20a1d0f4STakashi Iwai information on how much the application pointer can be rewound 98*20a1d0f4STakashi Iwai safely. 99*20a1d0f4STakashi Iwai 100*20a1d0f4STakashi Iwai- the link time can be used to track long-term drifts between audio 101*20a1d0f4STakashi Iwai and system time using the (tstamp-trigger_tstamp)/audio_tstamp 102*20a1d0f4STakashi Iwai ratio, the precision helps define how much smoothing/low-pass 103*20a1d0f4STakashi Iwai filtering is required. The link time can be either reset on startup 104*20a1d0f4STakashi Iwai or reported as is (the latter being useful to compare progress of 105*20a1d0f4STakashi Iwai different streams - but may require the wallclock to be always 106*20a1d0f4STakashi Iwai running and not wrap-around during idle periods). If supported in 107*20a1d0f4STakashi Iwai hardware, the absolute link time could also be used to define a 108*20a1d0f4STakashi Iwai precise start time (patches WIP) 109*20a1d0f4STakashi Iwai 110*20a1d0f4STakashi Iwai- including the delay in the audio timestamp may 111*20a1d0f4STakashi Iwai counter-intuitively not increase the precision of timestamps, e.g. if a 112*20a1d0f4STakashi Iwai codec includes variable-latency DSP processing or a chain of 113*20a1d0f4STakashi Iwai hardware components the delay is typically not known with precision. 114*20a1d0f4STakashi Iwai 115*20a1d0f4STakashi IwaiThe accuracy is reported in nanosecond units (using an unsigned 32-bit 116*20a1d0f4STakashi Iwaiword), which gives a max precision of 4.29s, more than enough for 117*20a1d0f4STakashi Iwaiaudio applications... 118*20a1d0f4STakashi Iwai 119*20a1d0f4STakashi IwaiDue to the varied nature of timestamping needs, even for a single 120*20a1d0f4STakashi Iwaiapplication, the audio_tstamp_config can be changed dynamically. In 121*20a1d0f4STakashi Iwaithe ``STATUS`` ioctl, the parameters are read-only and do not allow for 122*20a1d0f4STakashi Iwaiany application selection. To work around this limitation without 123*20a1d0f4STakashi Iwaiimpacting legacy applications, a new ``STATUS_EXT`` ioctl is introduced 124*20a1d0f4STakashi Iwaiwith read/write parameters. ALSA-lib will be modified to make use of 125*20a1d0f4STakashi Iwai``STATUS_EXT`` and effectively deprecate ``STATUS``. 126*20a1d0f4STakashi Iwai 127*20a1d0f4STakashi IwaiThe ALSA API only allows for a single audio timestamp to be reported 128*20a1d0f4STakashi Iwaiat a time. This is a conscious design decision, reading the audio 129*20a1d0f4STakashi Iwaitimestamps from hardware registers or from IPC takes time, the more 130*20a1d0f4STakashi Iwaitimestamps are read the more imprecise the combined measurements 131*20a1d0f4STakashi Iwaiare. To avoid any interpretation issues, a single (system, audio) 132*20a1d0f4STakashi Iwaitimestamp is reported. Applications that need different timestamps 133*20a1d0f4STakashi Iwaiwill be required to issue multiple queries and perform an 134*20a1d0f4STakashi Iwaiinterpolation of the results 135*20a1d0f4STakashi Iwai 136*20a1d0f4STakashi IwaiIn some hardware-specific configuration, the system timestamp is 137*20a1d0f4STakashi Iwailatched by a low-level audio subsystem, and the information provided 138*20a1d0f4STakashi Iwaiback to the driver. Due to potential delays in the communication with 139*20a1d0f4STakashi Iwaithe hardware, there is a risk of misalignment with the avail and delay 140*20a1d0f4STakashi Iwaiinformation. To make sure applications are not confused, a 141*20a1d0f4STakashi Iwaidriver_timestamp field is added in the snd_pcm_status structure; this 142*20a1d0f4STakashi Iwaitimestamp shows when the information is put together by the driver 143*20a1d0f4STakashi Iwaibefore returning from the ``STATUS`` and ``STATUS_EXT`` ioctl. in most cases 144*20a1d0f4STakashi Iwaithis driver_timestamp will be identical to the regular system tstamp. 145*20a1d0f4STakashi Iwai 146*20a1d0f4STakashi IwaiExamples of typestamping with HDaudio: 147*20a1d0f4STakashi Iwai 148*20a1d0f4STakashi Iwai1. DMA timestamp, no compensation for DMA+analog delay 149*20a1d0f4STakashi Iwai:: 150*20a1d0f4STakashi Iwai 151*20a1d0f4STakashi Iwai $ ./audio_time -p --ts_type=1 152*20a1d0f4STakashi Iwai playback: systime: 341121338 nsec, audio time 342000000 nsec, systime delta -878662 153*20a1d0f4STakashi Iwai playback: systime: 426236663 nsec, audio time 427187500 nsec, systime delta -950837 154*20a1d0f4STakashi Iwai playback: systime: 597080580 nsec, audio time 598000000 nsec, systime delta -919420 155*20a1d0f4STakashi Iwai playback: systime: 682059782 nsec, audio time 683020833 nsec, systime delta -961051 156*20a1d0f4STakashi Iwai playback: systime: 852896415 nsec, audio time 853854166 nsec, systime delta -957751 157*20a1d0f4STakashi Iwai playback: systime: 937903344 nsec, audio time 938854166 nsec, systime delta -950822 158*20a1d0f4STakashi Iwai 159*20a1d0f4STakashi Iwai2. DMA timestamp, compensation for DMA+analog delay 160*20a1d0f4STakashi Iwai:: 161*20a1d0f4STakashi Iwai 162*20a1d0f4STakashi Iwai $ ./audio_time -p --ts_type=1 -d 163*20a1d0f4STakashi Iwai playback: systime: 341053347 nsec, audio time 341062500 nsec, systime delta -9153 164*20a1d0f4STakashi Iwai playback: systime: 426072447 nsec, audio time 426062500 nsec, systime delta 9947 165*20a1d0f4STakashi Iwai playback: systime: 596899518 nsec, audio time 596895833 nsec, systime delta 3685 166*20a1d0f4STakashi Iwai playback: systime: 681915317 nsec, audio time 681916666 nsec, systime delta -1349 167*20a1d0f4STakashi Iwai playback: systime: 852741306 nsec, audio time 852750000 nsec, systime delta -8694 168*20a1d0f4STakashi Iwai 169*20a1d0f4STakashi Iwai3. link timestamp, compensation for DMA+analog delay 170*20a1d0f4STakashi Iwai:: 171*20a1d0f4STakashi Iwai 172*20a1d0f4STakashi Iwai $ ./audio_time -p --ts_type=2 -d 173*20a1d0f4STakashi Iwai playback: systime: 341060004 nsec, audio time 341062791 nsec, systime delta -2787 174*20a1d0f4STakashi Iwai playback: systime: 426242074 nsec, audio time 426244875 nsec, systime delta -2801 175*20a1d0f4STakashi Iwai playback: systime: 597080992 nsec, audio time 597084583 nsec, systime delta -3591 176*20a1d0f4STakashi Iwai playback: systime: 682084512 nsec, audio time 682088291 nsec, systime delta -3779 177*20a1d0f4STakashi Iwai playback: systime: 852936229 nsec, audio time 852940916 nsec, systime delta -4687 178*20a1d0f4STakashi Iwai playback: systime: 938107562 nsec, audio time 938112708 nsec, systime delta -5146 179*20a1d0f4STakashi Iwai 180*20a1d0f4STakashi IwaiExample 1 shows that the timestamp at the DMA level is close to 1ms 181*20a1d0f4STakashi Iwaiahead of the actual playback time (as a side time this sort of 182*20a1d0f4STakashi Iwaimeasurement can help define rewind safeguards). Compensating for the 183*20a1d0f4STakashi IwaiDMA-link delay in example 2 helps remove the hardware buffering but 184*20a1d0f4STakashi Iwaithe information is still very jittery, with up to one sample of 185*20a1d0f4STakashi Iwaierror. In example 3 where the timestamps are measured with the link 186*20a1d0f4STakashi Iwaiwallclock, the timestamps show a monotonic behavior and a lower 187*20a1d0f4STakashi Iwaidispersion. 188*20a1d0f4STakashi Iwai 189*20a1d0f4STakashi IwaiExample 3 and 4 are with USB audio class. Example 3 shows a high 190*20a1d0f4STakashi Iwaioffset between audio time and system time due to buffering. Example 4 191*20a1d0f4STakashi Iwaishows how compensating for the delay exposes a 1ms accuracy (due to 192*20a1d0f4STakashi Iwaithe use of the frame counter by the driver) 193*20a1d0f4STakashi Iwai 194*20a1d0f4STakashi IwaiExample 3: DMA timestamp, no compensation for delay, delta of ~5ms 195*20a1d0f4STakashi Iwai:: 196*20a1d0f4STakashi Iwai 197*20a1d0f4STakashi Iwai $ ./audio_time -p -Dhw:1 -t1 198*20a1d0f4STakashi Iwai playback: systime: 120174019 nsec, audio time 125000000 nsec, systime delta -4825981 199*20a1d0f4STakashi Iwai playback: systime: 245041136 nsec, audio time 250000000 nsec, systime delta -4958864 200*20a1d0f4STakashi Iwai playback: systime: 370106088 nsec, audio time 375000000 nsec, systime delta -4893912 201*20a1d0f4STakashi Iwai playback: systime: 495040065 nsec, audio time 500000000 nsec, systime delta -4959935 202*20a1d0f4STakashi Iwai playback: systime: 620038179 nsec, audio time 625000000 nsec, systime delta -4961821 203*20a1d0f4STakashi Iwai playback: systime: 745087741 nsec, audio time 750000000 nsec, systime delta -4912259 204*20a1d0f4STakashi Iwai playback: systime: 870037336 nsec, audio time 875000000 nsec, systime delta -4962664 205*20a1d0f4STakashi Iwai 206*20a1d0f4STakashi IwaiExample 4: DMA timestamp, compensation for delay, delay of ~1ms 207*20a1d0f4STakashi Iwai:: 208*20a1d0f4STakashi Iwai 209*20a1d0f4STakashi Iwai $ ./audio_time -p -Dhw:1 -t1 -d 210*20a1d0f4STakashi Iwai playback: systime: 120190520 nsec, audio time 120000000 nsec, systime delta 190520 211*20a1d0f4STakashi Iwai playback: systime: 245036740 nsec, audio time 244000000 nsec, systime delta 1036740 212*20a1d0f4STakashi Iwai playback: systime: 370034081 nsec, audio time 369000000 nsec, systime delta 1034081 213*20a1d0f4STakashi Iwai playback: systime: 495159907 nsec, audio time 494000000 nsec, systime delta 1159907 214*20a1d0f4STakashi Iwai playback: systime: 620098824 nsec, audio time 619000000 nsec, systime delta 1098824 215*20a1d0f4STakashi Iwai playback: systime: 745031847 nsec, audio time 744000000 nsec, systime delta 1031847 216