A question about TimestampMaster-s

[sruditsky]

To handle discontinuities in an HLS stream ExoPlayer has a notion of TimestampMaster which is a stream sample wrapper able to initialize TimestampAdjuster. ExoPlayer allows having several TimestampMaster-s (e.g. one controlled by a video playlist and another one by an audio playlist) at the same time.

This works assuming that the playlists used by all the TimestampMaster-s have discontinuities at exactly the same time. However, if the discontinuities are happening at (even slightly) different times the result is that the timestamps can be reset at a discontinuity to a lower value than one already supplied in an input to the decoders.
We have some HLS stream sources which create playlists with discontinuities happening at slightly different times for audio and video (and as a result timestamps jumping back at discontinuities), and some decoders do not like this.

The questions are:

• What would be the reason to have more than one TimestampMaster ? Is this only to improve the chances of timestamps being updated promptly or there are other reasons?

• Is there any way to configure ExoPlayer to use a single TimestampMaster?

• If not, would replacing audioVideoSampleStreamWrapperCount with 1 in this line:
  

  media/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsMediaPeriod.java

  Line 366 in 3f36610

  sampleStreamWrapper.setIsPrimaryTimestampSource(i < audioVideoSampleStreamWrapperCount);

  
  achieve this? Do you see any problem with such a modification?

[stevemayhew]

@sruditsky I've been looking at this for years now, there are no good solutions, this is sadly just a shortcoming of HLS (remember D-muxed audio was an after thought for HLS from the beginning). We ran into hangs where playback stopped when there are SCTE-35 markers in the stream and a single (video) master was forced, the change to allow multiple possible timestamp masters fixes that bug, but introduces other problems, see this ExoPlayer Issue 8952.

What would be the reason to have more than one TimestampMaster ? Is this only to improve the chances of timestamps being updated promptly or there are other reasons?

You can see the above issue for details, but the bottom line is if EXT-X-DISCONTINUITY occurs out of sync between two playlist then wait for a single stream to provide the timestamp reference (init the TimestampAdjuster) can deadlock

Is there any way to configure ExoPlayer to use a single TimestampMaster?

This is the way it was before, Video was always master. And it lead to the aforementioned issue.

[sruditsky]

Thank you @stevemayhew !

So, in other words, with the change which I mention above (i.e. with a single TimestampAdjuster) if all the EXT-X-DISCONTINUITY lines in the playlist which has the TimestampAdjuster occur before the EXT-X-DISCONTINUITY lines in the other playlists then the playback is likely to be OK.
However, if the EXT-X-DISCONTINUITY are shifted in the opposite direction then it is likely to cause a deadlock.

Is this a correct understanding?

[stevemayhew]

This is a real problem with the way HLS is specified, in DASH such changes are not allowed within a Period, so this forces any changes to the presentationTimeOffset to occur across all representations at once. Sadly HLS does not clearly specify how this should be done so ExoPlayer has the TimestampAdjuster to loosely map HLS to DASH semantics. Where it has not worked so well is when HLS authoring takes broad liberties in how what is effectively a Period change is coded.

So, in other words, with the change which I mention above (i.e. with a single TimestampAdjuster) if all the EXT-X-DISCONTINUITY lines in the playlist which has the TimestampAdjuster occur before the EXT-X-DISCONTINUITY lines in the other playlists then the playback is likely to be OK. However, if the EXT-X-DISCONTINUITY are shifted in the opposite direction then it is likely to cause a deadlock.

Is this a correct understanding?

I would have to see the specific example to be sure, also there is a dynamic component of how much buffer you have when the discontinuity is reached.

Each Segment (HlsMediaChunk) is passed a TimestampAdjuster matching the segment's current discontinuity sequence number (DSN), if that adjuster is newly created (the DSN incremented because of an EXT-X-DISCONTINUITY tag) then loading of samples from that segment must be delayed until all the other representations hit that same DSN (at this point one can initialize the offset and determine the correct new "presentationTimeOffset"). All sample streams in the current track selection must advance before the buffered level advances, so depending on the gap until this is true there can be a condition where loading is requested (but not possible) and the buffered level is too low for playback.