Occasional playback stuck and rebuffering for live DASH stream with non-uniform segment lengths

[albertcsm]

Issue description

I tried to play a live DASH stream with multiple periods with ExoPlayer demo player. Usually playback stuck after one or several minutes. The player entered rebuffering state and playback resumed after waiting for tens of second (which match with MPD's timeShiftBufferDepth).

There was no error/warning log when the issue happen. If I set a breakpoint in doSomeWork() of ExoPlayerImplInternal.java, I could find that the player is in Player.STATE_BUFFERING.

The issue can only be reproduced with live stream, there was no issue if I make it static. I tried increasing suggestedPresentationDelay and decreasing minimumUpdatePeriod, the issue was then less reproducible but still occur after one or several hours.

The DASH stream was generated with a proprietary encoder. It has non-uniform segment lengths of 2-5 seconds due to SCTE35 signaling needs, and therefore periods are also short and has non-uniform duration. It may be the reason for triggering playback stuck and rebuffering. There was no playback issue with Shaka player.

Reproduction steps

Add the URL of the DASH stream to media.exolist.json of demo player, then play it for several minutes.

Link to test content

A snapshot of the manifest when the issue happen:
exoplayer-rebuffer.mpd.txt

Version of ExoPlayer being used

Latest commit (ab6f9ae)

Device(s) and version(s) of Android being used

Android studio (Nexus 5X API 24)

[ojw28]

Please could you try with the dev-v2 branch? We've fixed a problem there that can cause stuck behavior for multi-period DASH streams in that branch.

[ojw28]

Marking duplicate of #1874. Let us know if you manage to reproduce.

[albertcsm]

The playback stuck is resolved in dev-v2.
I still see momentary rebuffering pause of about 0.5s. I will open another issue for this, thanks.

[ojw28]

That's likely to be an issue with your content (e.g. uneven end times of the audio and video representations at the end of each period). Which is the underlying cause of the player getting stuck that we've resolved in dev-v2. Please could you take a look at this, and make sure all your representations are of exactly the same duration (and actually contain that duration of media) when each period ends?

[albertcsm]

Yes, the endings of audio and video may have difference of about 1/29s. As far as I understand, such difference is due to the different sampling rates in audio and video streams, and so it is inevitable.

I found the DASH-IF-IOP-v4.0 spec (4.3.4.3. Multi-Period Content) mentioned that: there may be small overlaps or gaps with the Representation at the end of the preceding Period and the playout should be adjusted at the end of each Period to provide a continuous presentation without adding drift between the time documented in the MPD and the actual playout.

I wonder if it it possible to continue playing the content in the preceeding period beyond period boundary when the adaptation set is period-connected, or adjust playhead position at period boundaries if this doesn't make the situation worse.

[ojw28]

I wouldn't expect such a small discrepancy to cause a pause of 0.5s, so if they're really that close then I don't think that's the issue. Are you playing protected content (in which case the pause is probably due to a license round trip time)?

[albertcsm]

No, the content is clear.

0.5 is a rough estimation of the pause time, I think it is actually the time waiting for an explicit manifest update and downloading the latest segments at the time of rebuffering.

From my observation, the effect of the small discrepancy is that the playhead drift towards live edge gradually. For example, I tried playback with a large suggestedPresentationDelay, the playhead was initially far from live edge and there was no rebuffering at all, but I saw it approach the live edge gradually during playback, and finally reached the live edge and triggered rebuffering after tens of minutes. Since then, the rebuffering occured much more frequently at about once per 10 seconds because playhead is so close to live edge.

When you mentioned uneven end times of the audio and video representations, I think there is some sort of trimming of segment content by period boundary, etc. that cause the drift. Please correct me if wrong.

[ojw28]

We may drift forward as described, but I think we'd only do that if the content delivered is shorter than the actual elapsed realtime, and that we'd only drift by the size of that discrepancy. If you're sending less content than the time that's elapsing, it seems inevitable that we'd drift forward (or alternatively, insert little pauses at each period boundary).

I would also not expect frequent re-buffering when the playback gets too close to the live edge, since the act of re-buffering itself should push you back away from the live edge again (unless there's then another period boundary that gets you closer to the live edge once more).

[albertcsm]

The lengths of media content actually equal to the elapsed time, but the segment boundary in audio stream lag behind the the period boundary.

[ojw28]

Does this diagram imply the content for these periods is coming out of single encoder (that's not being reset)? I think that normally, period boundaries would occur because you're transitioning to different content (which would probably be encoded through a different encoder). If that's not the case here, why insert new periods at all? If it is the case, I'd at least expect the audio/video at the start of each period to be properly aligned.

[albertcsm]

Yes, it is from a single encoder. Periods are insert for ad-insertion opportunities, which may eventually be replaced with other content or played as-is.

[ojw28]

It would be good to better understand this use case. In particular, I'm assuming the replacement would happen on the serving side, with you possibly generating manifests with different replacement periods for different users, depending on whether ads are targeted to certain users or not. If that's the case, why bother inserting new periods if replacement doesn't actually occur? Or to put it another way, why is an unfulfilled replacement opportunity being exposed all the way to the client? That seems like unnecessary additional complexity for the client to be handling (and additional opportunity for the client to mess something up).

If you moved to a model where you start a new period only when a replacement opportunity is actually filled, you'd at least have the start of each ad period such that audio and video are properly aligned with one another, which certainly wouldn't make anything worse. To what extent it would improve things I'm not sure, but it'll likely make it a bit easier to get things working properly.

[albertcsm]

For my case, encoding and ad-insertion are currently run by different vendors. The encoding side provides multi-period ad-free manifest, the ad-insertion vendor rewrites manifest for ads. So the encoding side does not know whether replacement will actually take place.

[ojw28]

It feels like things would work better if the encoding side provided a single period manifest with some other form of metadata indicating where the ad opportunities are, since the media they're providing really is just a single continuous stream. The encoding side inserting multiple periods is quite artificial. Either that or there would be some way of the ad-insertion vendor merging periods if they don't actually make a replacement.