IndexOutOfBoundsException while doing com.google.android.exoplayer2.source.SampleDataQueue.readData(SampleDataQueue.java:309)

[dwc18]

[REQUIRED] Issue description

I'm effectively "reopening" issue #7580 which was closed as a duplicate of #7549 which was fixed on dev-v2 with 7bc5fa8

The exception reported in #7549 is different than I reported in #7580. I understand they could both be caused by the same underlying issue. However I have synced to dev-v2 at 7bc5fa8 and have been able to reproduce my crash.

In reviewing my reproduction steps from #7580 I realize I omitted a potentially critical piece of information. In addition to the code change mentioned, the DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS was reduced from 25sec to 2sec. This was in part because when playing at the live point with the streams we're testing, it's rare to have more than 25sec available, and because reducing the value to a very low level increases the likelihood of discarding and thus allows for better stress-testing of the discard functionality.

I certainly acknowledge that 2sec is likely an unrealistically-low number, especially given that our segment sizes are 6sec. However I was also able to reproduce the exception with an 8sec MIN_DURATION, though it took much longer to reproduce. I have not yet tested higher numbers to see if there's a point beyond which the issue no longer exhibits.

What's not clear to me yet is whether a low level of 8sec (so 2 sec longer than our segment size) is the cause of the crash (i.e., the algorithm should never be expected to handle such a case) or whether it's just making it easier to identify a race condition which should be addressed.

The reproduction steps, environment, etc are all the same as originally reported in #7580 but I'll repeat it here.

2020-07-27 17:57:10.901 31317-31317/com.google.android.exoplayer2.demo E/EventLogger: playerFailed [eventTime=5821.01, mediaPos=559.28, window=0, period=0
com.google.android.exoplayer2.ExoPlaybackException: Unexpected runtime error
at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:493)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:193)
at android.os.HandlerThread.run(HandlerThread.java:65)
Caused by: java.lang.IndexOutOfBoundsException: off=-4762, len=5027 out of bounds (size=65536)
at java.nio.Buffer.checkBounds(Buffer.java:587)
at java.nio.DirectByteBuffer.put(DirectByteBuffer.java:285)
at com.google.android.exoplayer2.source.SampleDataQueue.readData(SampleDataQueue.java:309)
at com.google.android.exoplayer2.source.SampleDataQueue.readToBuffer(SampleDataQueue.java:147)
at com.google.android.exoplayer2.source.SampleQueue.read(SampleQueue.java:346)
at com.google.android.exoplayer2.source.hls.HlsSampleStreamWrapper.readData(HlsSampleStreamWrapper.java:561)
at com.google.android.exoplayer2.source.hls.HlsSampleStream.readData(HlsSampleStream.java:79)
at com.google.android.exoplayer2.BaseRenderer.readSource(BaseRenderer.java:354)
at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.feedInputBuffer(MediaCodecRenderer.java:1286)
at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:835)
at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:815)
at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:404)
at android.os.Handler.dispatchMessage(Handler.java:102) 
at android.os.Looper.loop(Looper.java:193) 
at android.os.HandlerThread.run(HandlerThread.java:65) 
]

[REQUIRED] Reproduction steps

Environment:

client device: TiVo Stream 4k

wireless connection is intentionally set to be close to the threshold for the stream. In this particular case, the stream (a live stream) has 6Mbps, 4Mbps and 2Mbps variants and the access point is limiting the connection to 10Mbps

CODE CHANGE: in part because this stream uses the same resolution, and also because we'd like to eventually have the discard be evaluated for every shift to a higher bitrate (not just higher resolution) I disabled the checks for resolution and only evaluate bitrate when determining whether to discard samples. i.e., in AdaptiveTrackSelection.java:

// If the chunks contain video, discard from the first SD chunk beyond
// minDurationToRetainAfterDiscardUs whose resolution and bitrate are both lower than the ideal
// track.
for (int i = 0; i < queueSize; i++) {
MediaChunk chunk = queue.get(i);
Format format = chunk.trackFormat;
long mediaDurationBeforeThisChunkUs = chunk.startTimeUs - playbackPositionUs;
long playoutDurationBeforeThisChunkUs =
Util.getPlayoutDurationForMediaDuration(mediaDurationBeforeThisChunkUs, playbackSpeed);
if (playoutDurationBeforeThisChunkUs >= minDurationToRetainAfterDiscardUs
&& format.bitrate < idealFormat.bitrate
/* && format.height != Format.NO_VALUE && format.height < 720
&& format.width != Format.NO_VALUE && format.width < 1280
&& format.height < idealFormat.height*/) {

CODE CHANGE: changed:
//public static final int DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS = 25_000;
public static final int DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS = 8_000;

[REQUIRED] Link to test content

At the moment we are unable to make our video feed public. We are working on ways to address this.

[REQUIRED] A full bug report captured from the device

I will capture the bug report and email it.

[REQUIRED] Version of ExoPlayer being used

branch of dev-v2 at: commit 7bc5fa8

[REQUIRED] Device(s) and version(s) of Android being used

Primary device used for reproduction and testing is a Tivo Stream 4K running android:
dwc$ adb shell getprop ro.build.version.release
9
dwc$ adb shell getprop ro.build.version.sdk
28

[tonihei]

I'm investigating. For the record - it can also be reproduced with the following steps:

1. AdaptiveTrackSelection.evaluateQueueSize(): return queue.isEmpty() ? 0 : queue.size() - 1;
2. Play "HLS - Apple master playlist advances (TS)".
3. Wait for less than a minute.

Note that it doesn't reproduce with the "fmp4" sample, nor with DASH or SmoothStreaming.

[tonihei]

Update: The issue is caused by reusing the TsExtractor after discarding media chunks. The extractor ends up in an invalid state where the first sample after discard is allegedly written to an out-of-scope byte range (that doesn't actually happen, it's just left-over metadata from the discarded chunk). When the renderer tries to read this sample with invalid byte range it sometimes runs into the error above, but only when the invalid byte range falls outside the currently reading allocation node of the SampleDataQueue. We will fix that by preventing reuse of the extractor after the discard operation.