hls.js stalls shortly on audio buffer gap

[ghouet]

What version of Hls.js are you using?

v1.5.3,v1.4.14,v1.3.5

What browser (including version) are you using?

Chrome Version 121.0.6167.85 (Official Build) (x86_64)

What OS (including version) are you using?

Mac OS Sonoma 14.3

Test stream

https://storage.googleapis.com/trumantest-public/audio_gap/index.m3u8

Configuration

{
  "debug": true,
  "enableWorker": true,
  "lowLatencyMode": true,
  "backBufferLength": 90
}

Additional player setup steps

No response

Checklist

• The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
• The issue occurs in the stable client (latest release) on https://hlsjs.video-dev.org/demo and not just on my page
• The issue occurs in the latest client (main branch) on https://hlsjs-dev.video-dev.org/demo and not just on my page
• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

1. Got to https://hlsjs.video-dev.org/demo/?src=https://storage.googleapis.com/trumantest-public/audio_gap/index.m3u8
2. Wait for the end of the ad and notice the loading spinner before returning to content

Expected behaviour

No loading spinner before returning to content, smooth playback.
Other players like safari shows no issue.

What actually happened?

Ad the end of the ad before returning to content, the player shows a loading spinner.
Console shows the folowing error:
type: 'mediaError', details: 'bufferStalledError', fatal: false, error: Error: Playback stalling at @31.719066 due to low buffer ({"len":0.05159999999999698,"start":0.12,"…, buffer: 0.05159999999999698, …}
Then bufferNudgeOnStall makes the player resume playing normally.

A gap in audio content shouldn't cause a bufferStalledError, in fact the next media content is there it's just that there is no audio to play for a brief amount of time (<0.07s) before playing the next segment.

Console output

bufferNudgeOnStall', error: Error: Nudging 'currentTime' from 31.719066 to 31.819066000000003
    at GapController._tryNudgeBuf…, fatal: false, errorAction: {…}}details: "bufferNudgeOnStall"error: Error: Nudging 'currentTime' from 31.719066 to 31.819066000000003
    at GapController._tryNudgeBuffer (https://hlsjs.video-dev.org/dist/hls.js:27158:21)
    at GapController._tryFixBufferStall (https://hlsjs.video-dev.org/dist/hls.js:27039:14)
    at GapController.poll (https://hlsjs.video-dev.org/dist/hls.js:27000:12)
    at StreamController.checkBuffer (https://hlsjs.video-dev.org/dist/hls.js:27877:23)
    at StreamController.onTickEnd (https://hlsjs.video-dev.org/dist/hls.js:27326:12)
    at StreamController.doTick (https://hlsjs.video-dev.org/dist/hls.js:27322:12)
    at StreamController.tick (https://hlsjs.video-dev.org/dist/hls.js:7870:14)errorAction: {action: 0, flags: 0}fatal: falsetype: "mediaError"[[Prototype]]: Object
(anonymous) @ main.js:734
emit @ index.js:203
emit @ hls.ts:310
trigger @ hls.ts:318
_tryNudgeBuffer @ gap-controller.ts:353
_tryFixBufferStall @ gap-controller.ts:214
poll @ gap-controller.ts:168
checkBuffer @ stream-controller.ts:945
onTickEnd @ stream-controller.ts:217
doTick @ stream-controller.ts:212
tick @ task-loop.ts:106
setInterval (async)
setInterval @ task-loop.ts:68
startLoad @ stream-controller.ts:127
(anonymous) @ hls.ts:434
startLoad @ hls.ts:433
filterAndSortMediaOptions @ level-controller.ts:376
onManifestLoaded @ level-controller.ts:200
emit @ index.js:203
emit @ hls.ts:310
trigger @ hls.ts:318
handleMasterPlaylist @ playlist-loader.ts:429
onSuccess @ playlist-loader.ts:327
readystatechange @ xhr-loader.ts:239
XMLHttpRequest.send (async)
openAndSendXhr @ xhr-loader.ts:166
loadInternal @ xhr-loader.ts:125
load @ xhr-loader.ts:84
load @ playlist-loader.ts:352
onManifestLoading @ playlist-loader.ts:154
emit @ index.js:203
emit @ hls.ts:310
trigger @ hls.ts:318
loadSource @ hls.ts:420
loadSelectedStream @ main.js:372
(anonymous) @ main.js:130
dispatch @ jquery.min.js:3
r.handle @ jquery.min.js:3
Show 3 more frames
Show less
base-stream-controller.ts:241 [log] > [stream-controller]: media seeking to 31.819, state: ENDED
base-stream-controller.ts:1639 [log] > [stream-controller]: Reset loading state
base-stream-controller.ts:1781 [log] > [stream-controller]: ENDED->IDLE
base-stream-controller.ts:241 [log] > [audio-stream-controller]: media seeking to 31.819, state: ENDED
base-stream-controller.ts:1639 [log] > [audio-stream-controller]: Reset loading state
base-stream-controller.ts:1781 [log] > [audio-stream-controller]: ENDED->IDLE
base-stream-controller.ts:241 [log] > [subtitle-stream-controller]: media seeking to 31.819, state: IDLE
buffer-controller.ts:632 [log] > [buffer-controller] Queueing mediaSource.endOfStream()
base-stream-controller.ts:1781 [log] > [stream-controller]: IDLE->ENDED
buffer-controller.ts:643 [log] > [buffer-controller] Could not call mediaSource.endOfStream(). mediaSource.readyState: ended
buffer-controller.ts:632 [log] > [buffer-controller] Queueing mediaSource.endOfStream()
base-stream-controller.ts:1781 [log] > [audio-stream-controller]: IDLE->ENDED
buffer-controller.ts:643 [log] > [buffer-controller] Could not call mediaSource.endOfStream(). mediaSource.readyState: ended
stream-controller.ts:555 [log] > [stream-controller]: Media seeked to 31.819

Chrome media internals output

No response

[robwalch]

There may be enhancements to gap jumping that seek pre-emptively (before or stall). That is needed more for video gaps where, in Chrome, the stall occurs after playing through the gap. With audio, the stall happens at or before the gap.

The other possible enhancement is to fill audio gaps at discontinuities with silent audio. Eliminating the gaps in your media would eliminate the stall.

[robwalch]

A gap in audio content shouldn't cause a bufferStalledError, in fact the next media content is there it's just that there is no audio to play for a brief amount of time (<0.07s) before playing the next segment.

That is browser behavior, not HLS.js. HLS.js is recognizing that playback in the browser stopped advancing with a gap, and followed up with a seek "nudge" to restore playback.

Marking with "Works as expected" and "Stream Issue" labels because the stall is only temporary (resolved by the player) and it is a result of jagged start or end times between segments (AV start and end do not align).

Removing "Bug" and adding "Enhancement" as there are things hls.js could do to fill this type or gap, or seek over it when playback doesn't advance smoothly over it.

[ghouet]

Thanks @robwalch, just to be certain: when you say AV start and end do not align; isn't that a legit use case? Why do audio and video PTS have to align? Can't the video stream start with silence followed by the audio stream at a later PTS?

[ghouet]

Note that the same stream actually plays smoothly with shaka player in the same browser despite the gap:
https://shaka-player-demo.appspot.com/demo/#audiolang=en-US;textlang=en-US;uilang=en-US;asset=https://storage.googleapis.com/trumantest-public/audio_gap/index.m3u8;panel=CUSTOM%20CONTENT;build=uncompiled

AVPlayer in safari also plays it smoothly.

Shouldn't hls.js behave the same way as other video players?

[robwalch]

Shaka-player logs that it is seeking over the gap:

Jumping forward 0.39730300000000085 seconds because of gap starting at 31.766 and ending at 31.829999

An enhancement to seek proactively over gaps has not been implemented in HLS.js. See related issue #5631. It relates more to video buffer gaps which stall differently depending on audio availability. In either case, HLS.js needs to seek over gaps in SourceBuffers that would otherwise result in a stall.