Scrubbing slow in Safari in MCO, AVAnnotate

[elynema]

Description

Scrubbing seems to be responding quite slowly particularly in Safari (tested on desktop). I notice this particularly when dragging the scrubber bar; it seems to be faster to just point at a spot in the progress bar and click. In MCO and in Ramp, when dragging the scrubber in Safari, it seems to take 4-7 seconds for the video to re-start, which is a lot. This behavior was noticed by Jon Dunn when he was working on bringing MCO content into AVAnnotate, which is using the Aviary player. When testing in Safari he noticed this slowness in both MCO and AVAnnotate. Note that Aviary seems to use video.js, so that could possibly explain similar behavior across multiple tools.

In addition, in Ramp in Safari, when scrubbing, the player will spin, jump to a frame, spin, and possibly repeat those steps once or twice before actually starting playback again.

Example record in MCO: https://media.dlib.indiana.edu/media_objects/3r075k73g
Record in Ramp: https://ramp.avalonmediasystem.org/?iiif-content=https://media.dlib.indiana.edu/media_objects/3r075k73g/manifest.json
Record in AVAnnotate (using Aviary player): https://elynema.github.io/avalon-77-test/afrique-sur-seine
Record in Aviary IIIF Player: https://iiif.aviaryplatform.com/player?manifest=https%3A%2F%2Fmedia.dlib.indiana.edu%2Fmedia_objects%2F3r075k73g%2Fmanifest.json&tab=Annotations

Done Looks Like

• test scrubbing with these different tools in Safari and Chrome and write down what is found
• if we can identify a persistent slowness in Safari, investigate errors in console to see if can identify what the issue might be
• Goal would be 1-3 seconds response time when scrubbing (max)

QA

• test scrubbing in iOS native player, as well as Safari on desktop

[joncameron]

Could be related to Video.js; may be due to HLS + Video.js.

[Dananji]

I tested the same MCO item in the following viewers in Safari;

• Ramp (uses Video.js):

  • On MCO:
    • Click on a timepoint -> slight time lag when updating the frame
    • Dragging the scrubber -> updated the frame right away but then it jumps back to the previous timepoint (where the scrubber was when dragging started) and dragging is stopped and jumps to the new timepoint (where the dragging ended)
  • On demo site (has Video.js latest version and the Video.js re-setup work, which are not in MCO):
    • Click on a timepoint -> skips to the previous timepoint and then jumps back to the new timepoint
    • Dragging the scrubber -> updated the frame right away but then it jumps back to the previous timepoint (where the scrubber was when dragging started) and dragging is stopped and jumps to the new timepoint (where the dragging ended)

• Aviary (uses Video.js):

  • Click on a timepoint -> slight time lag when updating the frame
  • Dragging the scrubber -> experienced described behavior in the ticket description (more delayed updates than selecting a timepoint)

• Clover (uses HTML video/audio element with HLS.js): did not experience the described behavior with selecting a timepoint and dragging the scrubber. When trying to scrub again after the item is kept open in the tab for some time while testing on other viewers, the following request kept failing infinitely since HLS kept re-trying to fetch the failed frame;
  

• UV (uses mediaelement.js): did not experience the described behavior with selecting a timepoint and dragging the scrubber. Shows the spinner without revealing the player until a couple of HLS segments are downloaded.

In all these viewers there were streaming server requests that appeared to be failing (just after a page reload) in the network tab in dev tools. Some of these failed requests have the status code 200, while some requests have no status code;

Need to test an item from avalon-dev to see whether this delay is related to the streaming server.

[Dananji]

I tested an item from demo site and observed the same behavior for both Ramp and Aviary for scrubbing actions (selecting a timepoint and dragging the scrubber), as it was seen for the item from MCO.

[elynema]

Clover issue seems like a bug in Clover; HLS is set up to re-try endlessly in their code. But it could also be our streaming server, either slow or session expiration issues.

Dananji is blocking player in Ramp until at least a couple of HLS segments are loaded.

Overall, Dananji thinks this is a Safari issue where player events get dropped / events don't register as quickly, not really a video.js issue. We would expect to continue to see issues in Aviary with video.js in Safari after this fix.

[elynema]

Dananji hasn't tested on iPhone yet, but would not expect changes made here for Safari to impact the native player handling.

[elynema]

@Dananji @joncameron I just tested the record listed in this issue in the Ramp demo site in Safari on my Mac. It looks like clicking in the progress bar to seek is working ok. But dragging the progress bar to seek is still very slow, and it flips through a number of frames before landing on the one to play.

Private Zenhub Video

@joncameron Would you be willing to give this a try on your machine to see if you get similar results?

[elynema]

@Dananji Can you re-test this in Safari and see if you are seeing the same issues I was? I'll move it back to In Progress.

[Dananji]

Yes, I'm seeing the frames in between in the video.
But now, the time indicator (scrubber) in the progress bar is not sticking to those frames while the player updates as before. I'm not sure I know how to resolve the frame updating issue in Video.js.
To see whether this is related to the custom progress bar we've implemented in Ramp; I replaced the progress bar with Video.js' native progressControl element. And it still shows the frames in between when dragging the scrubber.

[elynema]

Testing this today in Safari on desktop with the Lunchroom Manners record on avalon-dev and the Afrique example on linked in the issue on Ramp demo site, and scrubbing worked fine. We did see some in-between frames as Dananji noted above, but we were within the 1-3 second scrubbing window as laid out in this ticket. I'm going to close this for now; native player needs more testing.