Not all jsmpeg live-views show up on Android HA client

[Sarah-Connelli]

Checklist:

• [YES] I updated to the latest version available
• [YES] I cleared the cache of my browser

Release with the issue:
frigate-hass-card v2.0.0
frigate integration v2.1.0
homeassistant (official docker) 2021.11.1

Last working release (if known):

Browser and Operating System:
Android HA client (2021.10.0-full)

Description of problem:

On the phone, either via local wifi on same subnet or via ISP.

I have 7 Frigate-hass-card's on a dedicated lovelace page so no other cards. If the live-source is set to just 'frigate', then all cards appear as expected and update accordingly.
However when I change all the cards to 'frigate-jsmpeg', then some cards appear and update accordingly, but others just appear as transparent empty cards with a 'sad smiley' icon in the top left just behind the 'F' menu button on the card. This never happens with live-view 'frigate' cards, they always work, just with jsmpeg-live-view cards.

Now if I choose one of the other options in one of those empty cards, for example 'snapshots/clips', and then choose the 'live' button, suddenly the live feed appears.

This doesn't happen on the desktop, in some cases there may be a delay on the desktop but they always start showing the feeds. However it seems that on the smartphone they time-out (the sad smiley face) I assume and don't do anything from that point on unless you manually intervene and force the live-feed by cycling through another menu option and back via clicking the live-feed menu button in the card.

Javascript errors shown in the web inspector (if applicable):

*Additional information:

*

[dermotduffy]

Hi @Sarah-Connelli,

Thanks for the report. Quick response: Yes, I can reproduce this on the Android HA client, and also Chrome on Android (slightly different behavior but very similar). This could be related to the maximum number of websockets the browser is allowing to be rendered -- each JSMPEG view will consume 1 websocket, and I know the max allowable websockets to a given domain can be fairly small depending on the browser (e.g. 6).

In my test case I had 8 cards, and 8 requests were made to Frigate -- but for some reason the browser only rendered 4 of them (in my case). Your manual intervention trick could get up to 7 to render, but never more than that (perhaps the websocket limit).

Will investigate further.

[dermotduffy]

If I remote debug this in Chrome on Android, I get:

Too many active WebGL contexts. Oldest context will be lost" (WebGLRenderer.js)

... so this appears to be hitting a limit of simultaneous WebGL contexts (in Android webview probably). The underlying JSMPEG library (which the Frigate card needs) uses 1 x WebGL context per card. When we pick a broken card and re-request live, it actually breaks another card on the page when it recycles a WebGL context ("Oldest context will be lost").

I just submitted a PR that allows setting arbitrary JSMPEG flags. If I configure it like this:

live:
  jsmpeg:
    options:
      disableGl: true

All cards will render. However, (at least for me) without WebGL for some cameras I do get mild visual artifacts (and probably performance implications), so this is not a perfect solution. I cannot think of anything better right now, except configuring the card with a different live provider.

[Sarah-Connelli]

Would it be possible to assign just one context, ie an offscreen canvas, and then each card just do a copy from that offscreen canvas (ie x*y, card 1 copies from 0x0, card 2 copies from 640x0?, depending on how many frigate cards there are the offscreen canvas could be initialised to cater for that size). Like how classic 2D platform game sprite engines work.

[dermotduffy]

Very creative idea! Something like that sounds theoretically possible, although unfortunately also sounds like a considerable amount of work most of which would be to the underlying JSMPEG player to get it to render to part of a canvas vs an entire one. Then we'd create X players based on the same underlying canvas and give it coordinates to limit writing/reading to, and then a shim to do the copying from offscreen canvas to 'display' canvas (without a GL context).

But given the amount of work, I doubt it's going to be something I'll be able to prioritize exploring further. Any interest in doing so yourself?

[dermotduffy]

I'm going to close this issue for now with a troubleshooting entry for the disableGl workaround. This is not satisfactory, but I don't see a time-feasible implementation here (for me at least!) without volunteerism from elsewhere to explore alternatives like those described in this bug, as well as securing the buy-in from the owners of the underlying JSMPEG libraries.

[boozelclark]

Sorry to revise a closed topic. I have this issue on Android and noticed that 4 of my Cameras load and then the remaining 3 show this error. However on the remaining 3 Camera's I then click on the frigate icon, select clips (that works) and then go back to the live view and the live view loads while still running the original 4 working streams. I can repeat this on the cameras that didn't initially load and then run all 7 camera's at once and working well.

I may be wrong but this gives the impression that in some way the WebGL context limits are only used up when loading. Is there not a potential option to look at adding a priority and loading the the cameras sequentially or reloading them when this limit is set?