YTM/Google Cast - Ongoing Issues as of 2024-04

[TheWebMachine]

I'm posting this as a Discussion instead of creating Issues as of yet, as I am still trying to sort out what is what with the issues I'm having - trying to obtain logs from within MA's interface gives an "Unauthorized" page that I haven't found a resolution for yet. I'm posting here to crowdsource my efforts and see what we come up with.

Setup Details:

• Versions:
  • HAOS with MA Add-on Container
  • Core 2024.4.3
  • Supervisor 2024.04.0
  • Operating System 12.2
  • MA Server 2.0.0b136
• Hardware:
  • Virtualized HA server, 8 CPU cores and 16GiB RAM
  • Bridged network adapter, iperf3 tested to full 1Gbit speeds with sub-2ms latency to rest of network
  • MA container CPU during normal playback: 0.5-3%
  • MA container RAM during normal playback: 4-6%
• Source: YTM source
• Destination: Google Home/Nest Speakers
• Network: 5GHz WiFi network with speakers getting clear signal (-50dBM or better) with 80-160Mbit connection speed - HA/MA running on wired 1Gbit connection
• Internet: 1Gbit fiber internet connection consistently getting 900+Mbit symmetrical
• The issues noted below have been ongoing since betas in the 80's

Issues experienced:

• Volume Normalization is causing the endless track repeat issue for me, not Flow Mode

  • the last bugfix referencing the issue did not resolve it for me; just tested and confirmed still exists when Normalization is enabled without Flow Mode
  • it plays through the Queue just fine until it randomly gets stuck on a track

• Volume Normalization causes 5-10s delay playing next track with Flow Mode off

  • player view shows the track time start counting and resetting to 0:00 multiple times before audio finally starts playing on the speaker
  • maybe related to endless repeat issue?

• Flow Mode, regardless if Crossfade or Normalization is enabled or not, results in audio quality degradation as time goes on

  • sometimes after just one or two tracks, sometimes after an hour or more
  • sounds like crackling, distortion, and minor cut-outs, most noticeable in the highs and lows (play music with a good beat to really notice it) reminiscent of a bad Bluetooth connection, as if bitrate is being lowered during the stream
  • it eventually becomes quite noticable as it continues to get worse as time goes on
  • it's worth noting that I don't get this audio degradation if I cast directly from YTM on my phone to my speakers or groups
  • this issue didn't improve when moved to the new MA cast player

As it stands right now, the only way I get reliable, quality audio playback is with Normalize, Crossfade, and Flow Mode all turned off.

Let me know what you think and what you need from me to dig deeper into these.

Also, any thoughts on the Unauthorized error when trying to obtain the logs? I have tried from within my network, both within HA > MA interface and directly via a browser without any luck.

Thanks! 🙂

[marcelveldt]

Please update to b137 - also to what value did you set volume normalization ?

[TheWebMachine]

I've tried both default (-17) and -15 (my preference), same results. Is the normalize routine waiting for the whole file to download before allowing playback? Because it seems to adjust leveling on the fly as the song plays, so a full download before allowing playback would seem unnecessary in that case.

[marcelveldt]

Volume normalization is a 2 step process: the first time you play a track it runs in dynamic mode. It will apply a basic version of leveling and at the same time analyzes the audio to measure integrated loudness, true peak etc. At the end of playback it collects a report which we store for next round. Whenever the track is played the next time, the measurement will be applied to the normalization config so it work work even better.

No normalize routine is not waiting for the entire track, that would run people out of memory and playback really stalls.

What is going on with YTM is that they heavily throttle the streaming, so we cant buffer much ahead which makes the whole chain a bit more difficult, especially when features are involved that require a bit of buffer such as volume leveling and crossfading. I did however optimize our streaming process a lot accounting for that after I noticed YTM is so heavily throttling/crippling playback.

So one important question is if the issue also occurs when you playback from another source ?

[TheWebMachine]

I'll have to figure out another source to add to test. I only use YTM.

[marcelveldt]

Please first test what I asked below just now (single speaker test)

[T3chArmy]

I'm also experiencing the crackling on b136, and b137.

[TheWebMachine]

Well, I am not experiencing it on any of the single speaker streams, regardless of generation or add ons to Flow Mode. I often cast to groups (Google Home-based), tho, and I've definitely experienced it there. Does that answer your theory?

I'm going to give these 4 individual streams another hour - total 4.5hrs vanilla Flow Mode, 2.5hrs Flow+Cross & Flow+Normal, then I'm going to perform the same test, but with three different Home groups:
Group 1 - 1 Nest Audio + 1 Nest Mini (both new gen)
Group 2 - 1 Max + 1 Mini (both 1st gen)
Group 3 - my Whole House group that includes 3 Max, 2 Mini (old+new), and 1 Nest Audio

Obviously, I can't test all 3 at once, but I can do Group 1 and 2 at the same time and then Whole House on its own.

[TheWebMachine]

Okay...now we are getting somewhere. With the 1st gen Mini in the mix in Group 2, I am now hearing a distinct audio distortion on that Mini. However, I am not hearing the distortion on the Max in that same group.

Group 1 (Nest Audio + Nest Mini) is clean thus far.

Flow Mode only, no frills.

[TheWebMachine]

More developments: I threw in a Group 4: 2 Max and a Nest Mini. The audio turned to garbage quick, starting with the Nest Mini, then one of the Max went bad soon after. One Max still sounds good.

Anyway, it seems to only be affecting groups, not single speakers. I even sent a single speaker stream to the previous old Mini that had issue in Group 2 and it sounds fine by itself.

I hope that helps!

[marcelveldt]

Thanks for all the testing! Really helpful. Something is indeed off with cast groups somehow. I think it's with 24 bits bit depth, caused by flow stream. I will have another good look.

[TheWebMachine]

No problem! Yeah, I think it's a lowest common denominator issue with groups...the Mini, particularly the 1st gen, has the weakest hardware. It sometimes even takes a couple seconds to start listening after the wake word. If I cast to identical speakers or even the Nest Audio and a Max, it seems fine, but throwing a Mini of either gen into the mix brings the chaos. I suspect they just can't handle the higher depth, but try anyway and then start chopping up after a while.

That said, I couldn't speculate on why not using Flow Mode seems to have no issues, no matter who is in the group. 🤷🏻‍♂️

I'm on the road for the next 2 weeks, but I can resume testing when I get home. 😎 Time for a much needed break!

[marcelveldt]

Please test beta 140/141 - I think I found the cause of this issue and fixed it.

[TheWebMachine]

Hi! I'm back from my trip. Not as relaxing as I'd have liked, but I digress...

Been testing latest (b146, 147, and now 148) over the last few days.

Now that the YTM album issues that appeared in my absence are resolved - I had to blow out the YTM provider again, allow whole DB to clear, then add it again, which corrupted the album DB with ghost entries that wouldn't play, then removed and reinstalled MA all over again to clean things up - but it's all working again now. 🤣

Anyway, I'm still getting audio choppiness with my all-inclusive "Whole House" group in Flow Mode (no CF, normalization on). I have been fussing with the format options you opened up and it seems no format, not even 44.1/16 is immune to it. 😭

[OzGav]

Are any of the speakers connected via a Ethernet? If not you might have a bandwidth problem?

[TheWebMachine]

Nah, we already crossed that bridge. They are all Google Nest/Max/Minis, so no ethernet. My network is no slouch, either. 1Gbit symmetrical fiber internet, distributed Wifi6 APs with segregated 2.4Ghz network and independent channels. I can pull 120Mbit over across 20mhz of the channel from anywhere in the house. Bandwidth is definitely not the issue; the music never runs out of buffer or anything like that, the audio quality simply degrades with crackles and the like. This also happens with as few as 2 speakers as soon as device models become mixed. As noted previously, things start to go sideways as soon as the Minis get tossed into the mix.

We previously thought it a format issue, but that appears to not be the case since we can change up format now and that has had no effect. My next set of tests will be with absolutely no Minis in the groups. It may be coming time to just retire them completely. As I mentioned before, they even appear to struggle with the Assistant these days. They may just not be up to any task anymore, and are causing the whole group to hitch as it struggles to keep up. 🤷🏻‍♂️

As an aside, since I'm a bit disappointed in the lack of options for entirely open HA/Assist/MA/etc compatible speakers, I've decided I'm doing to design (or, more likely, re-design) a kick-ass open source speaker design akin to the now-discontinued Google Max. The audio side of the ESPHome software is finally seeing some attention from the community, but the hardware out there to run a good HA speaker is kinda lacking yet - though I recently saw a promising design for a board that might be more what I'm looking for as a starting point. I'd really like to ditch Google, and not for another company like Sonos, who they constantly fight over patents with, ruining the experience for everyone involved. We need a great open option and I guess I don't have enough hobby projects yet. 🤣💀 I'm hoping to whip up something modular enough that you could rip the guts out of a Max and replace it with this, keeping the shell and speakers intact. We'll see how it goes, tho.

[OzGav]

Sounds great. If you nail down something that can be fixed with your testing then open an issue

[TheWebMachine]

Of course! That's why this has been only a Discussion thus far. 😎

[TheWebMachine]

Well, removing the Minis from the groups had no effect. They aren't the root cause, it would seem. I'm wondering if there is an issue in the Chromecast Group system, itself.

The way I understand it, Cast devices part of a group designate a group leader that acts as the receiver/controller for the group, then that device rebroadcasts the audio stream to the rest of the group members. If you look at when you cast to a group, the stream is sent to an IP of one of the speakers, but on a different port (random 32xxx instead of default cast port 8009).

So, my next round of testing will be regarding the group boradcaster. See if I can trick Google into assigning group leader to speaker of my choosing so I can see if different speakers handle the group functionality differently - "is the issue happening only when certain models are the group leader?"

Also, in an attempt to confirm some things, I decided to give the Universal Groups provider a try...did you guys forget about that plugin along the way? It's all sorts of broken at the moment. 🤣

1. Once created, you can't edit group options. The Save button stays disabled no matter which options in the existing Universal Group you change, so you are stuck with the defaults upon creation

2. You can't play anything to it. Even if you select the group, you are told that no players have been selected when you try to then play a playlist or track.

Just seems like it got left behind at some point through the evolution of the betas.

[OzGav]

Could this have been a factor https://www.googlenestcommunity.com/t5/Speakers-and-Displays/Having-trouble-with-Speaker-groups-Read-this-first/m-p/633127/page/7

[OzGav]

Old Question but noticed it hasn't been marked as answered. Has the groups rewrite resolved this?

[TheWebMachine]

Old Question but noticed it hasn't been marked as answered. Has the groups rewrite resolved this?

Whether resolved by Google's firmware updates and/or the groups rewrite, this appears to no longer be an issue. Thanks for following up!

In the end, I have all my speakers set to use 44.1KHz/16 bits (which you may recall still had problems back in May) since all of the speakers support this baseline sample rate. I haven't tried upping the quality or using the multi-select of sample rates, however. I'll give it another round of tests by adding back some of the higher sample rates to see if the issue presents itself again, just to be sure, before I mark it as Answered.

Could this have been a factor https://www.googlenestcommunity.com/t5/Speakers-and-Displays/Having-trouble-with-Speaker-groups-Read-this-first/m-p/633127/page/7

The caveat with Google's mention of the "group leader" being the source of the issue is the fact that Google doesn't give us the option to choose who a group leader is for any given casting group. It appears to be randomly selected by some magic on Google's side of things.

So, this presents more questions than it answers (for me, at least): If a Mini ends up "elected" leader of a group, does that impact performance of all others in that group at higher sample rates? Is Google smart enough to designate the most powerful speaker in a group the leader or is it purely random? When does this "election" happen; during group creation (permanent) or is it moved around based on speaker availability? Who knows!? I've tried tracing logs and network traffic to make sense of how they handle it, but I've not gotten far. What I do know is if the group leader is taken offline (pull the power), the entire group stops casting audio and the group is unavailable until that leader comes back online. I powered off a leader for a couple hours and never noticed a new leader selected or the group come back online...until I plugged the speaker back in. This implies group leaders are at least "sticky" for a long time, even if offline.

Honestly, I both love and hate my Google speakers, particularly for the group casting features. They seemingly offer the best audio synchronization between speakers of any solution available to us on HA/MA that I've tried so far (I NEVER perceive any audio drift), but being entirely at the whims of Google's questionable dev decisions is annoying, at best.

(I disregard the cloud aspects of Google's speakers because I've permanently disabled the built-in Mic/Google Assistant and use Atom Echos with HA Assist, effectively using the speakers as only speakers via local network casting and API calls; avoiding Google's cloud as much as is possible.)

Anyway, I'll mark this as Answered in a couple days once I've finished one final round of tests playing with the sample rates, but I suspect we are in the clear on this one.