Philips Hue bulb firmware OTA update "successful" but firmware is still the old one

[LeoCal]

What happened

I tried to update the firmware of my Philips Hue bulb A60 E27/B22, which is currently running firmware version 1.29.0_r21169 (20170908). Firmware upgrade seems to progress smoothly and it reaches 100% after ~20minutes. I also see the "device_announced" message, the "update_succeeded" message, but the reported firmware version is still the old one.
I've tried both on the latest stable version of zigbee2mqtt HA add-on (1.21.1-1) and on the latest dev version (commit ac44a50 on GitHub), but I got the same result.

What did you expect to happen

I expected to see Philips Hue bulb updated to the newer version.

How to reproduce it (minimal and precise)

I simply followed the Zigbee2mqtt OTA update procedure explained in the main wiki.
I've reported the exact steps I followed here.

Debug info

Zigbee2MQTT version: 1.21.1-1
Adapter hardware: CC2531
Adapter firmware version: CC2531ZNP-Prod

[davidjb]

Experiencing the same issue with the same firmware version; model for me is https://www.zigbee2mqtt.io/devices/9290012573A.html#philips-9290012573a. I've confirmed the newer firmware is successfully downloaded and verified, then OTA happens like @LeoCal describes but z2m reports that the firmware was "upgraded" from the current version to the same version.

I'm wondering if this might be similar to #8719 in terms of a solution - will try upgrading via the previous firmware since that worked for me previously on other Hue models.

Edit: yes, upgrading to 20191218 (1.50.2_r30933) first via https://github.com/Koenkk/zigbee-OTA/blob/4af6b2a99517f57c0111566a9735b40ecdbc2af7/index.json allowed the latest firmware upgrade to succeed. So the solution is the same as #8719 with a stepped upgrade process; it'd be interesting to see if the official Hue process is the same or if they have some extra process that allows direct upgrades.

[MattWestb]

I have reading that the Philips HUE HUB is doing the update in steps if its one very old firmware version but i cant remember where i have reading that but i think its the right way to going first one step higher and rebooting the light and then rolling the newer one and its working.

[LeoCal]

Thanks @davidjb and @MattWestb for your input on this, much appreciated!
I'll give it a try as soon as I'm at home and have some spare time.

[LeoCal]

Hi @davidjb,

can I ask you how you get the mapping between your exact Philips Hue bulb model and the firmware inside the index.json file?
Also, once you have the mapping, what is the procedure to point zigbee2mqtt to the specific line in the index.json to execute the upgrade with it? Any pointer to wiki/ReadMe I might have missed?
Sorry for the basic questions, but it's the first time I use the OTA functionality to upgrade the firmware.

Thanks,
Leo

[LeoCal]

Ok, I might have found a way.
I found out that the debug wiki of zigbee2mqtt explains how to enable debugs for your installation. Debugs should report the firmware being used during an upgrade I guess. That way, one can discover the firmware name/type being used and can then find an older version with a similar name in the index.json.
I'm gonna try that out as soon as I have some spare time and will report my findings here so that other people can benefit from it.
Cheers

[davidjb]

@LeoCal Because of how zigbee-herdsman-converters works, you can easily point your install of zigbee2mqtt at an older version of the complete firmware library. In short, change this line (https://github.com/Koenkk/zigbee-herdsman-converters/blob/master/lib/ota/zigbeeOTA.js#L1):

const url = 'https://raw.githubusercontent.com/Koenkk/zigbee-OTA/master/index.json';

to point to an older commit of zigbee-OTA. For me, I just picked the previous commit, before the Hue firmwares were modified, meaning the first line of zigbee-herdsman-converters's zigbeeOTA.js temporarily becomes:

const url = 'https://raw.githubusercontent.com/Koenkk/zigbee-OTA/4af6b2a99517f57c0111566a9735b40ecdbc2af7/index.json';

At that point, restart zigbee2mqtt and you'll be now upgrading all Philips Hue firmwares to their next-to-last versions. When done, restore zigbeeOTA.js to its original state and restart zigbee2mqtt again. Down the road, the history of index.json shows you the commits available in case further gymnastics are needed between firmware versions or with specific devices.

Enabling debug logging is a good idea too - it helps monitor internal details and progress during the upgrade & you can confirm the right version of zigbee-OTA is being used by the specs of firmware being downloaded.

[liuk4friends]

Hi @davidjb,
I am in the same situation as @LeoCal.
I understood the solution you proposed.
Probably the link that I have to replace to be able to update the firmware of my Philips light bulb [model LWB010 which is still at firmware version 1.29.1_r24931 (20180727)] is exactly what you proposed.

My problem is that I don't know where to find the zigbee-ota.js file to replace the link pointing to the previous firmware.

I am on HassOS 2021.9 on a Rpi 4b.
Inside my zigbee2mqtt folder I can see the files below:

• configuration.yaml
• coordinator_backup.json
• database.db
• device.yaml
• groups.yaml
• state.json
• log (sub-folder)

Probably mine is a stupid question because I think you were referring to the source files.
Simply put, I understand what I should do but I don't know how to do it.

Thank you

[LeoCal]

Thanks @davidjb, that worked for me!
I managed to upgrade to the latest version going through an intermediate version 20170908->20191218->20210331.

Hi @liuk4friends, you can find that file at the following location on your instance:
/app/node_modules/zigbee-herdsman-converters/lib/ota/zigbeeOTA.js

In the first line of that file, you can apply the changes suggested by David.
Of course, you need to apply the changes in a way that persists after container restarts. There are several ways to do that (mounting volumes, forking the image to make your own version of the container, etc.).
However, given the temporary nature of the changes, the easiest way is probably to use Portainer - here's the procedure to follow:
https://github.com/zigbee2mqtt/hassio-zigbee2mqtt/blob/master/CONTRIBUTING.md

Cheers,
Leo

[liuk4friends]

Unfortunately, I can't use that procedure.
My Home Assistant installation is HassOS on Raspberry Pi 4b.

@davidjb, May you help me? Any suggestions that are helpful in my situation?

@Koenkk, Do you think it is possible to modify the OTA procedure of zigbee2mqtt in order to choose the firmware to install? Beyond my problem, it could be an added value in different scenarios ...

thanks in advance

[davidjb]

@liuk4friends I imagine you're using https://github.com/zigbee2mqtt/hassio-zigbee2mqtt as an add-on in Home Assistant, which means your z2m installation is inside a Docker container. This makes things interesting since Docker containers' files are designed to be immutable (with the exception of config files on volumes). I don't use this environment myself but I see you having two options:

• Directly on your Pi; modified Docker container:

  1. Stop the z2m add-on container (in the HA UI or via SSH)
  2. Run a new version of this add-on's container and console into it (e.g. docker run -it name-of-container bash)
  3. Make the changes to zigbeeOTA.js (z2m appears to be installed in /app given the Dockerfile); you'll probably to have to install vi or another editor in the container
  4. Inside the container, run npm start in /app to boot z2m.
  5. While this is running, perform your OTA update
  6. When done, exit the container (this kills it) and start up your add-on again

• Create a second temporary Zigbee network; on a second computer or device:

  1. Install z2m + an MQTT broker like mosquitto on a second computer
  2. Borrow your zigbee dongle from your Pi and plug it in
  3. Configure z2m accordingly (with separate network details to your primary network)
  4. Make the URL changes to zigbeeOTA.js
  5. Start z2m, join your bulbs to this temporary network and do the first OTA upgrade
  6. Stop z2m, revert changes to zigbeeOTA.js
  7. Start z2m again, do the 2nd OTA upgrade
  8. Stop z2m - keep this install around if you want to upgrade more bulbs later
  9. Reattach your dongle to your Pi, restart the z2m HA Add-on and re-join your bulbs into your primary network

Hope that helps.

[liuk4friends]

Thanks @davidjb for the valuable advice.
Unfortunately, I don't think I'm up to it.

Regarding the first hypothesis, I verified that I can disable the protection system of the "SSH Terminal" add-on and then access the docker commands.
But having no skills, the risk of doing damage is very high. The warning message when trying to launch docker inside the terminal is crystal clear (and terrifying)

The second hypothesis I have to discard it because I don't have other hardware where to install z2m and mosquitto.
I could try to install docker on my MacBook Pro but as I said I have neither expertise nor experience.

Honestly I was hoping for something simpler.
I can only rely on @Koenkk 's availability, if he intends to evaluate the possibility of making the OTA procedure more configurable

[davidjb]

No worries @liuk4friends. Fwiw, installing z2m locally on a Mac directly is possible (e.g. without Docker); you can install and run with npm and brew install mosquitto to get an MQTT broker set up.

Otherwise, if you’ve got/get a spare MicroSD card for your Pi, you could flash Raspian and use that as a temporary environment, using the official instructions from https://www.zigbee2mqtt.io/getting_started/running_zigbee2mqtt.html to directly install.

[liuk4friends]

Thanks @davidjb you gave me hope.
I remembered having an old raspberry pi 2 (or 3 can't remember) in a drawer.
I got it back, updated raspbian, installed core mosquitto and finally zigbee2mqtt.

And here I have the first problem.
The instructions on the official website of zigbee2mqtt make me install the latest master version, 1.21.1.
This version unfortunately has incompatibilities with my coordinator device (CC2530). @Koenkk knows about it.
As a matter of fact it doesn't work.

How do I install version 1.20.0 which works perfectly instead?

Thanks again

[davidjb]

@liuk4friends installing an older version is a case of switching versions in git. In the installation instructions, just before you run npm ci checkout a specific release tag with git checkout 1.20.0 where the latter argument is a version number that you know works or want to try (from https://github.com/Koenkk/zigbee2mqtt/releases). If you need to switch versions later, then checkout another version and re-run npm ci to correctly install the dependencies for that version

[liuk4friends]

Hi @davidjb
I did it about an hour ago!!!
I now have fully functional zigbee2mqtt 1.20.0.
The light bulb has been recognized and I edited the zigbeeOTA.js file with the link you suggested a few days ago.
Unfortunately, that link probably points to too recent firmware versions and I can't update the light bulb.

@LeoCal managed to update his by proceeding first with the 20191218 version and then with the 20210331.

Can you tell me how I can retrieve the links of these intermediate versions?

[davidjb]

@liuk4friends Excellent, glad you've got a working z2m. The previous commit in zigbee-OTA has the older Philips Hue firmware you're looking for, which is the instructions mentioned in my comment above. If you've edited zigbeeOTA.js, make sure you restart z2m so the change get picked up. If you're still having issues, try enabling debug logging and look through what's occurring with the OTA process - hopefully that reveals what's happening.

[liuk4friends]

You @davidjb were absolutely right!
I had probably forgotten to restart z2m after editing the zigbee-OTA.js file.

I just managed to update the second bulb to firmware 1.50.2_r30933.
Now I'm updating another one.

As soon as I finish with the second one, I will edit the zigbee-OTA.js file again by re-inserting the link to the latest master version of the library.
So I should be able to bring these two to the latest version as well (1.88 I think).

I thank both you @davidjb and @LeoCal for getting me on the right path to solve this problem!
I really appreciated it!

edit:

Mission accomplished!
All my Philips Hue bulbs are updated to last 1.88.1 firmware!

[OttoL2]

Hi all, noob here so please excuse any stupid questions upfront. My setup is a HA supervised installation on Debian, Z2M installed under HA. I have 2 Hue White Ambiance E14 bulbs, that I bought second hand. Both work fine under Z2M. I tried to update the firmware via OTA. 1 was updated immediately, the other won't no matter what I try. I found this thread today outlining this specific issue. Turns out the non-updating bulb has some 2018 firmware. So I updated my /opt/zigbee2mqtt/node_modules/zigbee-herdsman-converters/lib/ota/zigbeeOTA.js as instructed, to point to an older respository and ran the update again. No luck. I tried an even older repository, no joy. The update runs completely but then the bulb just shows the old firmware again.

Any ideas what I might still try? This is no big thing for how I'm using the bulb, I don't need (I think) any newly introduced firmware options, but I just wonder why I can't get this to work. I can't find a log file that shows anything about the update... any pointers there?

Many thanks in advance!
Maarten

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

[davidjb]

@OttoL2 You might want to ensure you've restarted zigbee2mqtt to ensure your changes to zigbeeOTA.js are being picked up (and check that that installation in /opt/zigbee2mqtt is actively running etc). I don't run supervised HA but there might be more than one install. If you're still not having any luck, you can try ramping up the logging to debug levels to ensure the right firmware is being downloaded (which will also indicate your changes to zigbeeOTA.js are in effect).

You could also try a local or separate installation of z2m with a separate Zigbee network (use a separate dongle or borrow your existing one temporarily) - I've found this to be easier to ensure that the modified zigbeeOTA.js is running & you'll be able to enable more detailed debugging when starting z2m directly on the console.

(unstale)

[bilbolodz]

Could someone please put list of "const url = 'https://raw.githubusercontent.com/Koenkk/zigbee-OTA/master/index.json';" revisions which can allow to upgrade (revision by revision) my bulbs?

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

[fisch55]

Got the same issue with the hue go …

[Cyriuz]

Same issue here using HassOS, would be awesome if we could have a drop down or something in the OTA UI to be able to select previous configs, or just a setting to add our own urls.

[pascal-puetz]

Same issue here, but not on the bulbs but the remotes. I think simulating the multi-step upgrade process would be quite awesome as well. Allow the user to select the firmware (e.g. from a dropdown). Whenever an upgrade is known to cause problems when jumping versions, this could be prefilled with the latest possible version that you can upgrade directly to. Alternatively, do the step in the background, similiar to what the hue bridge is doing.

[joselcaguilar]

Thank you so much for the workaround @davidjb, it's working like a charm! 🎉

[jojeju9428]

hello, I want to update a bulb model 8718696598283 version 1.15.2_r19181 to 1.88.1

can you tell me how to update to 1.88.1

I tested with "https://raw.githubusercontent.com/Koenkk/zigbee-OTA/4af6b2a99517f57c0111566a9735b40ecdbc2af7/index.json" but it doesn't work

Thanks for your help

[jojeju9428]

Problem solved by switching to zigbee2mqtt version 1.23 with the URL 'https://raw.githubusercontent.com/Koenkk/zigbee-OTA/master/index.json';

Update process 1.15.2_r19181 -> 1.50.2_r30933 OK -> 1.88.1 OK

Thanks

[OttoL2]

Problem solved by switching to zigbee2mqtt version 1.23 with the URL 'https://raw.githubusercontent.com/Koenkk/zigbee-OTA/master/index.json';

Update process 1.15.2_r19181 -> 1.50.2_r30933 OK -> 1.88.1 OK

Thanks

Indeed this works! In the latest version of z2m all of a sudden the firmware of the hue w&a could be updated, first to 1.50something and then to the latest version. So finally the desired result, thanks z2m team!!

[Koenkk]

Thanks for confirming this @jojeju9428 @OttoL2

In z2m 1.23.0 the two-step upgrade process was added which has fixed this. Assuming this can be closed now.

[fisch55]

Hi, got anymore invalid image with the hue go.

Current version: 1.23.0-1

https://www.zigbee2mqtt.io/devices/7146060PH.html#philips-7146060ph

[jojeju9428]

Hello,

I have a new update problem on a Phillips Hue E14 bulb model 8718696695203 firmware 1.15.2_r19181

I can't update bulbs with Zigbee2MQTT 1.24

I tried with the URL "https://raw.githubusercontent.com/Koenkk/zigbee-OTA/4af6b2a99517f57c0111566a9735b40ecdbc2af7/index.json" and "https://raw.githubusercontent.com/Koenkk/zigbee-OTA/master /index.json"

Error log

Info Updating 'Lampe de chevet Joy' to latest firmware

Info MQTT publish: topic 'zigbee2mqtt/Lampe de chevet Joy', payload '{"brightness":254,"color_mode":"color_temp","color_temp":366,"linkquality":null,"state":"ON","update":{"state":"available"},"update_available":null}'

Info MQTT publish: topic 'zigbee2mqtt/bridge/response/device/ota_update/update', payload '{"data":{"id":"Lampe de chevet Joy"},"error":"Update of 'Lampe de chevet Joy' failed (Device didn't respond to OTA request)","status":"error","transaction":"di1xq-1"}'

Error Update of 'Lampe de chevet Joy' failed (Device didn't respond to OTA request)

Thanks for your help

[jojeju9428]

Hello,

I have a new update problem on a Phillips Hue E14 bulb model 8718696695203 firmware 1.15.2_r19181

I can't update bulbs with Zigbee2MQTT 1.24

I tried with the URL "https://raw.githubusercontent.com/Koenkk/zigbee-OTA/4af6b2a99517f57c0111566a9735b40ecdbc2af7/index.json" and "https://raw.githubusercontent.com/Koenkk/zigbee-OTA/master /index.json"

Error log

Info Updating 'Lampe de chevet Joy' to latest firmware

Info MQTT publish: topic 'zigbee2mqtt/Lampe de chevet Joy', payload '{"brightness":254,"color_mode":"color_temp","color_temp":366,"linkquality":null,"state":"ON","update":{"state":"available"},"update_available":null}'

Info MQTT publish: topic 'zigbee2mqtt/bridge/response/device/ota_update/update', payload '{"data":{"id":"Lampe de chevet Joy"},"error":"Update of 'Lampe de chevet Joy' failed (Device didn't respond to OTA request)","status":"error","transaction":"di1xq-1"}'

Error Update of 'Lampe de chevet Joy' failed (Device didn't respond to OTA request)

Thanks for your help

Problem solved by switching to link 'https://raw.githubusercontent.com/Koenkk/zigbee-OTA/9089fd7a5070110978a64d5e70a6ef9c5270aed5/index.json'; upgrade 1.15.2_r19181 to 1.50.2_r30933

next

switch link 'https://raw.githubusercontent.com/Koenkk/zigbee-OTA/master/index.json'; upgrade 1.88.1

Update process 1.15.2_r19181 (20160906) -> 1.50.2_r30933 (20191218) OK -> 1.88.1 (20210331) OK

Thanks

[fisch55]

How can I change this update links ? I have the same issue with hue go….

[ricmik]

How can I change this update links ? I have the same issue with hue go….

The link can be updated in Settings -> OTA updates -> OTA index override file name