Ultra low power

[Suxsem]

Hi, thank you for this project!

I noticed in the source code that there are a lot of timers (ex. long poll 20s, short poll 2s, ota checks) that I think will consume the battery too soon.

The tuya firmware basically does NOTHING until an hardware event is detected (other than a 4 hours ping). This is perfect for a sensor-only device such as switches, contact sensors and so on.

How can we ensure the device always sleeps until an actual event is detected (other than a 4 hours or so ping for reporting battery value maybe)?

[doctor64]

Hi!

Yes, I don't do any real power profiling yet, so mostly use SDK defaults. Power optimization will be done later :)
But even in this state devices have pretty good power performance, measures from TI EnergyTrace shows estimate 6 months on CR2032 cell (tested device is 1 button switch).

[doctor64]

Tasks:

• Measure power consumption of my firmware
• Measure power consumption of Tuya firmware
• Power optimization

[Suxsem]

I didn't though it was this low, nonetheless the fact that the device always responds to read commands withing 10 seconds, while with the original tuya firmware the device responded to commands only if wake up with an hardware event make me feel that we can achieve far greater lifetime (> 1 year) by preventing most of the wakes.
Understood that power optimization will come later, but if you can point me to the main source of wakes I can try working on them.
I successfully ported the firmware to a PIR motion sensor (tuya TS0202) and I'd like to still get the optimal battery lifetime I got with the original firmware, thank you!

[doctor64]

Looking to power histogram i can see two types of events - smaller around 7 mA, happens every seconds and larger, up to 26 mA, happens every 3 seconds.
First events, in my expectations, is 1 second interval timer for ZCL attribute reporting. Unfortunately, it cannot be changed easy, because it will be needed to re-write parts of SDK. I know what @slacky1965 worked on this in his watermeter project, but unfortunately, not yet have time to review his implementation
Second events, i think is a Poll events, where device check his uplink is any incoming packet available. Details can be found in ZCL document, in section about POLL_CONTROL.
In my opinion, this intervals should be configured bu user using POLL_CONTROL cluster, but because z2m not (yet?) have support for this - you can play with values of various POLL_RATE defines in tl_zigbee_sdk/zigbee/common/includes/zb_config.h
OTA request interval also configurable, see MY_OTA_PERIODIC_QUERY_INTERVAL in light subproject

PS: we have channel for zigbee developers in Telegram, for fast discussions https://t.me/zigdev

[doctor64]

I was wrong - first peaks of 7 mA is a ADC measuring battery.
I have a played a little with settings and get from IAS sensor :


Poll rate 7 seconds, so theoretically we will still be able to send commands to device without physically awake
OTA check 8 hours
Battery check 15 seconds

[doctor64]

Just to add - POLL_CONTROL cluster need to be disabled, because missing support for POL_Control check-ins in z2m does not allow to switch from fast poll mode to more energy-friendly mode.

[Suxsem]

WOW you did A LOT in just a few hours! I'm going to join the telegram group in a moment, thank you!

I wish I had the energy profiler, it really helps in this kind of situation. Any change you could profile the original firmware to have a baseline?

I saw your commit 70d3c1b, unfortunately it doesn't highlight relevant changes because the whole folder was renamed at the same time, any change you can split the 2 commits? I would like to port your optimizations to my motion sensor firmware (btw, now that you renamed the contact sensor to iasSensor, my firmware don't make much sense anymore :D still I'll show you when it's done, maybe we could add it to the repo).

About the POLL_CONTROL cluster in z2m, I think that this can be done here, am I right?

Just to add - POLL_CONTROL cluster need to be disabled, because missing support for POL_Control check-ins in z2m does not allow to switch from fast poll mode to more energy-friendly mode.

Can you please explain this phrase? I know the fast poll mode, but why you think z2m put the device in fast poll mode?

Finally (a bit unrelated, sorry) I would highlight that while the traditional method of flashing the firmware with the 1...1kohm resistor to the SWS pin is, in fact, unreliable, I found that you can use SWS, TX and RX pins together in this configuration

                 470 Ω 
             ┌-----▅--------SWS
Tx-----------└-----▅--------Rx
                  470 Ω   
Rx---------------------------Tx
RTS--------------------------RST

and use the https://github.com/Ai-Thinker-Open/TBXX_Flash_Tool/blob/master/Telink_Tools.py to reliably (100% of the times with a very cheap FTDI usb converter) and quickly flash new firmwares into the telink board:
Telink_Tools.py -p COM3 burn "motionSensor_TS0202.bin"
For now I'm not using the bootloader mode so it works as is, but with a simple edit to the python script is possible to change the start address in order to flash bootloader mode firmwares.
I was not able to use the read function to backup the original firmware, but my plans were to just throw the sensor in the trash so no big deal.
If you think it could be useful I could write a simple web page to simplify the upload process using this method (i don't think it exists yet, only for the old SWS-only method).

[Suxsem]

Ok I found relevant changes for power optimizations by diffing the old repo with the new one.
I think in zb_appCb.c you have to change other zb_setPollRate and ota_queryStart, specifically the ones in case BDB_COMMISSION_STA_SUCCESS: and else if(evt == OTA_EVT_COMPLETE).
Other than that it seems perfect to me!!!
I still think that 15s for battery check and 7s for poll interval are waaaay too short for a passive sensor like ias ones, but let's wait for a original firmware profiling to see if actually makes any difference.

btw I joined the telegram group but I don't speak your language unfortunately :(

[doctor64]

Yes, you are right about additional changes, i will add this. Thank you, missed it.
About poll_control - yes, you can send configuration commands using console, but one of of functionalities of POLL_CONTROL is what device periodically send check-in requests and coordinator answer to it. For details see ZCL chapter 3.16. In my understanding, currently device sends check-in requests but not get answers, so not switched from fast poll mode to long poll mode. Unfortunately, I'm completely dumb in javascript, so adding support to z2m out of my league.
Yes, we can make intervals even lower, I just not sure if it needed. for me 1 year on CR2032 is good enough, ikea device have similar life. Currently I prefer to keep it short enough so during debugging I can send commands to device without pressing buttons, etc. Later it can be much more.
I use TI 2652 Launchpad board as power profiler, it is cheap (~35 USD) and works. Nordic Power Profiler Kit 2 is better, you can synchronize power measurement with program execution using digital i/o as a trigger, but more expensive ~70-100 USD.
I use PB-03F kit based programmer, serial port method never works for me. But I will happily accept pull request with description of new programming method. :)
Yes, telegram channel is russian speaking, but if you ask question in english we will happily answer it in same language :)

[Suxsem]

Yes, you are right about additional changes, i will add this. Thank you, missed it.

You're welcome!

About poll_control - yes, you can send configuration commands using console, but one of of functionalities of POLL_CONTROL is what device periodically send check-in requests and coordinator answer to it. For details see ZCL chapter 3.16. In my understanding, currently device sends check-in requests but not get answers, so not switched from fast poll mode to long poll mode. Unfortunately, I'm completely dumb in javascript, so adding support to z2m out of my league.

I did my best understanding the specs, that I said it seems to me that the ED switches to the fast poll mode only if the parent (coordinator) answers to the check-in request saying that communication is needed. So in this case (z2m missing check-in support) I don't think ED would enter the fast poll modem, am I wrong?

Yes, we can make intervals even lower, I just not sure if it needed. for me 1 year on CR2032 is good enough, ikea device have similar life. Currently I prefer to keep it short enough so during debugging I can send commands to device without pressing buttons, etc. Later it can be much more.

I agree with you in keeping the interval low for development, but yes I think that for this kind of devices battery life is a very important thing for end users so it's critical to choose it wisely (it seems to me that tuya choose a 4hrs interval!)

I use TI 2652 Launchpad board as power profiler, it is cheap (~35 USD) and works. Nordic Power Profiler Kit 2 is better, you can synchronize power measurement with program execution using digital i/o as a trigger, but more expensive ~70-100 USD.

Didn't know it was that cheap! I will buy it (do you have any link to order it at that price?), in the meantime maybe I will ask you to take some measurements :D (sorry for pushing, but did you have time to measure the tuya original firmware?)

I use PB-03F kit based programmer, serial port method never works for me. But I will happily accept pull request with description of new programming method. :)

The "new" serial method is very reliable for me, I will make the PR!

Yes, telegram channel is russian speaking, but if you ask question in english we will happily answer it in same language :)

Gotcha, thanks!

[doctor64]

So in this case (z2m missing check-in support) I don't think ED would enter the fast poll modem, am I wrong?

Accordingly to ZCL spec you are right, device CAN switch to long poll.

Unfortunately, I see that Telink SDK does not. I expect bug here, but not yet dig into this.

measure the tuya original firmware?

Yes. Idle, no activity


Sensor activated

I agree with you in keeping the interval low for development, but yes I think that for this kind of devices battery life is a very important thing for end users so it's critical to choose it wisely (it seems to me that tuya choose a 4hrs interval!)

Yes, so I currently working to allow users decide they intervals from z2m :)

The "new" serial method is very reliable for me, I will make the PR!

Hmm, does this allow read flash?

[Suxsem]

Thank you for the profiles!
I have almost finished my firmware for PIR sensors and I'm testing with 4hrs polling rate without any issue.

The new serial method supports read flash, but due to a small software issue by Ai-Thinker, that not yet supported (i did found a way to actually read it but it's too involved). I asked for the source code (Ai-Thinker-Open/TBXX_Flash_Tool#5), if they provide it to me I will fix it.

But for erasing and flashing it works perfectly.

As a side note...I'm trying to ask a question in the zigdev telegram channel but it says that administrators have blocked me to send messages in the channel, can you please allow me? My telegram Nickname is Suxsem, thank you!

[Suxsem]

Done! https://github.com/Suxsem/TS0202-PIR-Sensor-Zigbee-firmware
It's working flawlessly for now, finger crossed :)

[Suxsem]

hi @doctor64 !
I'm testing my firmare for almost a month now and I'm very happy with it. I have a request for you, hope it's not too much for asking... Can you please power profile my firmware (https://github.com/Suxsem/TS0202-PIR-Sensor-Zigbee-firmware) for like 5 minutes in sleep and active states? It would be awesome. Thank you!

[pvvx]

Enormous Energy Consumption.

Firmware: switch.

With a short press of the button:

Overage 0.080 mA.

No pressing:

Overage 0.0549 mA.

Calculation for CR2032.
Impulse 26 mA. -> https://github.com/user-attachments/files/18102754/swra349.pdf
Battery 3.0V. Minimum working for SoC 1.8V.
Maximum internal resistance of the battery:
3-1.8 = 1.2V at 26mA
1.2/0.026 = 46.15 Om ->Less than 40% of the life of the most expensive CR2032 battery (cheap one will have even less)
100 mAh / 0.0549 = 1821.5 h -> Maximum 75 days !

Many unnecessary timers are enabled, POLL_CTRL*** is active for 6 sec. The TX level is set to a limit that is incompatible with the current for the CR2032.

*** POLL_CTRL are enabled by ZHA and other software. However, ZHA does not look at the settings offered by the device, as required by the Zigbee 3.0 documentation.

LONG_POLL = 6 * 4 # 6s
zha/zigbee/cluster_handlers/general.py

class PollControlClusterHandler(ClusterHandler):
    """Poll Control cluster handler."""

    CHECKIN_INTERVAL = 55 * 60 * 4  # 55min
    CHECKIN_FAST_POLL_TIMEOUT = 2 * 4  # 2s
    LONG_POLL = 6 * 4  # 6s
    _IGNORED_MANUFACTURER_ID = {
        4476,
    }  # IKEA

[pvvx]

Original firmware from Tuya.

https://pvvx.github.io/TS0041_TZ3000_fa9mlvja/
https://t.me/pvvx_new_ble_zigbee_devices

The firmware processes two events - single and double button presses.

With a short press of the button:

Overage 0.0239 mA (period of 60 sec)

No pressing:

Overage 2.3 uA (period of 240 sec), Sleep 1.8 uA.
Only the timer works for a period of 120 seconds. The battery charge is transferred at a long interval...

The proposed firmware has a consumption 25 times higher!
This shouldn't be offered to users. This is littering the planet with batteries. :)

[pvvx]

How can we ensure the device always sleeps until an actual event is detected (other than a 4 hours or so ping for reporting battery value maybe)?

Elementary - done by replacing several files in the SDK.

I was wrong - first peaks of 7 mA is a ADC measuring battery.

ADC in Telink's Zigbee SDK is always enabled. Each Flash call checks the battery level, which results in additional battery consumption.
You can't switch ADC in Telink Zigbee SDK.
ADC is initialized at each SoC startup. Polling is done by reading values ​​from ADC registers without initialization.
If you change anything in ADC - there will be a shutdown...

It is more difficult to fix - by replacing several files from the SDK and bypassing some functions and variables for lib. That is, by rewriting all calls to Flash and some initialization functions.
There are still many things in the SDK that were made on demand for the Telink debug boards for specific examples and require changes for real projects. No one has thought about optimizing the consumption or the amount of RAM occupied in the open Telink Zigbee SDK, since partners are provided with full source codes.
As a result, you can only work with the fixed SDK.

[doctor64]

@pvvx - I'm really impressed with your deep knowledge. But please, my firmware still not reach ever a beta status and not intended for end-users right now. Moreover, switch firmware completely lacks optimization in power part.

Each Flash call checks the battery level, which results in additional battery consumption.

Only if we have VOLTAGE_DETECT_ENABLE, isn't it?

[pvvx]

Only if we have VOLTAGE_DETECT_ENABLE, isn't it?

I haven't fully analyzed the options of the original Telink SDK. But ADC checks are written in libzb_coordinator.a, libzb_ed.a, libzb_router.a.
Most likely the libraries are already compiled with the VOLTAGE_DETECT_ENABLE option. In order to separate the ADС for your work from libraries, it is necessary to replace some procedures with your own.

And ADC is used to initialize "Random".

I just immediately changed everything related to VOLTAGE_DETECT_ENABLE in the SDK to my own.
https://github.com/pvvx/ZigbeeTLc/tree/master/src/patch_sdk

Also, why do need a bunch of extra code to determine the Flash type if it is intended for chips with external SPI-Flash?

The same with assigning a bunch of additional timers in "reporting". And a bunch of duplicate checks before that. Sending "report" and other messages is placed in the buffers array and if the transmitter is busy, then its own timer is set. But in the example, an additional timer is also set. As a result, there may not be enough timers if the device uses more than one "report". In the SDK examples, only one "report" is used...

As a result, there is a lot of unnecessary stuff in the Telink SDK, which not only causes problems, but also increases consumption. This is especially true when the network is lost. If the coordinator is disconnected or the device is not connected, then the network is continuously searched until the battery is completely discharged.

[doctor64]

@pvvx I think you are little exaggerating the level of consumption. Even if the ADC consumes energy to read the battery level on every flash write operation, regular devices rarely write to flash memory. I understand the importance of this for your project, with logging sensor states to flash, but for most devices, writing and checking the ADC happens very rarely—only during joining and configuring the device. I don't think this will constitute a significant part of the power budget.

I completely agree with you that the Telink Zigbee SDK is not the best example, but what can we do about it? I'm not ready to write a Zigbee stack from scratch or port ZBOSS to Telink, especially for free.

[pvvx]

The problem is not that ADC polling consumes anything, but that it cannot be used.
In the main() procedure, the ADC initialization and polling occurs every start and period of 200 ms. At the same time, the ZigBee polling period is 250 ms, that is, every exit from sleep ADC works.

And in terms of consumption, it turns out to be 25 times more, if you do not get into the user part of the SDK. The Zigbee stack has nothing to do with it.

These files - https://github.com/doctor64/tuyaZigbee/tree/master/switch have nothing to do with the Zigbee stack. This is all user configuration. That is, you wanted to use the Zigbee stack so much.

I understand the importance of this for your project, with logging sensor states to flash

I don't write anything to Flash in Zigbee.
Zigbee has no standard means of reading logs and other information. And there is no support for such actions in external software.
The stack writes all settings and changes to Flash. I only write the initialization if the user changes it. But these are one-time actions.

Tuya uses a analogical SDK, but different settings.

[doctor64]

You mean this part from apps/common/main.c ?

	while(1){
#if VOLTAGE_DETECT_ENABLE
		if(clock_time_exceed(tick, 200 * 1000)){
			voltage_detect(0);
			tick = clock_time();
		}
#endif

    	ev_main();

#if (MODULE_WATCHDOG_ENABLE)
		drv_wd_clear();
#endif

		tl_zbTaskProcedure();

#if	(MODULE_WATCHDOG_ENABLE)
		drv_wd_clear();
#endif
	}

[pvvx]

https://github.com/telink-semi/telink_zigbee_sdk/blob/master/tl_zigbee_sdk/proj/drivers/drv_hw.c#L285
https://github.com/telink-semi/telink_zigbee_sdk/blob/master/tl_zigbee_sdk/proj/drivers/drv_hw.c#L302
...