[New device support]: Tuya Zigbee Smart Vibration Detector

[flatsiedatsie]

Link

https://nl.aliexpress.com/item/1005003168207230.html

Database entry

{"id":26,"type":"EndDevice","ieeeAddr":"0x540f57fffeXXXXXX","nwkAddr":21950,"manufId":4098,"manufName":"_TZE200_kzm5w4iz","powerSource":"Battery","modelId":"TS0601","epList":[1],"endpoints":{"1":{"profId":260,"epId":1,"devId":81,"inClusterList":[0,4,5,61184],"outClusterList":[25,10],"clusters":{"genBasic":{"attributes":{"65503":"\u0000\u0000\u0000\u0000\u0004\u0000\u0000\u0000\u0000\u0004\u0000\u0000\u0000\u0000\u0005\u0000\u0000\u0000\u0000\u0005\u0000\u0000\u0000\u0000\u0012\u0000\u0000\u0000\u0000\u0012\u000b_O)\u0017\f_O)\u0013\u0000\u0000\u0000\u0000\u0005\u0000\u0000\u0000\u0000\u0012\u0000\u0000\u0000\u0000\u0012","65506":31,"65508":0,"modelId":"TS0601","manufacturerName":"_TZE200_kzm5w4iz","powerSource":3,"zclVersion":3,"appVersion":68,"stackVersion":0,"hwVersion":1,"dateCode":""}}},"binds":[{"cluster":0,"type":"endpoint","deviceIeeeAddress":"0x00212effffXXXXXX","endpointID":1}],"configuredReportings":[],"meta":{}}},"appVersion":68,"stackVersion":0,"hwVersion":1,"dateCode":"","zclVersion":3,"interviewCompleted":true,"meta":{"configured":821693351},"lastSeen":1641205346747,"useImplicitCheckin":true}

Comments

I'm trying to add a new Tuya device (a huge vibration sensor that runs on AAA batteries).

I tried following the guide but ran into issues documented in this other issue

These is the live debug output:

Zigbee2MQTT:debug 2022-01-03 12:38:40: Received Zigbee message from '0x540f57fffeXXXXXX', type 'commandGetData', cluster 'manuSpecificTuya', data '{"data":{"data":[1],"type":"Buffer"},"datatype":1,"dp":1,"fn":0,"status":0,"transid":20}' from endpoint 1 with groupID null
Zigbee2MQTT:debug 2022-01-03 12:38:41: Received Zigbee message from '0x540f57fffeXXXXXX', type 'commandGetData', cluster 'manuSpecificTuya', data '{"data":{"data":[0],"type":"Buffer"},"datatype":1,"dp":1,"fn":0,"status":0,"transid":21}' from endpoint 1 with groupID null
Zigbee2MQTT:debug 2022-01-03 12:38:41: Received Zigbee message from '0x540f57fffeXXXXXX', type 'commandGetData', cluster 'manuSpecificTuya', data '{"data":{"data":[1],"type":"Buffer"},"datatype":4,"dp":10,"fn":0,"status":0,"transid":21}' from endpoint 1 with groupID null

After doing a little modification I managed to get the tuya.dump.txt file. It looks like this (after removing privacy-sensitive data):

1641214402464 0x540f57fffeXXXXXX 00 19 0a 04 00 01
1641214403362 0x540f57fffeXXXXXX 00 1a 0a 04 00 00
1641214403491 0x540f57fffeXXXXXX 00 1a 01 01 00 00
1641214403610 0x540f57fffeXXXXXX 00 1a 0a 04 00 01
1641214404316 0x540f57fffeXXXXXX 00 1b 0a 04 00 00
1641214419725 0x540f57fffeXXXXXX 00 1c 01 01 00 01
1641214423133 0x540f57fffeXXXXXX 00 1d 0a 04 00 01
1641214424067 0x540f57fffeXXXXXX 00 1e 0a 04 00 00
1641214426246 0x540f57fffeXXXXXX 00 1f 0a 04 00 01
1641214427176 0x540f57fffeXXXXXX 00 20 0a 04 00 00
1641214428038 0x540f57fffeXXXXXX 00 21 01 01 00 00
1641214430173 0x540f57fffeXXXXXX 00 22 0a 04 00 01
1641214431110 0x540f57fffeXXXXXX 00 23 0a 04 00 00
1641214431239 0x540f57fffeXXXXXX 00 23 01 01 00 00
1641214432539 0x540f57fffeXXXXXX 00 24 0a 04 00 01
1641214433471 0x540f57fffeXXXXXX 00 25 0a 04 00 00
1641214497204 0x540f57fffeXXXXXX 00 26 01 01 00 01
1641214500500 0x540f57fffeXXXXXX 00 27 01 01 00 00
1641214506730 0x540f57fffeXXXXXX 00 28 01 01 00 01
1641214509555 0x540f57fffeXXXXXX 00 29 01 01 00 00
1641214511646 0x540f57fffeXXXXXX 00 2a 01 01 00 01
1641214514240 0x540f57fffeXXXXXX 00 2b 0a 04 00 01
1641214515166 0x540f57fffeXXXXXX 00 2c 0a 04 00 00
1641214518655 0x540f57fffeXXXXXX 00 2d 0a 04 00 01
1641214519576 0x540f57fffeXXXXXX 00 2e 0a 04 00 00
1641214523873 0x540f57fffeXXXXXX 00 2f 0a 04 00 01
1641214524794 0x540f57fffeXXXXXX 00 30 0a 04 00 00
1641214529319 0x540f57fffeXXXXXX 00 31 0a 04 00 01
1641214530276 0x540f57fffeXXXXXX 00 32 0a 04 00 00
1641214534691 0x540f57fffeXXXXXX 00 33 01 01 00 00
1641214534823 0x540f57fffeXXXXXX 00 33 03 02 00 00 00 00 61
1641214534943 0x540f57fffeXXXXXX 00 34 0a 04 00 01
1641214535777 0x540f57fffeXXXXXX 00 35 0a 04 00 00
1641214657232 0x540f57fffeXXXXXX 00 36 0a 04 00 01
1641214658098 0x540f57fffeXXXXXX 00 37 0a 04 00 00

I fed that into the python parser (tail -f -n +0 tuya.dump.txt | python3 read_tuya_dump.py) and got this output:

WARN: Invalid bool length
2022-01-03T12:38:40 0x540f57fffeXXXXXX (seq:    0, value: #20) [DP   1 unknown     ] => bool: False
WARN: Invalid bool length
2022-01-03T12:38:41 0x540f57fffeXXXXXX (seq:    0, value: #21) [DP   1 unknown     ] => bool: False
WARN: Invalid enum length
2022-01-03T12:38:41 0x540f57fffeXXXXXX (seq:    0, value: #21) [DP  10 frost detect] => enum: 0
WARN: Invalid enum length
2022-01-03T12:38:42 0x540f57fffeXXXXXX (seq:    0, value: #22) [DP  10 frost detect] => enum: 0

...

2022-01-03T13:55:30 0x540f57fffeXXXXXX (seq:    0, value: #50) [DP  10 frost detect] => enum: 0
WARN: Invalid bool length
2022-01-03T13:55:34 0x540f57fffeXXXXXX (seq:    0, value: #51) [DP   1 unknown     ] => bool: False
WARN: Invalid value length
2022-01-03T13:55:34 0x540f57fffeXXXXXX (seq:    0, value: #51) [DP   3 unknown     ] => int: 97

Since the device is not a frost detect sensor, I wonder how I should interpret this.

• The bool: False occurs when the magnet is moved away or back to the device.
• The enum: 0 occurs when the main body is shaken or hit (vibration)
• The int: 97 is probably the battery level?

What should my next step be?

External converter

const fz = require('zigbee-herdsman-converters/converters/fromZigbee');
const tz = require('zigbee-herdsman-converters/converters/toZigbee');
const exposes = require('zigbee-herdsman-converters/lib/exposes');
const reporting = require('zigbee-herdsman-converters/lib/reporting');
const extend = require('zigbee-herdsman-converters/lib/extend');
const e = exposes.presets;
const ea = exposes.access;
const tuya = require("zigbee-herdsman-converters/lib/tuya");

const definition = {
        fingerprint: [{modelID: 'TS0601', manufacturerName: '_TZE200_kzm5w4iz'}],
        model: 'TS0601',
        vendor: 'TuYa',
        description: 'Vibration sensor',
        supports: 'vibration',
        fromZigbee: [fz.tuya_data_point_dump],
        toZigbee: [tz.tuya_data_point_test],
        onEvent: tuya.onEventSetTime, // Add this if you are getting no converter for 'commandMcuSyncTime'
        configure: async (device, coordinatorEndpoint, logger) => {
            const endpoint = device.getEndpoint(1);
            await reporting.bind(endpoint, coordinatorEndpoint, ['genBasic']);
        },
        exposes: [], 
    };

module.exports = definition;

// e.battery(), e.vibration(), exposes.enum('sensitivity', exposes.access.STATE_SET, ['low', 'medium', 'high'])

Supported color modes

No response

Color temperature range

No response

[flatsiedatsie]

I tried getting it to work, but didn't get very far.

External converter:

const fz = require('zigbee-herdsman-converters/converters/fromZigbee');
const tz = require('zigbee-herdsman-converters/converters/toZigbee');
const exposes = require('zigbee-herdsman-converters/lib/exposes');
const reporting = require('zigbee-herdsman-converters/lib/reporting');
const extend = require('zigbee-herdsman-converters/lib/extend');
const e = exposes.presets;
const ea = exposes.access;
const tuya = require("zigbee-herdsman-converters/lib/tuya");

const definition = {
        fingerprint: [{modelID: 'TS0601', manufacturerName: '_TZE200_kzm5w4iz'}],
        model: 'TS0601',
        vendor: 'TuYa',
        description: 'Vibration sensor',
        supports: 'vibration',
        fromZigbee: [
            //fz.ignore_basic_report, // Add this if you are getting no converter for 'genBasic'
            fz.tuya_data_point_dump,
            fz.tuya_smart_vibration_sensor, // 1
            //fz.tuyaVibration, // 10 
            //fz.thitBatteryPercentage, // 3
        ],
        toZigbee: [],
        //onEvent: tuya.onEventSetTime, // Add this if you are getting no converter for 'commandMcuSyncTime'
        configure: async (device, coordinatorEndpoint, logger) => {
            const endpoint = device.getEndpoint(1);
            await reporting.bind(endpoint, coordinatorEndpoint, ['genBasic']);
        },
        exposes: [e.vibration()], 
    };

module.exports = definition;

I added tuyaVibration: 10, to tuya.js

I added this to from_zigbee.js:

    tuya_smart_vibration_sensor: {
        cluster: 'manuSpecificTuya',
        //type: ['commandStatusChangeNotification'],
        type: ['commandGetData'],
        convert: (model, msg, publish, options, meta) => {
            console.log("In converter!");
            const dp = msg.data.dp;
            console.log("- In converter, dp = ", dp);
            const value = tuyaGetDataValue(msg.data.datatype, msg.data.data); // This function will take care of converting the data to proper JS type
            console.log("- In converter, value = ", value);
            //if (utils.hasAlreadyProcessedMessage(msg)) return;
            switch (dp) {
            case tuya.dataPoints.tuyaVibration:
                return {vibration: {0: 'false', 1: 'true'}[value]}; // value is already converted to a number in JS
            default:
                meta.logger.warn(`zigbee-herdsman-converters:tuya_smart_vibration_sensor: NOT RECOGNIZED DP #${dp} with data ${JSON.stringify(msg.data)}`); // This will cause zigbee2mqtt to print similar data to what is dumped in tuya.dump.txt
            }
        },
    },

[gmbnomis]

The format of the tuya.dump.txt has changed recently. The log messages look like they were obtained on a non-development version of zigbee2mqtt, but the output of the Python script looks like the more recent version of the script. Can you try to obtain a tuya.dump.txt using the current development version of zigbee2mqtt?

[gmbnomis]

Also, the most recent version of how to add a Tuya device is not yet part of the published documentation, you can find it here: https://github.com/Koenkk/zigbee2mqtt.io/blob/develop/docs/advanced/support-new-devices/02_support_new_tuya_devices.md

[flatsiedatsie]

I installed the dev version on a second Raspberry Pi and had some different results:

2022-01-04T12:41:32 0x540f57fffe37cb66 (seq: 16896, value: #10) [DP   4 unknown     ] => raw: [   1 ]
2022-01-04T12:41:33 0x540f57fffe37cb66 (seq: 17152, value: #10) [DP   4 unknown     ] => raw: [   0 ]
2022-01-04T12:44:42 0x540f57fffe37cb66 (seq: 17408, value: #1) [DP   1 unknown     ] => raw: [   0 ]
2022-01-04T12:44:42 0x540f57fffe37cb66 (seq: 17408, value: #10) [DP   4 unknown     ] => raw: [   1 ]
2022-01-04T12:44:43 0x540f57fffe37cb66 (seq: 17664, value: #10) [DP   4 unknown     ] => raw: [   0 ]
2022-01-04T12:44:43 0x540f57fffe37cb66 (seq: 17664, value: #10) [DP   4 unknown     ] => raw: [   1 ]
2022-01-04T12:44:44 0x540f57fffe37cb66 (seq: 17920, value: #10) [DP   4 unknown     ] => raw: [   0 ]
2022-01-04T12:44:44 0x540f57fffe37cb66 (seq: 17920, value: #1) [DP   1 unknown     ] => raw: [   1 ]
2022-01-04T12:44:54 0x540f57fffe37cb66 (seq: 18176, value: #10) [DP   4 unknown     ] => raw: [   1 ]

This is great, because it means the device does send boolean data.

[flatsiedatsie]

TLRD: I managed to get it to work!

Just to share some things that aren't or weren't clear to me during the process, perhaps it's useful:

• Is the 'external converter' I'm making intended to a temporary file to work in until I feel the converter works ok? But since I'm supposed to modify files like tuya.js anyway, why not directly modify the file that will eventually need to be edited? Which file is that?
• Those converters in from_zigbee.js, are those supposed to refer to specific devices, or properties that may be available accross multiple devices? Most of the converters' names indicate the former, but not all.

Variables

• Since the python script gave raw: [ 1 ], does that mean I should use type: ['raw'], in my converter? The debug output gives something else: Zigbee2MQTT:debug 2022-01-04 13:49:23: Received Zigbee message from '0x540f57fffe37cb66', type 'commandDataResponse', cluster 'manuSpecificTuya', data '{"data":{"data":[1],"type":"Buffer"},"datatype":4,"dp":10,"fn":0,"seq":19712}' from endpoint 1 with groupID 0. That seems to indicate that I should use commandDataResponse instead of raw?
• Does (seq: 18688, value: #10) [DP 4 unknown ] => raw: [ 0 ] mean that the DP is 4? The debug line has "dp":10,, so I'm a little confused here.

If the debug output is correct, why not refer people to that instead of using read_tuya_dump.py? It seems much simpler.

How does exposes functionality work?

• I only see a few exposes related examples in from_zigbee.js, mostly around temperature and humidity sensors. How can I make sure a binary / vibration sensor gets exposed? I do see options: [exposes.options.vibration_timeout()], but that seems to work with devices that don't turn themselves off. My device uses a binary, and 'toggles itself' back to off after a second or so.
• If I add exposes: [e.contact(),e.battery(), e.vibration()], to my exernal converter file, how does that know which outputs of the converter to 'match with'? Is returning json with those variable names from my converter enough?
• I don't see link quality being exposed to mentioned anywhere. I take it Zigbee2MQTT handles this automatically.

In the end it seems that exposes was handled automatically. Adding those names to the external converter seems to be all I needed to do. Very nice.

About the example in the documentation: I found it very useful to have this inside my converter:

 meta.logger.warn(`zigbee-herdsman-converters: HANDLING DP #${dp} with data ${JSON.stringify(msg.data)}`);

And since I'm at it:

• Is the mac address of each zigbee device unique? Is it a (tiny) privacy risk to share that mac online? Could a hypothetical wardriver scan for zigbee 'mac' addresses? Do the devices rotate these 'mac' addresses?

[gmbnomis]

• Is the 'external converter' I'm making intended to a temporary file to work in until I feel the converter works ok? But since I'm supposed to modify files like tuya.js anyway, why not directly modify the file that will eventually need to be edited? Which file is that?

The device and converter files are in https://github.com/Koenkk/zigbee-herdsman-converters, which is installed as an npm module when installing zigbee2mqtt. I suppose that's why it is simpler to use an external converter first. But yes, once it works and if you want the device to be supported upstream, the changes need to go to zigbee-herdsman-converters.

• Those converters in from_zigbee.js, are those supposed to refer to specific devices, or properties that may be available accross multiple devices? Most of the converters' names indicate the former, but not all.

In general they are available across devices. I suppose that some are very specific to devices (for Tuya, it looks like device manufacturers pick data point IDs pretty much at random)

Variables

• Since the python script gave raw: [ 1 ], does that mean I should use type: ['raw'], in my converter? The debug output gives something else: Zigbee2MQTT:debug 2022-01-04 13:49:23: Received Zigbee message from '0x540f57fffe37cb66', type 'commandDataResponse', cluster 'manuSpecificTuya', data '{"data":{"data":[1],"type":"Buffer"},"datatype":4,"dp":10,"fn":0,"seq":19712}' from endpoint 1 with groupID 0. That seems to indicate that I should use commandDataResponse instead of raw?

The output of the Python script still looks very wrong. A log line written by the current development version looks like this:

Received Zigbee message from '0x84xxx', type 'commandDataResponse', cluster 'manuSpecificTuya', data '{"dpValues":[{"data":{"data":[1],"type":"Buffer"},"datatype":4,"dp":1},{"data":{"data":[0,0,0,90],"type":"Buffer"},"datatype":2,"dp":4}],"seq":0}' from endpoint 1 with groupID 0
For example, the fn field isn't there anymore (it did not make any sense, as it was always 0)

• Does (seq: 18688, value: #10) [DP 4 unknown ] => raw: [ 0 ] mean that the DP is 4? The debug line has "dp":10,, so I'm a little confused here.

Yes, it means that the DP is 4, but you are right, it is 10 indeed. As said, if you generated the dump file with a version giving out log lines with fn, that's still the old format. None of your DPs actually seem to be in "raw" format.

If the debug output is correct, why not refer people to that instead of using read_tuya_dump.py? It seems much simpler.

Usually, they should match 😉. The Python script actually decodes the data (into enum, int, string, bool, ...). But yes, it has limited value.

How does exposes functionality work?

• I only see a few exposes related examples in from_zigbee.js, mostly around temperature and humidity sensors. How can I make sure a binary / vibration sensor gets exposed? I do see options: [exposes.options.vibration_timeout()], but that seems to work with devices that don't turn themselves off. My device uses a binary, and 'toggles itself' back to off after a second or so.
• If I add exposes: [e.contact(),e.battery(), e.vibration()], to my exernal converter file, how does that know which outputs of the converter to 'match with'? Is returning json with those variable names from my converter enough?
• I don't see link quality being exposed to mentioned anywhere. I take it Zigbee2MQTT handles this automatically.

In the end it seems that exposes was handled automatically. Adding those names to the external converter seems to be all I needed to do. Very nice.

I am quite new to zigbee2mqtt development myself. The exposes part also looks like magic to me, still.

About the example in the documentation: I found it very useful to have this inside my converter:

 meta.logger.warn(`zigbee-herdsman-converters: HANDLING DP #${dp} with data ${JSON.stringify(msg.data)}`);

I wonder if:

1. we should add such logging to the tuya_data_point_dump converter and
2. tuya_data_point_dump converter should decode the value already (i.e. log the data type and the data in a human readable form (like the Python script does. dp values would still be numerical, though))

And since I'm at it:

• Is the mac address of each zigbee device unique? Is it a (tiny) privacy risk to share that mac online? Could a hypothetical wardriver scan for zigbee 'mac' addresses? Do the devices rotate these 'mac' addresses?

honestly, I have no idea.

[flatsiedatsie]

Thanks @gmbnomis

I like that idea of adding the logger. Still, i think the current debug output already gave me all I needed.

I just added a pull request :-)

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days