Announcement: ZSHARK - Wireshark Sniffer for ConBee (Beta)

[manup]

ZSHARK

A helper tool to transfer sniffer data from ConBee USB dongle to a Wireshark instance.

Features

• Local and remote capture
• For Ubuntu, Windows and Raspbian
• Automatic sniffer firmware installation (via GCFFlasher)

Remote capture allows forwarding sniffer data to another computer, for example sniff on a Raspberry Pi and watch in Wireshark on a MacBook.

The beta version for all platforms can be downloaded at https://www.dresden-elektronik.de/zshark

https://phoscon.de/downloads/zshark/

Note deCONZ firmware can not be active at the same time as sniffer firmware on the ConBee. However both firmwares can be used; the deCONZ settings on the ConBee NVRAM will be preserved.

[ebaauw]

macOS? Please?

[snozzlebert]

I think @manup means that you can use Wireshark on macOS while the ConBee with ZShark is running on another machine with Raspbian/Ubuntu/Windows.

[manup]

I think @manup means that you can use Wireshark on macOS while the ConBee with ZShark is running on another machine with Raspbian/Ubuntu/Windows.

Also possible ZShark is running in a Ubuntu VM on a Mac and forwards data to Wireshark which runs natively on the same Mac.

Not the perfect solution, a native macOS version is challenging due the firmware flashing part which needs super user rights. I can't provide a ETA but we may provide a ZShark version for macOS there firmware must be installed separately via GCFFlasher in a terminal (same goes for deCONZ).

[ebaauw]

I want to ditch my Ubuntu VM, if possible. I only use it for BitCatcher. Happy to flash the ConBee manually, if that what it takes.

I don't suppose I can use a single Raspberry with a RaspBee and a ConBee installed, running deCONZ on the RaspBee and ZShark on the ConBee in parallel.

[manup]

I don't suppose I can use a single Raspberry with a RaspBee and a ConBee installed, running deCONZ on the RaspBee and ZShark on the ConBee in parallel.

Just tried that, yes works too :)

[ebaauw]

Indeed, that works!

I get more junk (ICMP messages and ACKs) then ZigBee messages when filtering on port 17754. If I apply a display filter for zbee_nwk or zbee_aps, WireShark only shows the (ethernet frames with encapsulated) ZigBee frames.

I do seem to miss quite a few packages when sniffing the local deCONZ network.

The Network Settings dialog in the deCONZ GUI shows zeroes for all fields. Not sure if this is related to ZShark, or RaspBee firmware 0x261D0500 (I use a self compiled version of the REST API plugin, so it updated to this version). deCONZ seems to work fine nevertheless, even restarting does work. ZShark was happy to update the ConBee firmware (this is different from the BitCatcher firmware?) while deCONZ was running.
EDIT manually updated the RaspBee to 0x261E0500 and the network settings are shown again.

[ebaauw]

I do seem to miss quite a few packages when sniffing the local deCONZ network.

I suspect it's due to radio interference between the RaspBee and the ConBee (both connected to the same Raspberry). I dug up an old USB Female-A to A 5m extension cable (from a previous life), and connected the ConBee through that. It works (even through 5m is pushing from a USB standard perspective). I can now place the ConBee halfway in between the RaspBerry and the device being sniffed, and it looks like I now capture the full traffic.

Can I only enter an IP address in the Remote Capture field? I tried a hostname instead, but that doesn't seem to work?

[manup]

It works (even through 5m is pushing from a USB standard perspective). I can now place the ConBee halfway in between the RaspBerry and the device being sniffed, and it looks like I now capture the full traffic.

Good catch I'll try this too, radio interference can be a beast :)

We also are investigating in the missed packages in some scenarios like OTA traffic, for one the RX circular buffer was quite small (8) we raised that to 32. Also sniffer is running at 38400 baud which we can raise since ConBee has a FTDI in a future firmware.

Meanwhile here is the version with larger circular buffer, you may give it a try, it might help but this isn't verified yet:

sniffer-1.00-32packets.zip

Flashed as usual:

$ sudo GCFFlasher_internal -d 0 -f sniffer-1.00-32packets.bin

Can I only enter an IP address in the Remote Capture field? I tried a hostname instead, but that doesn't seem to work?

I'll forward this, the sniffer is developed by a colleague, should be easy to fix.

[Framspott]

Hi,

ist there a possibility to sniff the communication between a Zigbee device and a Raspbee?
I want to know the communication over the serial port /dev/ttyS0 between my Hue bulb and my Raspberry.

All the best

Framspott

[manup]

To see whole ZigBee packets you need a ConBee which acts as sniffer device to monitor the ZigBee traffic. The RaspBee can't act as sniffer and coordinator at the same time.

[manup]

ZSHARK is now officially released.

If you are running an older ZSHARK firmware it's strongly advised to update the firmware contained in the package, we fixed some nasty bugs which should improve sniffer sensitivity a lot.

https://www.dresden-elektronik.de/funktechnik/products/software/zshark/?L=1

[ebaauw]

Dare I ask about macOS support?

[manup]

We currently experimenting with the Qt Installer Framework (ZSHARK Windows) and automated builds (deCONZ Raspbian/Ubuntu). As soon as it's stable I hope to also derive automated macOS builds for ZSHARK and deCONZ.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ebaauw]

We might want to pin this topic.

[manup]

Yes maybe also add a reference in the Wiki, otherwise the Website always has the latest release.

https://www.dresden-elektronik.de/funktechnik/products/software/zshark/?L=1

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ebaauw]

Pin

[manup]

Cool didn't know there is a pin option :)

[timbru31]

I assume the CC2531 USB Stick is not supported? Have a spare one lying around. (so far I've used https://github.com/andrewdodd/ccsniffpiper)

---

A quick question, too: @manup - It seems that a NWK key with 32bit MIC (Message Integrity Code) is used (security level 5) - why no 128bit key for the MIC (security level 7)? Any reason?

[lynaghk]

Does ZShark support the ConBee II?

On Windows 10 deCONZ lists the ConBee II as a device option, but ZShark's device list is empty (zshark version 1.02):

I've also tried Ubuntu 18.04 LTS in a VirtualBox VM, but ZShark doesn't pick up the device there either (though lsusb shows that it's visible within the VM: Bus 002 Device 002: ID 1cf1:0030 Dresden Elektronik)

I was under the impression that it should be compatible, based on this PDF:

[manup]

Sorry this is an older PDF, ZSHARK will support ConBee II but it isn't ported yet. There is no ETA, but I hope it will be done within the next two months.

[manup]

Reflashing the RaspBee or ConBee with the ZShark firmware and then back with the deCONZ firmware will likely lose the data in the device’s non-volatile memory. You might want to double-check that the network parameters (PAN ID, network key, channel) are still valid.

The values should be stable and preserved since ZSHARK firmware doesn't touch the NVRAM of ConBee I and RaspBee I, which is stored in the EEPROM.

[dandrzejewski]

After installing this firmware and playing with it, then going back to the normal firmware, none of my zigbee devices can connect and are not responsive.

Reflashing the RaspBee or ConBee with the ZShark firmware and then back with the deCONZ firmware will likely lose the data in the device’s non-volatile memory. You might want to double-check that the network parameters (PAN ID, network key, channel) are still valid.

The OP in this issue states that this should not happen. But it did end up changing the PAN ID, which I changed back.

Interestingly, I power-cycled ONE of the bulbs and the entire mesh came back online.

That’s not how ZigBee works. Your bulbs have been “online” all the time; they don’t need the coordinator to communicate with each other. It’s just the RaspBee or ConBee not having discovered the network.

Got it. The UI doesn't make that clear (to me, at least).

The graph in the GUI is just a graphical representation of the info deCONZ has received from your devices. It does not show any “active” connections, just the info from the routing tables from the time the device was last queried. When you power-cycle a device, it sends a Device Announcement message, which deCONZ picks up to query the device with prejudice.

Right, well my point is that I want to rely on this device to be an interface between HomeSeer and ZigBee. After going through this, the fact that it lost track of all the devices when the original post said it should not is concerning. And the fact that things didn't automatically come back up is, too.

This same issue even happens if I simply power cycle the pi (shutting it down cleanly of course) and then bring it back up. Even after several hours the device is unable to talk to any other devices. I would expect things should be able to heal within minutes at worst.

Nevertheless, power cycling one bulb seems to get it "un-stuck," probably for the reason you stated - it sends a device announcement.

On closer inspection, it appears that the raspbee is only able to contact the other devices "through" the bulb I power cycled (that bulb is a repeater). That's according to the diagram in deCONZ, and evidenced by the fact that everything else stops working again if I power off that bulb.

Is there some parameter I'm missing, or is this a bug, or is there something I just don't understand about how ZigBee is supposed to work?

[ebaauw]

On closer inspection, it appears that the raspbee is only able to contact the other devices "through" the bulb I power cycled (that bulb is a repeater). That's according to the diagram in deCONZ, and evidenced by the fact that everything else stops working again if I power off that bulb.

In theory, that could be valid, if this one bulb is the only device in direct range of the RaspBee or ConBee. What happens after you power-cycle another bulb?

[Cyrille63]

Hi
ConBee II is not supported by Zshark, have you got a support date?
Thk

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tkl-git]

Unfortunately the ConBee II is not yet supported in ZSHARK, the support for it is scheduled for 2020.

Hi, do you have updated information? I think Zshark support for ConBee 2 would be a very useful feature and would really help people to debug Zigbee network problems.

[abmantis]

Any way to run zshark on a headless machine (without VNC nor X-Forwarding)?

[thedayofcondor]

Any progress on the zshark support for the conbee ii?

[deennoo]

how to revert from sniffer on classic Conbee Usb Dongle Please ?

On each update 2 solutions :

• if zshark is on and connected to conbee usb : gccf try to rebot again and again and again the device
• if zshark is on and not connected to conbee usb : Fw update fail because of snifferv1.0 fw

STARTING APP
Sniffer V1.0 [ras (426 ms)
STARTING APP
Sniffer V1.0 [raspbee]
 (442 ms)
could not sync with bootloader
retry, failed

Can some one help ?

[Smanar]

The classic command line to update firmware not working ? https://github.com/dresden-elektronik/deconz-rest-plugin/wiki/Update-deCONZ-manually

[manup]

The classic command line to update firmware not working ? https://github.com/dresden-elektronik/deconz-rest-plugin/wiki/Update-deCONZ-manually

Yes that's the correct way to install the deCONZ firmware again (note all configuration parameters are preserved).

A note on ConBee II support, unfortunately this didn't get finished in 2020. There is some active work in progress to get it out in Q1 2021.

[abmantis]

Sniffer V1.0 [raspbee]

I am also unable to revert a raspbee from sniffer fw to deconz fw:

GCFFlasher V3_17 (c) dresden elektronik ingenieurtechnik gmbh
Reboot device /dev/ttyAMA0 (RaspBee)
action: reset device RaspBee
wiringPi 2.52 initialized
RaspBee Bootloader premium
Vers. 1.02
build 2013/08/01
STARTING (286 ms)
STARTING APP
 (288 ms)
STARTING APP
Sniffer  (318 ms)
STARTING APP
Sniffer V1.0 [ra (320 ms)
STARTING APP
Sniffer V1.0 [raspbee]
 (323 ms)
could not sync with bootloader

EDIT: It seems that an older version of GCFFLasher is required. I used GCFFlasher V2_11, from deconz 2.05.10 and it worked.

[azurit]

Where can i download the latest firmware? Cannot find it anywhere, thank you.

[Smanar]

Good question.
The firmware is not provided on the package ?

[azurit]

It is but is it a latest one?

Anyway, if it's used, zshark is showing error (that's why i'm searching for other versions): Firmware install error

[Smanar]

It seem it s the good version, others users are using the same link https://forum.phoscon.de/t/announcement-zshark-is-now-available-for-conbee-ii/207

I have never tried on my side, I think there is a different firmware for conbee 1 and conbee 2 ? you are not using VM or docker ?

[azurit]

I'm running it directly on Ubuntu linux. It's a Conbee 2. I was also trying to install firmware manually but no luck:

$ sudo GCFFlasher_internal -d 0 -f /usr/share/zshark/fw/sniffer_v1_0.bin -x 3
GCFFlasher V3_17 (c) dresden elektronik ingenieurtechnik gmbh
11:38:20:469 using firmware file: /usr/share/zshark/fw/sniffer_v1_0.bin
11:38:20:499 ls dev: /dev/ttyACM0 (0x1CF1/0x0030) sn: DE2480257
11:38:20:499 ls dev: /dev/ttyS4 (0x8086/0x8C3D) sn:
11:38:20:499 ls dev: /dev/ttyS0 (0x0000/0x0000) sn:
11:38:20:499 search FTDI device: no.0
11:38:20:499 device 0 not found

deCONZ and Phoscon App are both working fine.

[azurit]

Also this:

$ sudo GCFFlasher_internal -d /dev/ttyACM0 -f /usr/share/zshark/fw/sniffer_v1_0.bin
GCFFlasher V3_17 (c) dresden elektronik ingenieurtechnik gmbh
Reboot device /dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DE2480257-if00 (ConBee II)
deCONZ firmware version 26720700
R21B18 Bootloader
Vers: 2.07
build: Jun 17 2019
flashing 11276 bytes: |=error: timeout flashing firmware after 3000 ms

[Smanar]

Have you tried just, to check device detection.

GCFFlasher_internal -l

[azurit]

$ GCFFlasher_internal -l
GCFFlasher V3_17 (c) dresden elektronik ingenieurtechnik gmbh
Path             | Vendor | Product | Serial     | Type
-----------------+--------+---------+------------+-------
/dev/ttyACM0     | 0x1CF1 | 0x0030  | DE2480257  | ConBee II

[Smanar]

Ha ?
All look fine, sorry no idea on my side, will ask to others devs if someone have already do it.

[ebaauw]

Did you try adding -t 60 to give GCFFlasher_internal some more time?

[Smanar]

Some have used the "Sniffer_v_1" for a Raspbee 1, not sure it's the same for the conbee 2.
What files you have for the firmware ?

Edit:
Try with the file zshark_conbee2_0x31000700.bin.GCF you have it in the same folder, have just checked.

[manup]

Hi, the ZShark + firmware was updated last week. zshark_conbee2_0x31000700.bin.GCF is the newest one for ConBee II.
It fixes a bug of the previous version, which once flashed couldn't be updated without detaching/attaching the USB dongle after GCFFlasher is started.