Bluetooth5.0+ PHY (LE 1M/2M/LongRange:500K/125K), CSA1/CSA2 (Channel Selection Algorithm)

[Sukalo0290]

Can you explain a bit more about this?

Is this implemented on the sensors from version 3.7? Even if the sensors are Bluetooth 4.2?

Are they now broadcasting on multiple channels so a BT chip capable of CSA can pick up more readings?

Thanks in advance, just trying to understand it.

[pvvx]

These options relate only to BLE Сonnection. Advertising packages are still broadcast on 3 typical channels, regardless of BT version. In new versions BT, the advertising package may indicate an additional channel, but in this case it is not used.

[hcgpalm]

Is this a limitation with the hardware/runtime libraries, or would it be possible to make it advertise over, for example, S8 coded PHY?
The, probably obvious, use case beeing able to monitor the device using the BLE 5 long range features which I suppose would greatly increase the current, rather limited, range(?)

[pvvx]

BLE LongRange (S2: 500K / S8: 125K) has been supported since version 3.6.

[hcgpalm]

So it is advertising over coded PHY?

[pvvx]

BLE standard: Only Extended Advertising come in other formats.

In the current version, "Extended Advertising" is not supported. Strongly increases consumption and does not make sense, because. the main advertising channels should still work on PHY 1 Mbps. And to find out on which channel "Extended advertising" is possible only after receiving on a typical channel (PHY 1 Mbps).

Coded PHY requires a special device to receive advertisements.
Regular adapters and smartphones do not work with advertising on shared channels on an Coded PHY.

Coded PHY is possible only after negotiation in connection mode on 1M PHY.
A connection request is always executed at 1 Mbps... Then you can switch the PHY format.

---

[hcgpalm]

Ah, I see, so what you're saying is that there's no standard for (optional) coded advertisements(?) Essentially making BLE long range per se practically useless since a peer always needs to be able to receive the standard mandatory 1M advertisements anyway, in order to negotiate a coded connection or learn about coded "extended" advertisements?

I know that coded PHY is an optional feature not supported by most devices today. The target audience here is obviously those of us that do have a long range capable adapter and need the extended range.

[pvvx]

I know that coded PHY is an optional feature not supported by most devices today. The target audience here is obviously those of us that do have a long range capable adapter and need the extended range.

TLSR825x supports ZigBee and has a ZigBee SDK from Telink.
OTA for the BLE variant does not have the ability to download the ZigBee version. ZigBee has a larger firmware code size.
Dual OTA solves ZigBee firmware. The first block is to change the Flash markup for OTA with ZigBee ... The second is the ZigBee code itself.
But I don't want to support the ZigBee version.

[hcgpalm]

Sorry, but I don't understand how ZigBee is related to this?

[pvvx]

ZigBee uses PHY 250 kbps -> greater transmission-reception distance.

[DienoX]

Will the LongRange setting allow me to use the thermometer over longer distances?

[pvvx]

Will the LongRange setting allow me to use the thermometer over longer distances?

No, because manufacturers ignore this feature in Bluetooth 5.0 on user devices.
Long Range provides a tenfold increase in distance.
Some newer BT5.2 adapters on Linux can be forced to receive ads in LongRange. But then they won't accept standard BLE.
There are also only a few implementations in smartphones that automatically accept standard BLE advertisements on different PHYs. Those. There is no implementation of Bluetooth 5.0 (6 December 2016) yet, although this is just a software solution.

Read the two posts above for more details.
PS: It is quite possible that this was done in order not to displace ZigBee.

[DienoX]

According to the data from the nRF Connect Android application, my device, i.e. the Samsung Galaxy S20, supports extended BLE range.

In this case, will setting a greater range from the thermometer allow me to connect over greater distances?

[pvvx]

According to the data from the nRF Connect Android application, my device, i.e. the Samsung Galaxy S20, supports extended BLE range.

It doesn't say anything.
All modern smartphones are capable of accepting "Extended Advertising" in any PHY format, but typical (for connection and polling) advertising is only on 1M PHY.
You didn't read the comment above:

Essentially making BLE long range per se practically useless since a peer always needs to be able to receive the standard mandatory 1M advertisements anyway, in order to negotiate a coded connection or learn about coded "extended" advertisements

First understand the standards...
The custom firmware supports LongRange on 1M PHY connection request and transition to LongRange.

[DienoX]

I'm very sorry. I read the comments but I just didn't understand them.
I just don't understand why there is no long range support for standard advertising.
It makes little sense to me not being able to connect with just a standard advertising.
This is probably due to my lack of very good knowledge of the Bluetooth standard.

The use of long-range Bluetooth has a lot of sense and applications.

[pvvx]

Almost every cheap smartphone from last year has the following options:

But that doesn't mean it will accept standard advertisements in different PHY formats.
To connect, you must accept standard advertisements on the 1M PHY. Only after a connection is established on the 1M PHY is it possible to switch the PHY format.
"Extended Advertising" is the transmission of a header on the 1M PHY that indicates on which channel and in what format the data itself will be transmitted. Those. without receiving on the 1M PHY, you will not know where and how to receive the LongRange message. Reception of standard advertising on the main channels in the longRange format is not implemented on most devices. And apparently this was done on purpose so as not to force ZigBee out of the market or for some other corporate purposes...
Even to receive "Extended Advertising" with LongRange in the "nRF Connect" program, you need to switch the options in the settings.
Work in BLE Long Range is programmatically limited on all user devices, including BT-USB adapters. And the point is not in the hardware implementation, but in ignoring the standard.

Enabling the BLE5.2 options in the custom firmware allows you to support "connection" on all PHY encoding options that your adapter requests. But the transmission of advertising on other than 1M PHYs in the current version is not implemented due to the lack of support from the reseived side.

[DienoX]

Thanks for the clarification. Is there any leaks or indications that bluetooth over longer distances will work better?

[pvvx]

https://novelbits.io/bluetooth-long-range-how-to-achieve-results-over-1km/
https://www.youtube.com/watch?v=w4PbxVwg83M

[hcgpalm]

After digging deeper into the subject, I do NOT agree that advertising on primary channels is limited to 1M PHY.
BLE 5.0 permits the ADV_EXT_IND PDU to be sent on the primary advertising channels using Coded PHY.
So it is true that it requires the use of extended advertising.

https://novelbits.io/bluetooth-low-energy-advertisements-part-1/

I also agree that it would significantly impact power requirements. Because S=8 requires 8x the radio time for the same volume of data, because of the overhead of extended advertisement, and because it still needs to do legacy advertisement i parallel.

Nevertheless, I'm tempted to try to cook up a proof-of-concept to explore this further, although I'm not sure I will be able to.

[DienoX]

If you managed to do something, please share it. Personally, I hope that bluetooth in the new versions will enable long range without extended advertising.

[pvvx]

Experimenting with the inclusion of LongRange for "Extended Advertising" on the main channels show that none of the available adapters in the application for 'HA' does not accept this option. Also checked dozens of smartphones in nRFConnect released that year - the situation is similar.
"Extended Advertising" transmitte with header on 1M PHY and data on S8 is successfully received by most (BT5.0+ adapters).

Further, the 'coded PHY' advertising option cannot be used immediately for connection and scannable.

And second:
Available SDK versions from Telink do not support multiple "Extended Advertising"

This is a BLE Spec standard interface，used to enable/ disable Extended Advertising，please refer to <Core_5.0> (Vol 2/Part E/7.8.56 “LE Set Extended Advertising Enable Command”), and understand it in the context of the SDK’s enumeration type definitions and demo usage. However, currently the SDK only supports 1 Adv Sets, so this API is not supported for the time being and is only reserved for multipleAdv sets in the future. However, the Telink SDK has written a simplified API based on this API function to operate on/off 1 Adv Sets for more efficient execution. The simplified API is shown below, with the same input parameters and return values as the standard API, but is only used to set 1 Adv
Set.

Using the library reverse, I got the ability to embed "Extended Advertising" into the "custom firmware" option. But such a version cannot support all current options and requires a separate implementation.
Also, the implementation of "Extended Advertising" in closed SDK libraries leads to unstable operation of the device as a whole. A normal implementation of the libraries or their open source is required.

---

As a result, the only option available to users to increase the range of reception and transmission remains the option of using not BLE, but the ESB protocol.
ESB (Enhanced ShockBurst) - in the common people it is "RF24". With settings on the BLE protocol...

[pvvx]

If you managed to do something, please share it. Personally, I hope that bluetooth in the new versions will enable long range without extended advertising.

This has not happened since the release of the BT5.0 standard in December 2016, although the hardware of 90% of the BT controllers already allowed it. :P

[DienoX]

Is this due to some conflict of interest?

[pvvx]

I do not know for sure. I can only guess...
The hardware of the chips does not care what synchronization to accept, since the modulation is the same for different PHY. The transmission frame header contains all the necessary information, and then the program's business.
And the message in the Coded PHY is simply discarded, since it does not match the software detector of the correct frame. Perceived as noise.

[DienoX]

Are there any open source drivers (I mean mostly Linux here) that have a patch made to make this work?

[pvvx]

On Linux, no patch is needed. You need to switch the adapter to encoded PHY and it will accept ads on LongRange. But others, standard BLE, will not be accepted.

Nobody is engaged in patches of internal software of adapters. It's easier to make your own BLE advertising message receiver or use a full-fledged sniffer.

[DienoX]

If I run an encoded Bluetooth PHY in Linux and then connect to a thermometer with your software, will I get a greater range? Of course in your firmware I will choose LongRange BLE.

I found such a way to run an encoded PHY. Do you think he is good? I can't find anything better on Google on this topic, but I'm probably looking in the wrong place.
apache/mynewt-nimble#893

[pvvx]

If I run an encoded Bluetooth PHY in Linux and then connect to a thermometer with your software, will I get a greater range? Of course in your firmware I will choose LongRange BLE.

Nothing will work - the program will switch the BT adapter to standard mode.

[pvvx]

Wariant : primary BLE_PHY_1M, secondary BLE_PHY_CODED

		blc_ll_setExtAdvParam(ADV_HANDLE0,
				ADV_EVT_PROP_EXTENDED_CONNECTABLE_UNDIRECTED,
				adv_interval, adv_interval + 10,
				BLT_ENABLE_ADV_ALL, // primary advertising channel map
				OWN_ADDRESS_PUBLIC, // own address type
				BLE_ADDR_PUBLIC, // peer address type
				NULL, // * peer address
				ADV_FP_NONE, // advertising filter policy
				TX_POWER_0dBm, // TODO: advertising TX power cfg.rf_tx_power
				BLE_PHY_1M, // primary advertising channel PHY type
				0, // secondary advertising minimum skip number
				BLE_PHY_CODED, // primary advertising channel PHY type
				ADV_SID_0,
				0); // scan response notify enable 
		// debug!!!  Fix CodedPHY channel
		blc_ll_setAuxAdvChnIdxByCustomers(20); // auxiliary data channel, must be range of 0~36

(Random T & H for test)

---

Wariant : primary BLE_PHY_CODED, secondary BLE_PHY_CODED
Not accepted on standard adapters.
nRF Sniffer:

[pvvx]

As a result, if LongRange is enabled, then it will be impossible to connect to this thermometer.
Switching format options on the fly does not allow the SDK and a lot of problems arise. Especially for ordinary users. It will raise a lot of questions and dissatisfaction :)

Similarly not accepted in 'HA'.

[pvvx]

I also agree that it would significantly impact power requirements. Because S=8 requires 8x the radio time for the same volume of data, because of the overhead of extended advertisement, and because it still needs to do legacy advertisement i parallel.

More than 8 times - advertising remains on the main channels + data, on another channel.
Full LongRange (Coded PHY S8, Coded PHY S8):

Legacy (1M PHY)

[hcgpalm]

Nothing will work - the program will switch the BT adapter to standard mode.

Could you please clarify - what "program" are you talking about here?

More than 8 times - advertising remains on the main channels + data, on another channel.

Yes, that's what I was trying to say.
My thought is that maybe there's some way to drastically reduce the coded PHY advertising interval?
Other long range/low power technologies like LoRaWAN, seems to use MUCH longer update intervals. Typically 5 minutes to an hour unless there's an event. Obviously for a good reason.

I agree that the device always needs to do 1M PHY advertising so that it's always visible and connectable for anyone. Switching advertising on primary channels to Coded PHY only would essentially brick it for most users.

Since Telink seems to only support 1 advertising set, would it be possible to, for example, make it do legacy 1M PHY advertising just like today and then periodically switch to extended advertising with Coded PHY on primary channels?

I.e. using blc_ll_setExtAdvParam() to set up extended advertising using coded PHY on both primary and secondary channels and then calling blc_ll_setExtAdvEnable_n() to switch extended advertising on/off periodically.

[pvvx]

Since Telink seems to only support 1 advertising set, would it be possible to, for example, make it do legacy 1M PHY advertising just like today and then periodically switch to extended advertising with Coded PHY on primary channels?

No. Connecting the ext_adv module does not provide for rollback.
This is not possible without reversing binary libraries. Some of which I have already done...

In addition, switching ad types in ext_adv causes glitches. There are also glitches when using ext_adv type and a single ext_adv configuration. Sometimes does not include ads after disconnect, etc. ext_adv = demo version :(

PS: As a result, everything is done in such a way as to exclude the operability of LongRange in BLE.

[pvvx]

Could you please clarify - what "program" are you talking about here?

About all implementations. Exception - DIY.

[pvvx]

And there is a second problem with ext_adv:
ext_adv cannot be 'connectable' and 'scanable' at the same time. Requires type switching or having two ext_adv variants at the same time, which is not implemented in the SDK. (Switching can be done through a cold boot with different options for initializing drivers. But this is an additional drain on the battery.)

---

Everything described can be bypassed or solved, but it takes time to reverse and debug. This, in the complete absence of information support, results in months ...

And one more problem. Xiaomi LYWSD03MMC has no buttons. As of today, I already have a working implementation with ext_adv for thermometers, including LongRange. But not for LYWSD03MMC.
The button temporarily switches the thermometer to the advertising option for the connection.

Also - the BLE implementation of HomeAssistant does not provide for temporary advertising or advertising only for changing data from sensors. After a built-in period, if there are no responses from the sensor, then the readings are transferred to the "Unavailable" state. This makes it impossible to save battery on the sensors or increase the period of transmission of BLE messages.

PS: As a result, everything is done in such a way as to exclude the operability of LongRange in BLE.
As a result, the main struggle occurs with third-party software, and not with the program for thermometers.
When third-party programs solved these issues, then it makes sense to enable LongRange support in thermometers for a general group of users.
And for today, only a DIY version is possible for those advanced in BLE matters. After some time (days) it will be in the source repository, but full support should not be expected. And most likely, support for the LongRange variant will be provided only in Russian. The English version will be when the blocking of information from the English side stops.

[pvvx]

Тестовая прошивка с поддержкой ‘Extended Advertising’ и ‘LongRange’ для LYWSD03MMC.
Файл https://github.com/pvvx/ATC_MiThermometer/blob/master/LR_v99.bin
На текущий момент эта прошивка не поддерживает опции связанные с герконом.
После сброса питания прошивка работает в стандартном режиме ‘1M PHY’.
Сброс и возобновление питания отключает опцию ‘Extended Advertising’ и ‘LongRange’.
Для включения режима ‘Extended Advertising’ в TelinkMiFlasher установите:

‘EXT.ADV’ без ‘LongRange’ включает передачу рекламы в формате LEGACY_CONNECTABLE_SCANNABLE_UNDIRECTED.

‘LongRange’ - включает передачу основной и дополнительной рекламы в формате ‘Coded PHY’ c S8 и варианте EXTENDED_CONNECTABLE_UNDIRECTED.

Далее следует убедиться, что приемная сторона поддерживает ‘Extended Advertising’ в режиме ‘LongRange’ и если нет – вытащить и вставить батарейку.

[DienoX]

Thank you very much for sharing the file.
Большое спасибо за то, что поделились файлом.

[pvvx]

Большое спасибо за то, что поделились файлом.

Исходники так-же обновлены с данными опциями.

[jagheterfredrik]

Any chance you could share your test code and procedure for flashing the Zigbee firmware onto a e.g. a CGG1? I'm interested in developing this further.

[pvvx]

Any chance you could share your test code and procedure for flashing the Zigbee firmware onto a e.g. a CGG1?

#221 (comment)

[jagheterfredrik]

I read the comment and I get that you don’t want to support Zigbee but being new to the platform it’d be great place to start.

Thank you for your hard work on the firmware!

[jagheterfredrik]

E.g. The two-stage OTA you mention: do you mean we need to first flash a minimal zigbee firmware with only OTA support (<128k over BLE OTA) and then flash >128k main zigbee firmware over zigbee OTA?

[pvvx]

С начала необходимо прошить малую прошивку BLE которая обрабатывает и очищает области Flash под OTA для кода Zigbee.
И только поле очистки Flash производится загрузка кода ZigBee в совершенно другую область, чем BLE прошивки.

Zigbee код самой минимальной версии имеет более 128 килобайт и не может быть загружен стандартной BLE OTA от Telink.

У Telink на сайте есть версия SDK tl_zigbee_ble_concurrent_sdk_8258. В ней поочередно работают оба варианта. Так же требует специальной OTA.

Из за большого объема прошивки Zigbee нет возможности сохранять историю измерений. И у большинства пользователей нет возможностей работы с Zigbee не имея специальных инструментов. Двойная загрузка вызовет массу ошибок у пользователей.
ZigBee не даст нормального увеличения дальности и времени работы батареи. LCD контроллер потребляет в среднем более половины тока от батарейки. Тем более транзакции ZigBee используют большие по продолжительности периоды потребления за 8 мА от батареи, что ещё сильнее скажется на продолжительности работы CR2032. А отсутствие очень большого конденсатора в цепи питания в Xiaomi LYWSD03MMC усугубит эту проблему.
По этим причинам я не хочу поддерживать Zigbee. Вариант с BLE LongRange более перспективен.

[DienoX]

Thanks @pvvx for the clarification, now I understand what you meant before.
Спасибо @pvvx за разъяснение, теперь я понимаю, что вы имели в виду раньше.

[jagheterfredrik]

Thanks! So a BLE firmware that sets up flash according to dual mode and with a dual mode OTA procedure, then flash a <192k zigbee firmware?

[pvvx]

http://wiki.telink-semi.cn/wiki/chip-series/TLSR825x-Series/
ZigBee Development Manual
Figure 26: “512k Flash space allocation chart”
Bluetooth LE Development Manual ...
Zigbee Bluetooth LE Concurrent Quick Guide
...

[jagheterfredrik]

… thanks again for your firmware …

[pvvx]

Для работы HA c LongRange желательно установить последний bluez и добавить в файле /lib/systemd/system/bluetooth.service
ExecStart=/usr/local/libexec/bluetooth/bluetoothd --experimental

Перед запуском HA необходимо переключить USB-BT5.2 адаптер на CodedPHY:
hcitool -i hci1 cmd 08 31 03 04 04
Номер hci - свой, желательно от второго адаптера, а первый пусть принимает стандартные рекламы...

[DienoX]

Thank you @pvvx for the useful post.
I would like to understand one thing from him. A Bluetooth 5.2 adapter is required for this to work, or is Bluetooth 5.0 enough?
Asking because I thought CodedPHY was introduced with BT5.0.

Спасибо @pvvx за полезный пост.
Я хотел бы понять от него одну вещь. Для этого требуется адаптер Bluetooth 5.2 или достаточно Bluetooth 5.0?
Спрашиваю, потому что я думал, что CodedPHY был представлен с BT5.0.

[pvvx]

Достаточно BT5.0
Low-cost Realtek RTL8761 BT adapter (current version of linux fw on RTL8761 supports BT5.1)

lsusb -s 5:2
Bus 005 Device 002: ID 0bda:8771 Realtek Semiconductor Corp. Bluetooth Radio

hcitool -i hci0 cmd 08 31 03 04 04

# btmgmt phy
Supported phys: BR1M1SLOT BR1M3SLOT BR1M5SLOT EDR2M1SLOT EDR2M3SLOT EDR2M5SLOT EDR3M1SLOT EDR3M3SLOT EDR3M5SLOT LE1MTX LE1MRX LE2MTX LE2MRX LECODEDTX LECODEDRX
Configurable phys: BR1M3SLOT BR1M5SLOT EDR2M1SLOT EDR2M3SLOT EDR2M5SLOT EDR3M1SLOT EDR3M3SLOT EDR3M5SLOT LE2MTX LE2MRX LECODEDTX LECODEDRX
Selected phys: BR1M1SLOT BR1M3SLOT BR1M5SLOT EDR2M1SLOT EDR2M3SLOT EDR2M5SLOT EDR3M1SLOT EDR3M3SLOT EDR3M5SLOT LECODEDTX LECODEDRX

# hciconfig -a
hci0:   Type: Primary  Bus: USB
        BD Address: 8C:88:2B:20:8B:42  ACL MTU: 1021:6  SCO MTU: 255:12
        UP RUNNING
        RX bytes:2020 acl:0 sco:0 events:178 errors:0
        TX bytes:23965 acl:0 sco:0 commands:178 errors:0
        Features: 0xff 0xff 0xff 0xfe 0xdb 0xfd 0x7b 0x87
        Packet type: DM1 DM3 DM5 DH1 DH3 DH5 HV1 HV2 HV3
        Link policy: RSWITCH HOLD SNIFF PARK
        Link mode: SLAVE ACCEPT
        Name: 'nanopineoplus2'
        Class: 0x000000
        Service Classes: Unspecified
        Device Class: Miscellaneous,
        HCI Version: 5.1 (0xa)  Revision: 0x999
        LMP Version: 5.1 (0xa)  Subversion: 0x646b
        Manufacturer: Realtek Semiconductor Corporation (93)

[pvvx]

Спасибо @pvvx за полезный пост. Я хотел бы понять от него одну вещь. Для этого требуется адаптер Bluetooth 5.2 или достаточно Bluetooth 5.0? Спрашиваю, потому что я думал, что CodedPHY был представлен с BT5.0.

BLUETOOTH SPECIFICATION Version 5.0 | Vol 1, Part C page 291
Core Specification Change History
9 CHANGES FROM v4.2 TO 5.0
9.1 NEW FEATURES
Several new features are introduced in the Bluetooth Core Specification 5.0
Release. The major areas of improvement are:
• Slot Availability Mask (SAM)
• 2 Msym/s PHY for LE
• LE Long Range
• High Duty Cycle Non-Connectable Advertising
• LE Advertising Extensions
• LE Channel Selection Algorithm #2

Да, с 2016 года так и продолжается "круговая порука" с BLE Long Range. Все ссылаются друг на друга.
Это типа "заговор" чтобы не дать народу использовать увеличение расстояния связи с BLE устройствами.
Примеры тут:
So, I think the issue is with the Bluetooth integration then, isn't it? Perhaps @bdraco can help with this?
Home Assistant Bluetooth support is entirely based on bleak so we only support what bleak supports.
Bleak not supports reading, writing and getting notifications from GATT servers, as well as a function for discovering BLE devices Bluetooth Core Specification Version 5.0 and next version?
И с моим знанием английского это бесполезно пробивать для пользователей.
Проще сделать версию только для русскоязычных... Для Android уже делается вариант с приемом всех форматов PHY одновременно на смартфоне...

Причина в том, что большинство не знает, что в варианте Long Range сильно увеличивается расстояние приёма-передачи для BLE. А думают по старинке, основываясь на опыте работы с устройствами BT4.2.

---

Последний ответ от 'Bleak' означает, что Home Assistant не будет поддерживать Bluetooth Specification 5.0 и далее.

[pvvx]

В версии 4.2 будет включен полный "LE Long Range" для всех поддерживаемых термометров.
Обновление будет через пару часов, как отредактирую описания и TelinkMiFlasher.html.