Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BLE GATT server support to espressif port #5926

Closed
tannewt opened this issue Jan 26, 2022 · 24 comments · Fixed by #9222
Closed

Add BLE GATT server support to espressif port #5926

tannewt opened this issue Jan 26, 2022 · 24 comments · Fixed by #9222
Assignees
Labels
Milestone

Comments

@tannewt
Copy link
Member

tannewt commented Jan 26, 2022

ESP32, ESP32-S3 and ESP32-C3 can all do BLE.

Let's add _bleio support on top of NimBLE that the IDF uses for BLE.

Once it's added, we can then support the BLE workflow on these ESP chips.

@tannewt tannewt added enhancement espressif applies to multiple Espressif chips ble esp32-s3 esp32-c3 labels Jan 26, 2022
@tannewt tannewt added this to the Long term milestone Jan 26, 2022
dhalbert pushed a commit that referenced this issue Jan 27, 2022
Everything else should raise NotImplementedError.

First step in #5926
dhalbert pushed a commit that referenced this issue Jan 28, 2022
Services and characteristics still won't work.

Progress on #5926
dhalbert pushed a commit that referenced this issue Feb 17, 2022
This allows you to connect to GATT services on the other device.
It also adds connection initiation (GAP central).

More progress on #5926
@tannewt tannewt changed the title Add BLE support to espressif port Add BLE GATT server support to espressif port Mar 21, 2022
@tannewt
Copy link
Member Author

tannewt commented Mar 21, 2022

This is about half done. The two remaining large pieces are adding GATT server support and bonding support. Unfortunately GATT server support probably needs NimBLE modifications to allow for adding servers, characteristics and descriptors after things are running. The SoftDevice can do this so we've assumed it in our APIs ability to just Service() at any point.

@djbutler
Copy link

djbutler commented Nov 2, 2022

@tannewt - I'm a general programmer (not a BLE expert), but I might be interested in helping with this if it would be useful

@henrygab
Copy link

It sounds as though the current problems with BLE service support relate to dynamically adding services / characteristics / descriptors after things are running. IIRC, a similar limitation exists for USB HID devices. The solution for USB was to require things to be setup within boot.py.

Could a similar requirement to setup BLE in boot.py avoid the dynamic update problems you listed above, and thus enable BLE on these boards in an earlier milestone?

@tannewt
Copy link
Member Author

tannewt commented Oct 19, 2023

Unfortunately, BLE is implemented differently than USB. BLE is reset after every VM run and relies on BLE reconnection. USB starts and connects once.

The best way forward is to enhance the IDF's BLE library to support changing services like the BLE spec allows.

@henrygab
Copy link

Bummer! Still, I appreciate your responding. I now have five ESP32-S3 devices, which I have really wanted to used with CircuitPython for BLE HID (mouse, keyboard, joystick, etc.). I did not realize until a few days ago that it wouldn't work for my needs. I'll be watching this issue, although sad because of the likely long timeline involved.

Keep up the great work! I'm overall a very happy customer of Adafruit hardware!

@tannewt
Copy link
Member Author

tannewt commented Oct 23, 2023

Yup, I was bummed too. I started bleio implementation thinking it'd be straight forward and it wasn't. Hopefully we can circle back to it after 9.0 is out.

@Anutrix
Copy link

Anutrix commented Nov 5, 2023

https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-reference/bluetooth/esp_gatts.html

Just posting for reference.

Maybe we can list down what is missing, so work on IDF's side can start.

@tannewt
Copy link
Member Author

tannewt commented Nov 6, 2023

https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-reference/bluetooth/esp_gatts.html

Just posting for reference.

Maybe we can list down what is missing, so work on IDF's side can start.

That doc is referring to the bluedroid BLE stack. It does look like it can do dynamic services. Unfortunately, we started the existing BLE impl based on Nimble with the hope of standardizing ports to it for BLE host. So, switching to bluedroid could make this work. It isn't a simple task though.

@tannewt
Copy link
Member Author

tannewt commented Dec 1, 2023

Looks like dynamic service support has been added to ESP nimble and is in IDF 5.1.2: espressif/esp-idf@94ad8f1

@robholmes
Copy link

Looks like dynamic service support has been added to ESP nimble and is in IDF 5.1.2: espressif/esp-idf@94ad8f1

@tannewt I saw you managed to get IDF 5.1.2 into CircuitPython 9.0.0-alpha.6 🙇 so I went to give it a try and spotted that for the Seeed Xiao ESP32C3, it's now missing the _bleio module that was previously in 8.2.9 and is needed for BLE .

Is there anything I can do, I'm happy to raise a PR, but not sure if it's as simple as reverting this change, or there's more to it than that?

@tannewt
Copy link
Member Author

tannewt commented Jan 16, 2024

Looks like dynamic service support has been added to ESP nimble and is in IDF 5.1.2: espressif/esp-idf@94ad8f1

@tannewt I saw you managed to get IDF 5.1.2 into CircuitPython 9.0.0-alpha.6 🙇 so I went to give it a try and spotted that for the Seeed Xiao ESP32C3, it's now missing the _bleio module that was previously in 8.2.9 and is needed for BLE .

I did but I haven't improved bleio on ESP to use it.

Is there anything I can do, I'm happy to raise a PR, but not sure if it's as simple as reverting this change, or there's more to it than that?

Yeah, there's more work to do than just enabling it. It will take code space so it may not be re-enabled on boards with limited flash space.

@nrgbistro
Copy link

Don't want to rush this, but I'm wondering if there's anything I can do as a workaround to get bluetooth working on my esp32-s3 in any capacity. I'm an esp32 and circuitpython noob, so if I'm missing something obvious or if it cannot currently be done with circuitpython please let me know. Ideally, I want to set up my esp32 to act as a BLE peripheral, but can't seem to get that working due to this missing bleio library.

@dhalbert
Copy link
Collaborator

@nrgbistro There is no workaround. Now that ESP-IDF 5.1.2 has added dynamic service support to the nimble base, we are no longer blocked in terms of implementation. But we still have to write quite a bit of code.

@nrgbistro
Copy link

Understood, thanks for the quick update.

@shanecromer24
Copy link

Has anyone written the implementation?

@tannewt
Copy link
Member Author

tannewt commented Mar 20, 2024

No but its near the top of my list. I'm updating CP to IDF 5.2.1 before getting back to BLE.

@henrygab
Copy link

🎉 I'm super excited. The Reverse TFT feathers (ESP32-S2 and ESP32-S3) provide such a nice form factor ... but then I had to shelve those projects. I've been eagerly watching this thread for updates, and now ... excitement! Thank you, Scott!

@peterfox
Copy link

Second, I've been watching this thread for a while. Would happily contribute the cost of a well-earned beverage or two once it's there.

@tannewt tannewt self-assigned this Apr 23, 2024
@tannewt tannewt modified the milestones: Long term, 9.1.0 May 3, 2024
@henrygab
Copy link

henrygab commented May 8, 2024

Wow, that's a huge change. Thank you for keeping this alive, and the (obviously huge) effort to make ESP32 boards have full BLE capability in CircuitPython! I know it's not ready yet, but it's not too early to say "THANK YOU!"

@tannewt
Copy link
Member Author

tannewt commented May 13, 2024

Wow, that's a huge change. Thank you for keeping this alive, and the (obviously huge) effort to make ESP32 boards have full BLE capability in CircuitPython! I know it's not ready yet, but it's not too early to say "THANK YOU!"

:-) Feel free to test it out. It isn't perfect but testing would be helpful. Note, it doesn't have bonding and pairing support so HID may not fully work with another device.

@henrygab
Copy link

Ok ... Per your PR comments, seems only ESP32-S3 is likely to work yet?

A couple of questions:

  1. I searched both your source branch's Actions and the PR Add ESP BLE GATT server support #9222 actions, but neither one appears to have proceeded to build. Where would be the place to pull build artifacts for ... (pulls out shelved project) ... an ESP32-S3 Reverse TFT board?

  2. Should issues be filed at https://github.com/tannewt/circuitpython/issues instead of https://github.com/adafruit/circuitpython/issues until this is merged?

  3. Do you have a place to track the list of functional areas to be checked? (e.g., maybe a thread under https://github.com/tannewt/circuitpython/issues, where you keep the top post updated with the list?)

Looking forward to pointers to the early alpha build artifacts!

@tannewt
Copy link
Member Author

tannewt commented May 14, 2024

Ok ... Per your PR comments, seems only ESP32-S3 is likely to work yet?

A couple of questions:

1. I searched both your source branch's Actions and the PR [Add ESP BLE GATT server support #9222](https://github.com/adafruit/circuitpython/pull/9222) actions, but neither one appears to have proceeded to build.   Where would be the place to pull build artifacts for ... (pulls out shelved project) ... an [ESP32-S3 Reverse TFT](https://www.adafruit.com/product/5691) board?

I'll make this a top priority tomorrow when I have brain space to dive back into it. So, the PR artifacts will be the place.

2. Should issues be filed at https://github.com/tannewt/circuitpython/issues instead of https://github.com/adafruit/circuitpython/issues until this is merged?

PR comments are probably best. At some point I'll want to merge it even if it isn't perfect too.

3. Do you have a place to track the list of functional areas to be checked?  (e.g., maybe a thread under https://github.com/tannewt/circuitpython/issues, where you keep the top post updated with the list?)

PR is probably the best place too.

Looking forward to pointers to the early alpha build artifacts!

Thanks! Will hopefully get the CI going tomorrow.

@tannewt
Copy link
Member Author

tannewt commented May 15, 2024

PR Artifacts are here: https://github.com/adafruit/circuitpython/actions/runs/9103419532 Check the PR for the latest CI run too.

@henrygab
Copy link

Ok ... dipping my toes into this uncharted space. Seems to at least boot on the ESP32-S3 Reverse TFT. No code loaded yet, and I have to re-acquaint myself with CircuitPython again. I hope to make steady (if slow) progress. Will update on the PR when I hit roadblocks or success milestones.

dhalbert pushed a commit that referenced this issue May 17, 2024
Enable BLE where we can. Switch 4MB, non-USB board default partitioning
over to a single 2MB firmware bank. Old boards override this setting to
keep the same behavior.

This also adds alpha support for the ESP32-C2 (aka ESP8684).

Fixes #5926 and fixes #7170
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants