Skip to content
This repository has been archived by the owner on Jan 21, 2025. It is now read-only.

Refactor code - replace DYI structs with STL objects #39

Merged
merged 11 commits into from
Jun 27, 2024

Conversation

vortigont
Copy link
Collaborator

obsolete all local LinkedList containers, replaced with std::list
replace self-implemented mutex structures with std:mutex

@mathieucarbou take you time to get through this
This is WIP, I'm currently testing it, but I do not have any heavy-running setups, so any feedback would be appreciated

@mathieucarbou
Copy link
Owner

Thanks!

I have several apps with a lot of websocket usage so I will be able to test, but once you will be ready, please make sure the PR is rebased on top of main, and build pass first before testing :-)

Use STL's platform implementation for mutex locks
replace container with an std::list of unique pointers
use a list of unique_ptrs for _clients container
-    LinkedList<AsyncWebRewrite*> _rewrites;
+    std::list<std::shared_ptr<AsyncWebRewrite> > _rewrites;
repalce LinkedList with std::list<std::unique_ptr<AsyncWebHandler> > _handlers;
@vortigont
Copy link
Collaborator Author

yeah, I needed your CI check aruduino and 8266 builds, tnx.
Will update with some small fixes

@mathieucarbou
Copy link
Owner

@vortigont : i've looked at all the changes: that's pretty cool what you did with the lock class removal also by just applying the mutex for esp32. I love that! Everything is fine for me. I will let you convert the PR back from draft when you'll have finished on your side.

I am deploying the PR now in 2 app to test...

Note: I won't be able to test the SSE part, but looking at the changes, there's no big risk...

src/ESPAsyncWebServer.h Outdated Show resolved Hide resolved
@vortigont
Copy link
Collaborator Author

SSE part

I've never used that also. Not sure this has any real-life scenarios once WebSockets were introduced.
Removing home-brewed LinkedList were a bit more comlicated actually. There are discrepancies with the ownership of the pointed objects that is impossible to resolve without braking the API. So those things are migrated to a new design also, but I'm not that happy with that. User could easily shot his own foot with this. I guess it's one of the reasons for so many complains to "instabilities" in AsyncServer

@mathieucarbou
Copy link
Owner

@vortigont : please see my updated comment: #39 (comment)

I think that is fine for the const, I will just indicate that in the release note and move the major digit version since this is still a breaking change. Better do it now than later.

Let me know when the PR will be tested on your side and ready to be merged. I already reviewed and tested on my side, all is OK.

@mathieucarbou mathieucarbou marked this pull request as ready for review June 27, 2024 07:13
mathieucarbou
mathieucarbou previously approved these changes Jun 27, 2024
@vortigont
Copy link
Collaborator Author

Sure! Will add some cosmetic changes by tomorrow

@vortigont
Copy link
Collaborator Author

vortigont commented Jun 27, 2024

@mathieucarbou
almost same situation is with AsyncWebHeader* AsyncWebServerRequest::getHeader(const String& name) methods. All of AsyncWebHeader 's methods are const, well, except copy constructor. I guess there is no reason to return non-const ptr to it unless somebody is going to modify assigned header in request via copy :)
Should I also remove non-const getHeader versions? What do you think?

@mathieucarbou
Copy link
Owner

mathieucarbou commented Jun 27, 2024

getHeader(const String

AsyncWebHeader is an immutable object so yes its the same...

All these look weird lol':

    AsyncWebHeader* getHeader(const String& name);
    const AsyncWebHeader* getHeader(const String& name) const;
    AsyncWebHeader* getHeader(const __FlashStringHelper * data);
    const AsyncWebHeader* getHeader(const __FlashStringHelper * data) const;
    AsyncWebHeader* getHeader(size_t num);
    const AsyncWebHeader* getHeader(size_t num) const;

and const specifiers for immutable returns
make &String arg overloads over const char*
+ some minor cleanups
@vortigont
Copy link
Collaborator Author

I think that's it. Enough for one PR :)

@mathieucarbou mathieucarbou merged commit c79f2d0 into mathieucarbou:main Jun 27, 2024
8 checks passed
@mathieucarbou
Copy link
Owner

mathieucarbou commented Jun 27, 2024

@vortigont : FYI I had quite a few issues when trying to include the release in some other projects.

Most of them being linked that beginResponse_P was removed for ESP32. This method was heavily used by ESP32 libs relying on ESP-IDF or PIO embedded binary data (see explanation here: https://github.com/mathieucarbou/ESPAsyncWebServer/releases/tag/v3.0.2).

So I added an equivalent overload for ESP32.

This still breaks the API, but at least users can just remove the _P to migrate.

But this caused me hours of headaches because I add to update / release more than 10 projects / libraries.

If we have some complains, we will need to put back beginResponse_P for ESP32 too but mark it deprecated at least.

That's the only issue I saw.

Also, I added to library.json (just in main, not released yet) "libCompatMode": "strict". I've merged the RP2040 compat' that was in a dev branch, and if we do not use strict in CI and pio file, the lib from RPI comes in the project and create compilation issues.

I updated the CI to add this lib_compat_mode = strict flag in my projects and libraries, but when doing that I thought it would be better t put it directly in the library.json here.

@vortigont
Copy link
Collaborator Author

Great, thanks!
yeah, those _P methods just drives me crazy. Arduino core were trying to make it compatible with some defines, but anyway it looks ugly. So they ended up with declaring __FlashStringHelper into abstract method and now all those _P and F stuff results in compiler wrapping RO const char* via String's. Hate it!
OK, let's see, I think deprecation warning with wrapper in a worst case scenario is a good idea actually.

I also want to refactor all those static char strings with mime-types, etc... into separate units for ESP32 and 8266 to get rid of those F() wrappers and make it available in user code to reuse. Otherwise I have to duplicate those mime strings again in my libs.
Will come with another PR a bit later.

@mathieucarbou
Copy link
Owner

mathieucarbou commented Jun 28, 2024

@vortigont : FYI have opened #46 to put back these methods but with a deprecation notice.

Would you be able to have a look quickly before I merge ?

Thanks!

@vortigont
Copy link
Collaborator Author

Sure, let me check it and get back!

@mathieucarbou
Copy link
Owner

mathieucarbou commented Jun 29, 2024

@vortigont : I ran into a stack trace today, linked to the object destructions:

Backtrace: 0x40083f05:0x3ffe5270 0x4008ecb5:0x3ffe5290 0x400945b2:0x3ffe52b0 0x4009373d:0x3ffe53e0 0x40084292:0x3ffe5400 0x400945f1:0x3ffe5420 0x40116345:0x3ffe5440 0x40116355:0x3ffe5460 0x400e5147:0x3ffe5480 0x400e527e:0x3ffe54a0 0x400e5289:0x3ffe54c0 0x400f9bce:0x3ffe54e0 0x400fbca5:0x3ffe5500 0x400f6cf6:0x3ffe5520 0x400f6d01:0x3ffe5540 0x400f6d29:0x3ffe5560 0x401c122b:0x3ffe5580 0x401c156a:0x3ffe55b0 0x401c1580:0x3ffe55d0 0x401c1a77:0x3ffe55f0 0x401c1acd:0x3ffe5610 0x40091506:0x3ffe5640
  #0  0x40083f05 in panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c:466
  #1  0x4008ecb5 in esp_system_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/port/esp_system_chip.c:84
  #2  0x400945b2 in __assert_func at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/assert.c:81
  #3  0x4009373d in multi_heap_free at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/heap/multi_heap_poisoning.c:276 (discriminator 1)
  #4  0x40084292 in heap_caps_free at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/heap/heap_caps.c:388
  #5  0x400945f1 in free at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/heap.c:39
  #6  0x40116345 in String::invalidate() at /Users/mat/.platformio/packages/framework-arduinoespressif32@src-63eea845228ecc62b8fd85f819107145/cores/esp32/WString.cpp:160
  #7  0x40116355 in String::~String() at /Users/mat/.platformio/packages/framework-arduinoespressif32@src-63eea845228ecc62b8fd85f819107145/cores/esp32/WString.cpp:144
  #8  0x400e5147 in AsyncAbstractResponse::~AsyncAbstractResponse() at .pio/libdeps/pro-esp32-debug/ESP Async WebServer/src/WebResponseImpl.h:45
  #9  0x400e527e in AsyncJsonResponse::~AsyncJsonResponse() at .pio/libdeps/pro-esp32-debug/ESP Async WebServer/src/AsyncJson.h:123
  #10 0x400e5289 in AsyncJsonResponse::~AsyncJsonResponse() at .pio/libdeps/pro-esp32-debug/ESP Async WebServer/src/AsyncJson.h:123
  #11 0x400f9bce in AsyncWebServerRequest::~AsyncWebServerRequest() at .pio/libdeps/pro-esp32-debug/ESP Async WebServer/src/WebRequest.cpp:55 (discriminator 1)
  #12 0x400fbca5 in AsyncWebServer::_handleDisconnect(AsyncWebServerRequest*) at .pio/libdeps/pro-esp32-debug/ESP Async WebServer/src/WebServer.cpp:128 (discriminator 1)
  #13 0x400f6cf6 in AsyncWebServerRequest::_onDisconnect() at .pio/libdeps/pro-esp32-debug/ESP Async WebServer/src/WebRequest.cpp:218
  #14 0x400f6d01 in AsyncWebServerRequest::AsyncWebServerRequest(AsyncWebServer*, AsyncClient*)::{lambda(void*, AsyncClient*)#3}::operator()(void*, AsyncClient*) const at .pio/libdeps/pro-esp32-debug/ESP Async WebServer/src/WebRequest.cpp:41
  #15 0x400f6d29 in std::_Function_handler<void (void*, AsyncClient*), AsyncWebServerRequest::AsyncWebServerRequest(AsyncWebServer*, AsyncClient*)::{lambda(void*, AsyncClient*)#3}>::_M_invoke(std::_Any_data const&, void*&&, AsyncClient*&&) at /Users/mat/.platformio/packages/toolchain-xtensa-esp32/xtensa-esp32-elf/include/c++/12.2.0/bits/invoke.h:61
      (inlined by) __invoke_r<void, AsyncWebServerRequest::AsyncWebServerRequest(AsyncWebServer*, AsyncClient*)::<lambda(void*, AsyncClient*)>&, void*, AsyncClient*> at /Users/mat/.platformio/packages/toolchain-xtensa-esp32/xtensa-esp32-elf/include/c++/12.2.0/bits/invoke.h:111
      (inlined by) _M_invoke at /Users/mat/.platformio/packages/toolchain-xtensa-esp32/xtensa-esp32-elf/include/c++/12.2.0/bits/std_function.h:290
  #16 0x401c122b in std::function<void (void*, AsyncClient*)>::operator()(void*, AsyncClient*) const at /Users/mat/.platformio/packages/toolchain-xtensa-esp32/xtensa-esp32-elf/include/c++/12.2.0/bits/std_function.h:591
  #17 0x401c156a in AsyncClient::_fin(tcp_pcb*, signed char) at .pio/libdeps/pro-esp32-debug/Async TCP/src/AsyncTCP.cpp:965
  #18 0x401c1580 in AsyncClient::_s_fin(void*, tcp_pcb*, signed char) at .pio/libdeps/pro-esp32-debug/Async TCP/src/AsyncTCP.cpp:1377
  #19 0x401c1a77 in _handle_async_event(lwip_event_packet_t*) at .pio/libdeps/pro-esp32-debug/Async TCP/src/AsyncTCP.cpp:167
  #20 0x401c1acd in _async_service_task(void*) at .pio/libdeps/pro-esp32-debug/Async TCP/src/AsyncTCP.cpp:199
  #21 0x40091506 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:162

Which led me to this line:

https://github.com/mathieucarbou/ESPAsyncWebServer/blob/main/src/AsyncJson.h#L123

Not sure it is the culprit, but what do you think about removing it ? There is no destructor nowhere in the super classes or in the sub classes anymore. I don't know why only this class would have a virtual destructor and not the other ones. And ~AsyncAbstractResponse() does not exist in fact.

Note: the stack trace above iis really hard to reproduce - I need to stress the web server a bit directly at startup.

@vortigont
Copy link
Collaborator Author

that's pretty weird. There only String I see in AsyncAbstractResponse is String _head;, doesn't look like a weak spot.
Empty d-tor ~AsyncJsonResponse() {} could be removed, compiler should deal with it I guess. Not sure how trace will look like if that is not defined :)) Try it )

@mathieucarbou
Copy link
Owner

@vortigont : FYI reminder for next time ;-)
This _P method changed slipped in my review.
ayushsharma82/ElegantOTA#201 (comment)

@vortigont
Copy link
Collaborator Author

Ugh... people are still using 8266's code on esp32, need to push them somehow anyway, bit by bit :)
OK, I'm working now on redefining all those inlined F() things to const chars*, at least for server's code it wi=ould be free from ambiguity. I hope so...

@mathieucarbou
Copy link
Owner

mathieucarbou commented Jun 30, 2024 via email

@mathieucarbou
Copy link
Owner

An other second one is the char* equivalent of the String one... So a mere overload like we have elsewhere.

So except the macro use for const char*, there is nothing related to 8266.

@vortigont
Copy link
Collaborator Author

oh, yeah, now I got it. So some other's code were using _P to send that embedded bin data, I see.
I would rather cast it to cost char in my code then touch those _P's, ha-ha :)))

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants