[BUG]

[notixor]

arduino ide
esp32

when using sse - server side events, a reboot occurs when the connection is broken - the client is lost.

!!! enter AsyncEventSource::send !!!
   --> enter generateEventMessage !!!
   <-- exit generateEventMessage !!!
      ->> enter AsyncEventSource::send     c->write !!!
E (134045) task_wdt: Task watchdog got triggered. The following tasks/users did not reset the watchdog in time:
E (134045) task_wdt:  - async_tcp (CPU 1)
E (134045) task_wdt: Tasks currently running:
E (134045) task_wdt: CPU 0: IDLE0
E (134045) task_wdt: CPU 1: IDLE1
E (134045) task_wdt: Aborting.
E (134045) task_wdt: Print CPU 1 backtrace

Backtrace: 0x4008a4c3:0x3ffbccc0 0x400ea1aa:0x3ffbcce0 0x4008cd07:0x3ffbcd00 0x4008e896:0x3ffbcd20

ELF file SHA256: 21d9c42caab72306

Rebooting...
ets Jul 29 2019 12:21:46

code from AsyncEventSource.cpp

void AsyncEventSource::send(
  const char* message, const char* event, uint32_t id, uint32_t reconnect) {
  Serial.println("!!! enter AsyncEventSource::send !!!");
  String ev = generateEventMessage(message, event, id, reconnect);
#ifdef ESP32
  std::lock_guard<std::mutex> lock(_client_queue_lock);
#endif
int i =0;
  for (const auto& c : _clients) {
    i++;
    if (c->connected()) {
      Serial.println("      ->> enter AsyncEventSource::send     c->write !!!");

      c->write(ev.c_str(), ev.length());
            Serial.println("      <-- exit AsyncEventSource::send     c->write !!!");

    }
    Serial.println(i);

  }
  Serial.println("!!! exit AsyncEventSource::send !!!");

}

I think due to the features of sse, check
if (c->connected()) {
incorrect and subsequent code execution
c->write(ev.c_str(), ev.length());
leads to infinite wait and reboot via WDT

[notixor]

Let me clarify: if the client disconnects himself, then everything is fine, but if you disconnect the wifi, then the client is not disconnected because it is sse, but you CANNOT send him messages!

[mathieucarbou]

thanks! will have a look.

[mathieucarbou]

OK I think I might have found the issue... In a PR I've merged containing a bunch of fixes for AsyncTCP (me-no-dev/AsyncTCP#173), there's a busy loop:

Around line 1100 in size_t AsyncClient::write(const char* data, size_t size, uint8_t apiflags) { in AsyncTCP.cpp

    while (connected() && !send()) {
        taskYIELD();
    }

Can you try removing it locally and test again ? It is not there in the original repo.

Please let me know...

[mathieucarbou]

Also, please let me know how to reproduce (could you please paste a little code to reproduce).
Right now, with the code from yesterday (https://github.com/mathieucarbou/ESPAsyncWebServer/blob/6b98ada46f1108ec54bc98de38bea69b931295ce/examples/issues/Issue14/Issue14.ino) even if I update it to connect to a WiFi, I cannot reproduce,
I connect to a SSID, then turn the WiFi of the router box off, ESP does not crash.

  WiFi.mode(WIFI_STA);
  WiFi.begin("IoT");

[notixor]

i use your code from (https://github.com/mathieucarbou/ESPAsyncWebServer/tree/main/examples/issues/Issue14)
arduino ide
esp32 boards 3.0.2
tested on esp-wroom-32

[mathieucarbou]

i use your code from (https://github.com/mathieucarbou/ESPAsyncWebServer/tree/main/examples/issues/Issue14) arduino ide esp32 boards 3.0.2 tested on esp-wroom-32

I have the same setup... weird... You reproduce with the code in Issue14 directly or with some modifs ? It is using AP mode (i also tested in AP mode and cut my computer WiFi).

[notixor]

I commented out the loop block and now esp doesn't crash instantly. BUT firstly the code begins to grow endlessly
<<< exit AsyncEventSourceClient::_runQueue() i.send !!!

enter AsyncEventSourceClient::_runQueue() i.send !!!
<<< exit AsyncEventSourceClient::_runQueue() i.send !!!
enter AsyncEventSourceClient::_runQueue() i.send !!!
<<< exit AsyncEventSourceClient::_runQueue() i.send !!!
enter AsyncEventSourceClient::_runQueue() i.send !!!
<<< exit AsyncEventSourceClient::_runQueue() i.send !!!
enter AsyncEventSourceClient::_runQueue() i.send !!!
<<< exit AsyncEventSourceClient::_runQueue() i.send !!!
enter AsyncEventSourceClient::_runQueue() i.send !!!
<<< exit AsyncEventSourceClient::_runQueue() i.send !!!

this is for a DISCONNECTED client, and so on ad infinitum until wdt works again.

Another interesting thing is that if you reconnect, the server does not recognize the client as already connected and creates a new one :) as a result, messages are sent to 2 clients

[notixor]

(https://github.com/mathieucarbou/ESPAsyncWebServer/tree/main/examples/issues/Issue14) without modification

[mathieucarbou]

I hope we'll find out what's wrong here.
Not being able to reproduce is not helping me find any issue, and I do not have any SSE use cases (I am only heavily using the websocket).
I am using PIO and vscode myself (it helps avoid any dependency conflict).
For the queue being full, this is normal.
For the client cleanup, the websocket part has a cleanup procedure.
When I tested, after wifi reconnects, the browser was recovering and the loop was sending events normally. I only had 1 event per second, 1 client.

[mathieucarbou]

@notixor : were you able to test the fix above ? Removing te busy while loop...

I pushed the removal in the dev branch if you want to download the zip directly: https://github.com/mathieucarbou/AsyncTCP/tree/dev

[mathieucarbou]

@notixor : finally found the bug. Thanks to your analysis, I was suspecting this busy loop that was introduced in AsyncTCP as part of a PR containing several fixes (me-no-dev/AsyncTCP#173).

I have modified the example (https://github.com/mathieucarbou/ESPAsyncWebServer/blob/main/examples/issues/Issue14/Issue14.ino) to add some prints before and after send(), and also added some prints in this AsyncTCP loop.

So when the WiFi was turned off, AsyncTCP was looping forever in this busy loop, and the program was stalled, the print following the sending of the SSE was not executed.

I reverted back to the original code this part.

You can use:

  mathieucarbou/AsyncTCP @ 3.2.3
  mathieucarbou/ESPAsyncWebServer @ 3.1.1

It should work now.

FYI, the example send events at a rate of 1 sec, but pay attention that if you increase the rate, i.e. to 200ms, you need to rework your program to only send events form the main loop only, and just set some states in the handlers.