Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

esp32 s3 serial2 read out invalid value when data size is big #7027

Closed
1 task done
flybeyond opened this issue Jul 22, 2022 · 10 comments
Closed
1 task done

esp32 s3 serial2 read out invalid value when data size is big #7027

flybeyond opened this issue Jul 22, 2022 · 10 comments
Assignees
Labels
Resolution: Unable to reproduce With given information issue is unable to reproduce Type: Question Only question

Comments

@flybeyond
Copy link

Board

ESP32S3 Dev Module

Device Description

plain module

Hardware Configuration

GPIO 17 18 as serial2

Version

v2.0.4

IDE Name

Arduino

Operating System

N/A

Flash frequency

40Hz

PSRAM enabled

no

Upload speed

921600

Description

I use esp32 s3 to read data from a cat1 module. when the module send out big data size say 1000, the read out value will be wrong after 200 bytes to 0xff. Use Serial2.setRxBufferSize(2048) will not resolve the issue.
You can see the debug message, full of 0xff which is not correct.

Sketch

none

Debug Message

15:25:02.443 -> d3 00 d0 43 25 a6 6d 5d fc 62 3f 80 24 15 28 06 00 00 00 00 20 20 41 00 67 7e 77 e7 7a 3a 2a 42 12 32 8a 3a 4a 78 e8 ad 25 91 9d 02 aa cc 50 50 2b bb 
15:25:02.443 -> bb 73 00 ab 91 51 a2 b0 09 4d 12 c4 25 a2 4e 07 c5 97 85 ce 17 7c 2c f8 68 e6 75 17 52 a3 d7 3a 3d f4 e4 e9 4b a9 2b 54 2e a1 7f d5 df a4 bf 49 60 89 
15:25:02.443 -> c0 57 82 38 9d 39 2d 32 66 40 3a 81 09 03 64 34 7c 67 d0 d6 20 fb f2 07 a4 38 b4 40 67 ff d1 06 75 a8 ee 71 ea ce cf 9d d6 f6 e4 f1 01 e1 fb c7 67 e4 
15:25:02.443 -> f4 de 80 1d 04 cb ef f1 2b a3 92 39 7f 9c a1 7e 83 f6 fd d7 c7 e6 3b cf 28 ae bc 75 d1 f3 62 7c 25 30 e0 8d c1 02 42 3c fe a9 af f8 ff ff ff ff ff ff 
15:25:02.475 -> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 85 bf e2 b7 c3 70 72 0d ba ac 38 e1 c0 79 b9 81 ea d0 07 fb 83 da d4 bf 77 af be 09 6f 11 e0 6c 4a e1 
15:25:02.475 -> 01 34 12 3e ec e7 fb 73 eb e5 90 4f c3 cd 7d fe c5 f8 82 ef ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff fc 00 00 00 00 2d b6 ea 2b ba 9a ed a6 
15:25:02.475 -> cb 2e c3 08 e3 9a fc b3 be fb f0 d7 5b 6a b6 9a 31 ca eb 40 0f e7 79 0d 0a 0d 0a 4f 4b 0d 0a ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 
15:25:02.500 -> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 
15:25:02.500 -> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 
15:25:02.500 -> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 
15:25:02.500 -> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 
15:25:02.500 -> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 
15:25:02.500 -> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 
15:25:02.500 -> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

  • I confirm I have checked existing issues, online documentation and Troubleshooting guide.
@flybeyond flybeyond added the Status: Awaiting triage Issue is waiting for triage label Jul 22, 2022
@SuGlider SuGlider self-assigned this Jul 22, 2022
@SuGlider
Copy link
Collaborator

@flybeyond - How are you reading the data?

200 bytes is the size for the UART driver to copy UART FIFO data to the internal Arduino Serial ringbuffer.
In order to read, say 1,000 bytes in a single operation, it shall be done using Arduino Stream Class, with a time out, such as Serial.readBytesUntil(character, buffer, length) or Serial.readBytes(buffer, length)

https://www.arduino.cc/reference/en/language/functions/communication/serial/
https://www.arduino.cc/reference/en/language/functions/communication/serial/readbytes/
https://www.arduino.cc/reference/en/language/functions/communication/serial/readbytesuntil/

@SuGlider SuGlider added Type: Question Only question and removed Status: Awaiting triage Issue is waiting for triage labels Jul 24, 2022
@flybeyond
Copy link
Author

After checking source code, the issue is caused by setRxBufferSize will not change buffer size dynamically. It just changes a variable and the variable will take effect in construct function. Say, if the code looks like Serial.begin(); Serial.setRxBufferSize(1024); it will not work. And Serial.setRxBufferSize(1024); Serial.begin(); It will work. The behavior is not the same as other arduino devices. It's better to change for consistence.

@VojtechBartoska
Copy link
Contributor

maybe related? #6998

@flybeyond
Copy link
Author

@SuGlider I tried Serial.readBytes(buffer, length) to read big data, it takes very long time to complete. 699 bytes take about 1.3s.

09:06:21.341 -> data len 699 left len 0
09:06:21.341 -> Start to read bytes ...
09:06:22.060 ->
09:06:22.060 ->
09:06:22.710 -> End to read bytes ...

@flybeyond
Copy link
Author

flybeyond commented Jul 26, 2022

@VojtechBartoska they are not the same issue. In this case, the setRxBufferSize will not take effect after begin(). And in my test, set the rx buffer size to 2k. I found some data should not shown in read function, show up. Some data show in previous readout value. Highly suspect there are some memory corruption. I think this is a critical issue.

@flybeyond
Copy link
Author

Here I found a similar issue. Some one said uart reception of long data has issue for years. Is that true? Any plans to fix it ?
espressif/esp-idf#7478

@SuGlider
Copy link
Collaborator

SuGlider commented Aug 8, 2022

@flybeyond - I used this code and I can read 1024 bytes on UART 0, by sending data using Arduino Serial Monitor:

char uartBuff[1024];

// 128 Chars to send to UART:  SerialMonitor also send \n \r depending on the setup...
//0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@

void setup() {
  Serial.setRxBufferSize(1024);
  Serial.begin(115200);

  Serial.println("\nStart reading up to 1024 bytes...");
  size_t alreadyRead = 0;
  while (alreadyRead < sizeof(uartBuff)) {
    size_t s = 0;
    while (Serial.available() && alreadyRead < sizeof(uartBuff)) {
      uartBuff[alreadyRead++] = Serial.read();
      s++;
    }
    if (s) {
      Serial.printf("So far read %d bytes\n", s);
    }
    delay(50);
  }
  
  // dump data read 
  Serial.println("UART has read 1024 bytes");
  for (int i = 0; i < sizeof(uartBuff); i++) {
    if (!(i & 0x3f)) Serial.println();
    Serial.print(uartBuff[i]);
  }
}

void loop() {
  // put your main code here, to run repeatedly:
}

Ouput:

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371 
ets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1344
load:0x40078000,len:13864
load:0x40080400,len:3608
entry 0x400805f0

Start reading up to 1024 bytes...
read 360 bytes
read 664 bytes
UART has read 1024 bytes

0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#
0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@

@SuGlider
Copy link
Collaborator

SuGlider commented Aug 8, 2022

@flybeyond - Another test with an ESP32 sending data and an ESP32-S3 receiving data.

ESP32 sends data using Serial UART 0 port. Sends data no-stop.
ESP32-S3 reads data from Serial 2, GPIO 17, as the issue stated.
Communications goes at 115200 baudrate.
ESP32 GPIO 1 (Serial Tx pin) is connected to ESP32-S3 GPIO 17 (Serial 2 Rx pin) and both boards GND are also connected to each other (common ground).

ESP32 Data Sender Sketch:

// Sends 64 bytes block - non stop - no \n \r
char buff[] = "0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#0123456789abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@";
void setup() {
  Serial.begin(115200);
}

void loop() {
  Serial.write((const uint8_t *) buff, sizeof(buff) - 1);
  Serial.flush();
}

ESP32-S3 Data Receiver Sketch:

char uartBuff[1024];

void setup() {
  Serial2.setRxBufferSize(2048);
  Serial2.begin(115200, SERIAL_8N1, 17, 18);
  Serial.begin(115200);

  Serial.println("\nStart reading up to 1024 bytes...");
  delay(100);
  Serial2.flush(false); // flush RX & TX
  
  
  size_t alreadyRead = 0;
  while (alreadyRead < sizeof(uartBuff)) {
    size_t s = 0;
    while (Serial2.available() && alreadyRead < sizeof(uartBuff)) {
      uartBuff[alreadyRead++] = Serial2.read();
      s++;
    }
    if (s) {
      Serial.printf("So far read %d bytes\n", s);
    }
  }
  
  // dump data read 
  Serial.println("UART has read 1024 bytes");
  
  for (int i = 0; i < sizeof(uartBuff); i++) {
    if (!(i & 0x3f)) Serial.println();
    Serial.print(uartBuff[i]);
  }
}

void loop() {
  // put your main code here, to run repeatedly:
}

ESP3-S3 Output:

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x43c
load:0x403c9700,len:0xbec
load:0x403cc700,len:0x2a3c
SHA-256 comparison failed:
Calculated: 3611f330726ad4edf64f62982b0568332a5800bc59425cab90be5379db92c108
Expected: ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
Attempting to boot anyway...
entry 0x403c98d8

Start reading up to 1024 bytes...
So far read 120 bytes
So far read 120 bytes
So far read 120 bytes
So far read 120 bytes
So far read 120 bytes
So far read 120 bytes
So far read 120 bytes
So far read 120 bytes
So far read 64 bytes
UART has read 1024 bytes

89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@01234567
89abcdefghijklmnopqrstuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#01234567

@SuGlider SuGlider added the Resolution: Unable to reproduce With given information issue is unable to reproduce label Aug 8, 2022
@SuGlider
Copy link
Collaborator

SuGlider commented Aug 8, 2022

If we add a delay(2000) at this point:

  Serial.println("\nStart reading up to 1024 bytes...");
  delay(100);
  Serial2.flush(false); // flush RX & TX

  delay(2000);  // <<<<<===== Here... we force Serial2 to receive all 1024 at once in the "background" of this delay
  
  size_t alreadyRead = 0;

The output is:

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x43c
load:0x403c9700,len:0xbec
load:0x403cc700,len:0x2a3c
SHA-256 comparison failed:
Calculated: 3611f330726ad4edf64f62982b0568332a5800bc59425cab90be5379db92c108
Expected: ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
Attempting to boot anyway...
entry 0x403c98d8

Start reading up to 1024 bytes...
So far read 1024 bytes
UART has read 1024 bytes

tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$#0123456789abcdefghijklmnopqrs
tuwxyzABCDEFGHIJKLMNOPQRSTUWXYZ!*$@0123456789abcdefghijklmnopqrs

Which means that the Serial 2 Rx buffer has stored at least 1024 bytes at once in the meantime that the sketch was doing something else (delay(2000)).

So, everything works fine....

@VojtechBartoska
Copy link
Contributor

@flybeyond Can this be considered as solved?

@VojtechBartoska VojtechBartoska added the Resolution: Awaiting response Waiting for response of author label Sep 6, 2022
@SuGlider SuGlider closed this as completed Nov 5, 2022
@VojtechBartoska VojtechBartoska removed the Resolution: Awaiting response Waiting for response of author label Nov 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Resolution: Unable to reproduce With given information issue is unable to reproduce Type: Question Only question
Projects
None yet
Development

No branches or pull requests

3 participants