UART.read() MemoryError

[ventrue3000]

CircuitPython version

Adafruit CircuitPython 8.1.0-rc.0 on 2023-05-17; Teensy 4.0 with IMXRT1062DVJ6A

Code/REPL

# UART setup:
UART=busio.UART(rx=board.D0, tx=board.D1, baudrate=250000, timeout=0, receiver_buffer_size=1024)

# Code line that causes the error:
byte=UART.read(1)

# Complete function where it's used:
def UARTReceive():
	"""
	Receive Message via UART
	return:		String/None - Message if present, otherwise None
	"""

	Message=[]
	
	byte=UART.read(1)
		
	if byte==b'<':		# Check for Message beginning
		Message.append(chr(byte[0]))
		TimeoutStart=time.monotonic()
		while time.monotonic()<TimeoutStart+Settings['UARTLoopTimeout']:		# This loop is exited by returning the Message and exiting the function once it's been fully received
			if UART.in_waiting:
				byte=UART.read(1)
				Message.append(chr(byte[0]))
				if byte==b'>':		# Check for Message end
					# Return Message as string
					return ''.join(Message)
			else:
				continue
	else:
		# If beginning character is wrong, no valid Message is being received, so exit function	
		return None

# Same function with try...excepts to catch it:
def UARTReceive():
	"""
	Receive Message via UART
	return:		String/None - Message if present, otherwise None
	"""

	Message=[]
	
	try:		# Catch a rare memory error
		byte=UART.read(1)
	except MemoryError:
		byte=None
		if Settings['VerboseOutput']==True: print (f"{PrintColour['BGRed']}{PrintColour['FGWhite']}UART.read() caused a memory error at code point 1!{PrintColour['Reset']}")
		return None
	
	if byte==b'<':		# Check for Message beginning
		Message.append(chr(byte[0]))
		TimeoutStart=time.monotonic()
		while time.monotonic()<TimeoutStart+Settings['UARTLoopTimeout']:		# This loop is exited by returning the Message and exiting the function once it's been fully received
			if UART.in_waiting:
				try:		# Catch a rare memory error
					byte=UART.read(1)
					Message.append(chr(byte[0]))
					if byte==b'>':		# Check for Message end
						# Return Message as string
						return ''.join(Message)
				except MemoryError:
					byte=None
					if Settings['VerboseOutput']==True: print (f"{PrintColour['BGRed']}{PrintColour['FGWhite']}UART.read() caused a memory error at code point 2!{PrintColour['Reset']}")
					return None
			else:
				continue

	else:
		# If beginning character is wrong, no valid Message is being received, so exit function	
		return None

Behavior

One of two things occasionally happens:

• A MemoryError like this: MemoryError: memory allocation failed, allocating 880342 bytes for the line marked in the code snippet. The allocation amount varies, but is always quite large and larger then the amount of free memory.
• The program crashes completely without any errors.

Both can be caught with a try...except for the marked line and then the program continues normally. I feel like the program crashes much more often without the try...except than the amount of errors that would be caught, but I may be imagining that.

Description

No response

Additional information

Might be specific to the Teensy 4.0. I'm using mostly the same code on a Pico W with CP 8.0.5 and haven't seen any issues there.

[tannewt]

Definitely sounds like a bug on iMX RT.

[bill88t]

880kb is an absurd amount of data. I can't even begin to fathom what kind of bug caused such an allocation.

[jepler]

we tried to correct a related problem in #5541 -- in that case, it was reported that the returned value could be negative. It looks like there must still be a case where the returned value can unexpectedly be an erroneously positive number instead.

I think #5541 is probably a mistaken change; the comment above the original return statement outlines why the result of LPUART_TransferGetReceiveCount was not used to give the amount of data actually transferred.

LPUART_TransferGetReceiveCount can fail, returning kStatus_NoTransferInProgress (not checked in code) and not assigning to its *count output-parameter. However, as this was initialized to 0 it would not lead to the problem discussed here.

It might be worth copying the content of LPUART_TransferAbortReceive from the sdk and adding code to return the count without any data races. The function could be called LPUART_TransferAbortReceiveGetCount and looks straightforward to implement (just copy the count from the internal structure before zero'ing it, then return it).

It's funny-sad how the sdk comment refers to a field that doesn't seem to exist:

 * brief Aborts the interrupt-driven data receiving.
 *                                                                       
 * This function aborts the interrupt-driven data receiving. The user can get the remainBytes to find out
 * how many bytes not received yet.

[tannewt]

Please retry this with absolute latest and document what characters you send in. The example code also looks incomplete because it doesn't loop. Thanks!