dir() inside class property/method causes board reset

[michalpokusa]

CircuitPython version

Adafruit CircuitPython 8.2.4 on 2023-08-22; Adafruit Feather ESP32-S2 TFT with ESP32S2
Board ID:adafruit_feather_esp32s2_tft
UID:487F305FFE63

Code/REPL

class SimpleClass:

    def __init__(
        self,
        param1: int,
        param2: str,
    ):
        self.param1 = param1
        self.param2 = param2

    @property
    def property1(self):
        print(dir(self)) # <-- This is the line that causes the board to crash
        return (self.param1, self.param2)

    def some_func(self):
        print(dir(self)) # <-- This is the line that causes the board to crash



input("Press Enter to start...")

simple_object = SimpleClass(1, "2")
# print(dir(simple_object)) # <-- If dir() called outside, no crash
simple_object.property1
simple_object.some_func()

Behavior

Board crashed every time it reaches the dir(self), no output, simply the sound of USB device disconnected and then after a few secodns connected again

Description

• if the dir() is inside a property or a method, it crashes the board
• if the dir() is called outside a class, it works as expected
• in CPython it works as expected

Additional information

No response

[todbot]

Weird. I just tried this on Adafruit CircuitPython 8.2.4 on 2023-08-22; Saola 1 w/Wroom with ESP32S2 and after typing in the class above in REPL paste mode, instantiating SimpleClass in the REPL, and doing dir() in a few ways, I get weird results, but no crash:

>>> simple_object = SimpleClass(1, "2")
>>> dir()
['simple_object', 'SimpleClass', '__name__', '__file__']
>>> dir(simple_object)
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
>>>
>>> simple_object.property1
[['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
(1, '2')
>>> simple_object.some_func()
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
['__class__', '__init__', '__module__', '__qualname__', '__dict__', 'param1', 'param2', 'property1', 'some_func']
>>>

[jepler]

This is probably a duplicate of

• Property/field mix-up leads to crash after recursion, safe mode restart #6524

There may be two separate problems.

First, dir() is implemented differently in CircuitPython than in desktop Python. In particular, calling dir() on an object actually retrieves all of the object's property values, which can have side effects. When a property getter calls dir() on the object, this creates an unbounded recursion. I did not find this explicitly documented anywhere so it may be that we should improve documentation in this area. This difference is inherited from Micropython and we aren't likely to lift it.

Recursion uses up a finite resource called the stack. When the stack is exhausted, this is supposed to be detected by an exception. In standard Python there's a dedicated type for this exception (RecursionError); in CircuitPython this is just a RuntimeError.. In an example where the same line is printed repeatedly, this occurs because the 'innermost' dir() has raised RecursionError, and then each recursive call to dir() can finish. (An exception other than AttributeError in a property makes dir() conclude the attribute is present, which is why the property in question is shown when some dir() results do get printed)

FWIW I reproduced this on Adafruit CircuitPython 9.0.0-alpha.1-9-g5c23e28208-dirty on 2023-09-04; Adafruit Camera with ESP32S3 and this causes a Hard Fault safe mode reset.

[jepler]

I found that when I increased the amount of 'grace area' in the stack check from 1kB to 2kB I didn't reproduce the problem anymore. There's no particular justification for this value that I could find; it depends how much "additional stuff" can be placed on the stack subsequent to a call to mp_stack_check and I don't think there's any simple way to calculate the maximum such value.

diff --git a/main.c b/main.c
index 94a3feac6b..d0727beb53 100644
--- a/main.c
+++ b/main.c
@@ -163,7 +163,7 @@ STATIC void start_mp(supervisor_allocation *heap, supervisor_allocation *pystack
     if (stack_bottom != NULL) {
         size_t stack_length = stack_get_length();
         mp_stack_set_top(stack_bottom + (stack_length / sizeof(uint32_t)));
-        mp_stack_set_limit(stack_length - 1024);
+        mp_stack_set_limit(stack_length - 2048);
     }
 
 

[bill88t]

In ljinux I have seen plenty of dir() weirdness too, from not being able to filter out stuff that .startwith("_") to just random exceptions.
I have put my dir()'s in a try: except: and never seen hardfaults.
I just blamed all the weirdness to the fact i run 20 exec()'s deep.
Didn't think to try on the repl.

[dhalbert]

First, dir() is implemented differently in CircuitPython than in desktop Python. In particular, calling dir() on an object actually retrieves all of the object's property values, which can have side effects. When a property getter calls dir() on the object, this creates an unbounded recursion. I did not find this explicitly documented anywhere so it may be that we should improve documentation in this area. This difference is inherited from Micropython and we aren't likely to lift it.

The corresponding MicroPython issue is micropython#4546.

At one point I looked at fixing this but it was not so easy.

Dupes:

• dir() calls properties when collecting their names #4171
• Property/field mix-up leads to crash after recursion, safe mode restart #6524 (as noted by @jepler)

We could consolidate this to a single issue but there is interesting info in all of them.

[dhalbert]

The root cause of this is #2179. Closing in favor of that issue.