Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dir() calls properties when collecting their names #4171

Closed
caternuson opened this issue Feb 10, 2021 · 17 comments
Closed

dir() calls properties when collecting their names #4171

caternuson opened this issue Feb 10, 2021 · 17 comments

Comments

@caternuson
Copy link

Re this thread:
https://forums.adafruit.com/viewtopic.php?f=58&t=175096

Some weirdness trying to use dir() with the CPX. Can recreate it:

Adafruit CircuitPython 6.1.0 on 2021-01-21; Adafruit CircuitPlayground Express with samd21g18
>>> from adafruit_circuitplayground import cp
>>> type(cp)
<class 'Express'>
>>> dir(cp)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "adafruit_circuitplayground/express.py", line 82, in _unsupported
NotImplementedError: This feature is not supported on Circuit Playground Express.
>>> 

It's not a simple issue with frozen modules, as trying with something else works fine:

Adafruit CircuitPython 6.1.0 on 2021-01-21; Adafruit CircuitPlayground Express with samd21g18
>>> import adafruit_lis3dh
>>> dir(adafruit_lis3dh)
['__class__', '__file__', '__name__', 'const', 'digitalio', 'math', 'namedtuple', 'struct', 'time', '__version__', '__repo__', 'LIS3DH_I2C', 'RANGE_8_G', 'RANGE_16_G', 'RANGE_4_G', 'RANGE_2_G', 'DATARATE_1344_HZ', 'DATARATE_400_HZ', 'DATARATE_200_HZ', 'DATARATE_100_HZ', 'DATARATE_50_HZ', 'DATARATE_25_HZ', 'DATARATE_10_HZ', 'DATARATE_1_HZ', 'DATARATE_POWERDOWN', 'DATARATE_LOWPOWER_1K6HZ', 'DATARATE_LOWPOWER_5KHZ', 'STANDARD_GRAVITY', 'AccelerationTuple', 'LIS3DH', 'LIS3DH_SPI']
>>> 

The imported cp otherwise works fine:

>>> cp.pixels.fill(0xADAF00)
>>> cp.play_tone(440, 1)
>>> 

got expected lit NeoPixels and a tone. Even tab completion is working.

@jfurcean
Copy link

I was able to reproduce this error. Commenting out the raise exception in _unsupported in adafruit_circuitplayground.express allowed for it to operate as expected.

    @property
    def _unsupported(self):
        """This feature is not supported on Circuit Playground Express."""
        raise NotImplementedError(
            "This feature is not supported on Circuit Playground Express."
        )
    @property
    def _unsupported(self):
        """This feature is not supported on Circuit Playground Express."""
        pass
        #raise NotImplementedError(
        #    "This feature is not supported on Circuit Playground Express."
        #)

@kattni
Copy link

kattni commented Feb 10, 2021

Thank you for reproducing this. I assumed it was something connected to how we were handling unsupported features. I would like to figure out why it's happening, and sort out a way to still provide an error for the unsupported features.

@jfurcean
Copy link

I am not sure what causes it but you can fix this issue by adding a __dir__ function to this class and return the dir(super()). This is a little bit of a hack, but since this function doesn't have any other methods I think it should work.

def __dir__(self): 
        return dir(super())

@caternuson
Copy link
Author

I'd be interested in knowing why dir() has this behavior also.

But, in terms of supporting this a different way, you could have stubbed versions of these functions added to circuit_playground_base that raise the exception. Boards that support the function would override and implement. Others would do nothing, in which case the base stub would get called and raise the exception.

@jfurcean
Copy link

I think it has something to do with it being a property, but I don't know why. I was able to reproduce this if I commented everything out related to the _unsupported property and created a new property that raised a different exception.

@caternuson
Copy link
Author

Oh, good call. I've gotten so used to @property I don't even notice it much now. Hmmmm....why is it a property? For the items that get remapped:

    sound_level = _unsupported
    loud_sound = _unsupported
    play_mp3 = _unsupported

the last two are functions.

Simply commenting out the @property line gets around it also:

Adafruit CircuitPython 6.2.0-beta.1-194-gf6603aa56-dirty on 2021-02-10; Adafruit CircuitPlayground Express with samd21g18
>>> from adafruit_circuitplayground import cp
>>> dir(cp)
['__class__', '__dict__', '__init__', '__module__', '__qualname__', 'gamepad', 'temperature', 'acceleration', 'shake', 'tapped', '_int1', '_i2c', '_audio_out', 'sound_level', 'loud_sound', 'play_mp3', '_sample', 'stop_tone', '_speaker_enable', 'light', 'detect_taps', '_touch', 'touch_A1', 'touch_A2', 'touch_A3', 'touch_A4', 'touch_A5', 'touch_A6', 'touch_TX', 'adjust_touch_threshold', 'pixels', 'button_a', 'button_b', 'were_pressed', 'switch', 'red_led', '_sine_sample', '_generate_sample', 'play_tone', 'start_tone', 'play_file', '_a', '_b', '_switch', '_led', '_pixels', '_temp', '_light', '_touches', '_touch_threshold_adjustment', '_lis3dh', '_sine_wave', '_sine_wave_sample', '_detect_taps', 'touch_A7', '_unsupported']
>>> cp.sound_level
<bound_method>
>>> cp.sound_level()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "adafruit_circuitplayground/express.py", line 82, in _unsupported
NotImplementedError: This feature is not supported on Circuit Playground Express.
>>> 

@dhalbert
Copy link
Collaborator

It would appear that dir() is trying to call the unsupported properties. If you hit tab-complete, a property gets called. This may be a side-effect of that, but if so, bleh.

@jfurcean
Copy link

Interesting note in the python docs about dir().

Note: Because dir() is supplied primarily as a convenience for use at an interactive prompt, it tries to supply an interesting set of names more than it tries to supply a rigorously or consistently defined set of names, and its detailed behavior may change across releases. For example, metaclass attributes are not in the result list when the argument is a class.

Maybe __dir__ should be explicitly defined to avoid inconsistencies, especially for user libs?

@ajs256
Copy link

ajs256 commented Feb 11, 2021

I also bumped into this in #3748. I agree that this is annoying and should be fixed.

@dhalbert
Copy link
Collaborator

The basic problem is that dir() is calling the properties:

test.py:

class Test:
    def func(self):
        pass

    @property
    def prt(self):
        print("prt was called")

    @property
    def prop(self):
        raise NotImplementedError("no prop")
Adafruit CircuitPython 6.2.0-beta.2-dirty on 2021-02-11; Adafruit PyPortal with samd51j20
>>> from test import Test
>>> t = Test()
>>> dir(t)
prt was called
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "test.py", line 11, in prop
NotImplementedError: no prop
>>> 

@dhalbert dhalbert changed the title Can not use dir() with frozen CPX library dir() calls properties when collecting their names Feb 11, 2021
@dhalbert dhalbert added this to the Long term milestone Feb 11, 2021
@caternuson
Copy link
Author

Is this a duplicate of #3748 ?

@jfurcean
Copy link

It looks like a duplicate to me. Also, should this particular issue be raised in the library?

@dhalbert
Copy link
Collaborator

dhalbert commented Feb 11, 2021

Yes, good catch, it's a dupe. I wouldn't say it's a library bug, because it's just the base bug being exercised. There may be other issues caused by calling dir(cp), such as instantiating all the TouchIn's, etc. I closed #3748 in favor of this more extensive discussion.

@dhalbert
Copy link
Collaborator

Same bug reported in MicroPython, not fixed there yet either: micropython#4546.

@Neradoc
Copy link

Neradoc commented Feb 11, 2021

Note that hasattr() and tab completion have the same effect (they use the same underlying method).
Except tab catches the exception by passing true to mp_load_method_protected.

@tannewt tannewt added the bug label Feb 11, 2021
@dhalbert
Copy link
Collaborator

For interactive use, cp.<tab> is a workaround that can list the available functions and properties

jepler added a commit to jepler/circuitpython that referenced this issue Jun 30, 2022
This helps with Python-compatibility (see issue adafruit#4171) but doesn't
completely resolve it.

Now, `dir()` still computes any properties of the underlying object,
HOWEVER, if the property raises an exception this expression is
captured.

This ability to capture exceptions always existed in
`mp_load_method_protected`, we just need to turn it on via the
`catch_all_exc` boolean parameter.
@dhalbert
Copy link
Collaborator

Dupe of #2179

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants