[BUG] ImportError when using importlib with setuptools 60.9.0+

[edmorley]

setuptools version

setuptools==62.1.0

(The repro below uses the latest setuptools version, however, the first affected version was setuptools==60.9.0)

Python version

Python 3.7

OS

Docker image python:3.7.13 which is Debian 11 (it also occurs on Ubuntu on Heroku)

Additional environment information

Reduced requirements.txt:

celery==5.2.2
Django==3.2.8
importlib-metadata==0.20

Description

Between setuptools 60.8.2 and 60.9.0, a previously working Django project now fails when using importlib.import_module():

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/testcase.py", line 2, in <module>
    from celery import Celery
ImportError: cannot import name 'Celery' from 'celery' (/usr/local/lib/python3.7/site-packages/celery/__init__.py)

In the original project (which was from a customer), the importlib.import_module() usage was inside gunicorn, when it loads the provided WSGI application file. However, I've removed gunicorn (and much of the rest of the project) as part of creating a reduced testcase.

The changes between those two setuptools versions are:
v60.8.2...v60.9.0

In addition to the workaround of downgrading setuptools to 60.8.2, I also found that making any of the following changes (each in isolation) prevents the error from occurring:

• Upgrading Python to 3.8+
• Upgrading importlib-metadata from 0.20 to 0.21+ (changelog)
• Upgrading Django from 3.2.8 to 3.2.9+ (changelog, of which this seems relevant)
• Upgrading Celery from 5.2.2 to 5.2.5 (changelog, of which this seems relevant) (skipped 5.2.3 and 5.2.4 since they pin setuptools to an earlier version that isn't affected by this)

Expected behavior

Either:

1. The import succeed with newer versions of setuptools, as it did before.
2. Or, setuptools outputs a warning/error to make debugging this easier, or otherwise blocks usage with incompatible importlib-metadata versions.

How to Reproduce

1. docker run --rm -it python:3.7.13 bash
2. echo -e "from django.core.wsgi import get_wsgi_application\nfrom celery import Celery" > testcase.py
3. pip install setuptools==62.1.0
4. pip install celery==5.2.2 Django==3.2.8 importlib-metadata==0.20
5. python -c 'import importlib; importlib.import_module("testcase")'

Output

$ docker run --rm -it python:3.7.13 bash
...
root@030ec79ad5e2:/# echo -e "from django.core.wsgi import get_wsgi_application\nfrom celery import Celery" > testcase.py
root@030ec79ad5e2:/# pip install setuptools==62.1.0
...
root@030ec79ad5e2:/# pip install celery==5.2.2 Django==3.2.8 importlib-metadata==0.20
...
Installing collected packages: wcwidth, pytz, cached-property, billiard, zipp, vine, typing-extensions, sqlparse, six, prompt-toolkit, importlib-metadata, asgiref, amqp, kombu, Django, click, click-repl, click-plugins, click-didyoumean, celery
Successfully installed Django-3.2.8 amqp-5.1.1 asgiref-3.5.0 billiard-3.6.4.0 cached-property-1.5.2 celery-5.2.2 click-8.1.2 click-didyoumean-0.3.0 click-plugins-1.1.1 click-repl-0.2.0 importlib-metadata-0.20 kombu-5.2.4 prompt-toolkit-3.0.29 pytz-2022.1 six-1.16.0 sqlparse-0.4.2 typing-extensions-4.2.0 vine-5.0.0 wcwidth-0.2.5 zipp-3.8.0

root@030ec79ad5e2:/# python -c 'import importlib; importlib.import_module("testcase")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/testcase.py", line 2, in <module>
    from celery import Celery
ImportError: cannot import name 'Celery' from 'celery' (/usr/local/lib/python3.7/site-packages/celery/__init__.py)

[abravalheri]

Thank you very much @edmorley for reporting this.

What seems to be happening here is that this version of django imports distutils and the recent versions of setuptools replace distutils with setuptools._distutils in anticipation to its removal in Python > 3.11.

It seems that any exception raised while processing import distutils is ignored, therefore we cannot see the same error as the one reported in #3293.

I think what we can do here is to add a warning about the incompatibility (right now setuptools depend on importlib_metadata and downgrading it would only cause the same problem for projects using a more recent version).

This way even when the exception is supressed, the users will still be able to see the warning and act accordingly.

What do you think?

[edmorley]

No problem - this was definitely an interesting one to track down. It only occurs with a precise combination of versions of four packages + Python version!

If the Celery import is swapped for something else (eg urllib3) then the issue also doesn't occur - there must be an interaction not only with Django but Celery too?

I think what we can do here is to add a warning about the incompatibility

What would this warning trigger on?

What do you think?

I trust your wisdom! :-)

[abravalheri]

there must be an interaction not only with Django but Celery too?

Celery seems to be using some dynamic programming techniques for importing its internal components. My guess is that the import distutils performed by Django may prevent importlib_metadata from installing its MetaPathFinder and this in turn might interfere with Celery's internal magic...

What would this warning trigger on?

This would be triggered when the error in #3293 is detected.
I also tried to raise a custom exception, but for some circumstances general exceptions are suppressed.

So what would happen is:

• On import disutils, users would see a warning.
• On import setuptools, users would see a warning + the same exception as in [BUG] AttributeError: 'str' object has no attribute 'name' with setuptools 60.9.1+ #3293.

The warning that I have in mind, is something along the lines of:

`importlib_metadata` version is incompatible with `setuptools`.
This problem is likely to be solved by installing an updated version of `importlib_metadata`.

[edmorley]

That sounds great! :-)

[abravalheri]

I tried the following to test the open PR:

> docker run --rm -it python:3.7.13 bash
echo -e "from django.core.wsgi import get_wsgi_application\nfrom celery import Celery" > testcase.py
pip install 'setuptools @ git+https://github.com/abravalheri/setuptools@issue-3292'
pip install celery==5.2.2 Django==3.2.8 importlib-metadata==0.20
python -c 'import importlib; importlib.import_module("testcase")'

The new output is:

/usr/local/lib/python3.7/site-packages/setuptools/_importlib.py:23: UserWarning: `importlib-metadata` version is incompatible with `setuptools`.
This problem is likely to be solved by installing an updated version of `importlib-metadata`.
  warnings.warn(msg)  # Ensure a descriptive message is shown.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/testcase.py", line 2, in <module>
    from celery import Celery
ImportError: cannot import name 'Celery' from 'celery' (/usr/local/lib/python3.7/site-packages/celery/__init__.py)

It is not much, but at least the users can see the recommendation to update importlib-metadata...

[abravalheri]

@edmorley, I understand that this solution is not ideal, sorry for that 😢

Currently importlib-metadata is an important piece in setuptools plans to move past pkg_resources, and in the latest releases we started to depend more and more on it.

Eventually we will be able to use the stdlib's importlib.metadata and the situation will improve, but right now we are in a tight spot.

Setuptools cannot have real dependencies (instead we have to bundle them). This means that setuptools cannot influence pip's resolution algorithm to ensure importlib-metadata version is higher than 0.21. The core metadata spec also does not provide any mechanism to add constraints without creating a dependency relationship.

[edmorley]

Totally understand! I think that warning will be sufficient - it at least gives end-users some bread-crumbs to follow so they can resolve the issue :-)

[PleezDeez]

To anyone else encountering this error, make sure the Windows Media Feature Pack is installed on your computer. If you have the legacy Windows Media Player, you have it, but I was encountering this error repeatedly no matter what I did until I installed it. Go to Settings, Apps, Optional Features, then select it from the list and click next. wait for it to do it's thing then restart to finish installation