Setuptools bootstrap fails in presence of installed setuptools_scm

[KOLANICH]

a@a:~$ sudo pip3 uninstall -y setuptools_scm 2> /dev/null 1> /dev/null
a@a:~$ git clone --depth=1 https://github.com/pypa/setuptools 2>/dev/null
a@a:~$ cd setuptools/
a@a:~/setuptools$ python3 ./bootstrap.py 2> /dev/null 1> /dev/null
a@a:~/setuptools$ python3 ./setup.py bdist_wheel 2> /dev/null 1> /dev/null
a@a:~/setuptools$ sudo pip3 install --upgrade ./dist/*.whl
Processing ./dist/setuptools-50.3.2.post20201124-py3-none-any.whl
a@a:~/setuptools$ git clone --depth=1 https://github.com/pypa/setuptools_scm 2>/dev/null
a@a:~/setuptools$ cd setuptools_scm/
a@a:~/setuptools/setuptools_scm$ python3 ./setup.py bdist_wheel 2> /dev/null 1> /dev/null
a@a:~/setuptools/setuptools_scm$ sudo pip3 install --upgrade ./dist/*.whl 2> /dev/null 1> /dev/null
a@a:~/setuptools/setuptools_scm$ cd ..
a@a:~/setuptools$ rm -rf ./setuptools.egg-info/
a@a:~/setuptools$ python3 ./bootstrap.py
adding minimal entry_points
Regenerating egg_info
Traceback (most recent call last):
  File "/home/a/setuptools/pkg_resources/__init__.py", line 2676, in version
    return self._version
  File "/home/a/setuptools/pkg_resources/__init__.py", line 2810, in __getattr__
    raise AttributeError(attr)
AttributeError: _version
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "setup.py", line 196, in <module>
    dist = setuptools.setup(**setup_params)
  File "/home/a/setuptools/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.8/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/home/a/setuptools/setuptools/dist.py", line 423, in __init__
    _Distribution.__init__(self, {
  File "/usr/lib/python3.8/distutils/dist.py", line 292, in __init__
    self.finalize_options()
  File "/home/a/setuptools/setuptools/dist.py", line 694, in finalize_options
    for ep in sorted(eps, key=by_order):
  File "/home/a/setuptools/setuptools/dist.py", line 693, in <lambda>
    eps = map(lambda e: e.load(), pkg_resources.iter_entry_points(group))
  File "/home/a/setuptools/pkg_resources/__init__.py", line 2446, in load
    self.require(*args, **kwargs)
  File "/home/a/setuptools/pkg_resources/__init__.py", line 2469, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/home/a/setuptools/pkg_resources/__init__.py", line 753, in resolve
    if dist is None or (dist not in req and replace_conflicting):
  File "/home/a/setuptools/pkg_resources/__init__.py", line 3120, in __contains__
    item = item.version
  File "/home/a/setuptools/pkg_resources/__init__.py", line 2684, in version
    raise ValueError(msg, self) from e
ValueError: ("Missing 'Version:' header and/or PKG-INFO file at path: /home/a/setuptools/setuptools.egg-info/PKG-INFO", setuptools [unknown version] (/home/a/setuptools))
Traceback (most recent call last):
  File "./bootstrap.py", line 57, in <module>
    __name__ == '__main__' and ensure_egg_info()
  File "./bootstrap.py", line 38, in ensure_egg_info
    run_egg_info()
  File "./bootstrap.py", line 54, in run_egg_info
    subprocess.check_call(cmd)
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3', 'setup.py', 'egg_info']' returned non-zero exit status 1.

[jaraco]

Gah. That's terrible.

I'd really like to get away from setuptools being a special snowflake when it comes to building/bootstrapping. In the meantime, my best recommendation is not to ever install setuptools_scm anywhere ever. Just rely on PEP 518 to make it present when needed.

[KOLANICH]

In the meantime, my best recommendation is not to ever install setuptools_scm anywhere ever. Just rely on PEP 518 to make it present when needed.

I prefer not to use build isolation. For me it is easier to just uninstall setuptools_scm for the time of bootstrapping setuptools, and then build and install it again, the newer version.

But it is still a bug in setuptools itself, if an entry point can disrupt its bootstrapping process.

[lazka]

Does anyone know a workaround for this? (without having to uninstall setuptools_scm)

[webknjaz]

@lazka have you checked the recent version in the main branch? I believe, the bootstrap.py requirement has been removed just recently.

[jaraco]

The latest version of setuptools has a stub for bootstrap.py and since #2582, supplies the bootstrap metadata in the git repo.

I do have some evidence that this new approach works better:

draft $ cat Dockerfile
FROM jaraco/multipy-tox
RUN pip uninstall -y setuptools
RUN pip install setuptools_scm
RUN git clone https://github.com/pypa/setuptools
WORKDIR setuptools
CMD python setup.py egg_info
draft $ docker run -it @$(docker build -q .)
running egg_info
creating setuptools.egg-info
writing setuptools.egg-info/PKG-INFO
writing dependency_links to setuptools.egg-info/dependency_links.txt
writing entry points to setuptools.egg-info/entry_points.txt
writing requirements to setuptools.egg-info/requires.txt
writing setuptools.egg-info/PKG-INFO
writing dependency_links to setuptools.egg-info/dependency_links.txt
writing entry points to setuptools.egg-info/entry_points.txt
writing requirements to setuptools.egg-info/requires.txt
writing top-level names to setuptools.egg-info/top_level.txt
writing manifest file 'setuptools.egg-info/SOURCES.txt'
package init file 'setuptools/tests/indexes/__init__.py' not found (or not a regular file)
package init file 'setuptools/tests/indexes/test_links_priority/__init__.py' not found (or not a regular file)
package init file 'setuptools/tests/indexes/test_links_priority/simple/__init__.py' not found (or not a regular file)
package init file 'setuptools/tests/indexes/test_links_priority/simple/foobar/__init__.py' not found (or not a regular file)
package init file 'pkg_resources/tests/data/__init__.py' not found (or not a regular file)
package init file 'pkg_resources/tests/data/my-test-package-source/__init__.py' not found (or not a regular file)
package init file 'pkg_resources/tests/data/my-test-package_unpacked-egg/__init__.py' not found (or not a regular file)
package init file 'pkg_resources/tests/data/my-test-package-zip/__init__.py' not found (or not a regular file)
package init file 'pkg_resources/tests/data/my-test-package_zipped-egg/__init__.py' not found (or not a regular file)
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.py' under directory 'tests'
warning: no files found matching 'Makefile' under directory 'docs'
writing manifest file 'setuptools.egg-info/SOURCES.txt'

Does this approach work for your use-cases, @KOLANICH and @lazka ?

[KOLANICH]

@jaraco, I have just tested with setuptools removed from the distro on a local machine. It worked. I haven't tested on CI yet though.

[lazka]

Normal setup.py install seems to work fine now

[jaraco]

That's great news. I'll close this issue, but happy to re-open if there are lingering issues.