[BUG] string version check regression (1.0.3.linux-x86_64)

[goasChris]

setuptools version

setuptools==71.1.0

Python version

Python 3.8

OS

Ubuntu 20.04.6

Additional environment information

CI runner info:
Current runner version: '2.317.0'
Operating System
Ubuntu
20.04.6 LTS
Runner Image
Image: ubuntu-20.04
Version: 20240721.1.0
Included Software: https://github.com/actions/runner-images/blob/ubuntu20/20240721.1/images/ubuntu/Ubuntu2004-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu20%2F20240721.1

Description

When I run the setuptools string/version check it fails a string that it was previously not (1.0.3.linux-x86_64), which I would think is a regression. It is afaik default Ubuntu running on github CI.

Noticed on my github CI workflow that installation of pyserial-3.5 was failing due to failing versioncheck (InvalidVersion), after setuptools got updated.

Working:
setuptools-70.3.0

Failing:
setuptools-71.1.0

Expected behavior

I expected that the version string check (for 1.0.3.linux-x86_64) that was previously succeeding would not fail when I run the same code/installation, with seemingly the same string.

How to Reproduce

1. Be on Ubuntu 20.04.6 (equivalent to github runner image, or setup a default Ubuntu 20 runner)
2. pip3 install --user -U pip setuptools (should yield setuptools 71.1.0)
3. pip3 install --user -U pyserial
4. Pyserial installation should fail due to faulty string check (1.0.3.linux-x86_64)

Output

Collecting pyserial
  Downloading pyserial-3.5-py2.py3-none-any.whl (90 kB)
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/base_command.py", line 186, in _main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/_internal/commands/install.py", line 357, in run
    resolver.resolve(requirement_set)
  File "/usr/lib/python3/dist-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/usr/lib/python3/dist-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/usr/lib/python3/dist-packages/pip/_internal/legacy_resolve.py", line 281, in _get_abstract_dist_for
    req.populate_link(self.finder, upgrade_allowed, require_hashes)
  File "/usr/lib/python3/dist-packages/pip/_internal/req/req_install.py", line 249, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/lib/python3/dist-packages/pip/_internal/index/package_finder.py", line 898, in find_requirement
    best_candidate_result = self.find_best_candidate(
  File "/usr/lib/python3/dist-packages/pip/_internal/index/package_finder.py", line 881, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
  File "/usr/lib/python3/dist-packages/pip/_internal/index/package_finder.py", line 825, in find_all_candidates
    package_links = self.process_project_url(
  File "/usr/lib/python3/dist-packages/pip/_internal/index/package_finder.py", line 797, in process_project_url
    package_links = self.evaluate_links(
  File "/usr/lib/python3/dist-packages/pip/_internal/index/package_finder.py", line 779, in evaluate_links
    candidate = self.get_install_candidate(link_evaluator, link)
  File "/usr/lib/python3/dist-packages/pip/_internal/index/package_finder.py", line 764, in get_install_candidate
    return InstallationCandidate(
  File "/usr/lib/python3/dist-packages/pip/_internal/models/candidate.py", line 18, in __init__
    self.version = parse_version(version)  # type: _BaseVersion
  File "/usr/local/lib/python3.8/dist-packages/packaging/version.py", line 56, in parse
    return Version(version)
  File "/usr/local/lib/python3.8/dist-packages/packaging/version.py", line 202, in __init__
    raise InvalidVersion(f"Invalid version: '{version}'")
packaging.version.InvalidVersion: Invalid version: '1.0.3.linux-x86_64'

[lazka]

There is no indication that this has anything to do with pyserial or setuptools.

It's also not reproducible in Docker given your instructions.

One weird thing is that your instructions install pip, but your error shows using the system provided pip.

[lazka]

Maybe related to pypa/pip#12825 ?

[goasChris]

I see. Yes, that is strange. Apologies for the subpar reproduction steps.
Interesting to see ArduPuilot chiming in, as my code is indeed based on their script ArduPilot/ardupilot#27689

A bit unsure what to make of this, but it seems they had to do something similar. Maybe the issue lies assumptions made in the script instead...

[jaraco]

The major change between setuptools 70.3 and 71.1 is the exposure of and reliance on natural dependencies, so the presence of an extant 'packaging' could very well affect the build.

However, if you're using the latest version of pip (as indicated), I'd also expect you'd be building in an isolated environment, so the version of setuptools or packaging in the system environment should be irrelevant.

Whatever the issue, until we can trace it to something with Setuptools, there's not much else we can do here. Please feel free to continue to investigate or refine the repro steps.

[goasChris]

I understand. I appreciate your time and effort.

[Ryanf55]

The major change between setuptools 70.3 and 71.1 is the exposure of and reliance on natural dependencies, so the presence of an extant 'packaging' could very well affect the build.

However, if you're using the latest version of pip (as indicated), I'd also expect you'd be building in an isolated environment, so the version of setuptools or packaging in the system environment should be irrelevant.

Whatever the issue, until we can trace it to something with Setuptools, there's not much else we can do here. Please feel free to continue to investigate or refine the repro steps.

Hey there, Ryan here from the Ardupilot dev team.

ArduPilot is NOT building Ubuntu 22 in an isolated environment on Ubuntu jammy, and we pinned a different version of setuptools to account for this bug. It broke other tools such as colcon that were expecting setuptools to be the system version.
We were pinning many python packages because they release ABI breaking changes under our feet, but the recent PR of setuptools broke a few other things (and accidentally disabled all our tests in ROS 2 because tests_require=['pytest'], is treated as a warning instead of an error if you supply it). Not all the maintainers trust the canonical team to maintain ABI.

TLDR: Installing a different version of setuptools to account for this bug outside of a virtual environment had nasty sideffects, so now that it's fixed, we're going back to apt for stability. PR here: ArduPilot/ardupilot#28056