Python: package set updates primarily for Python 3.8 support

[FRidh]

Motivation for this change

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

Notify maintainers

cc @

[FRidh]

@jonringer when pushing do so in batches, otherwise the evaluator runs often

[jonringer]

@GrahamcOfBorg eval

[jonringer]

im working on textacy, aiohttp, numpydoc, pybind11 right now. Just a heads up

[FRidh]

ok, thanks

[jonringer]

also, it seems like zope_exceptions is a common failing package for a lot of other packages..... it seems to build fine, looks like the site-packges installs fine... but none of the other packages seem to be able to import it

[FRidh]

probably a namespace package using the wrong namespace implementation

[jonringer]

that's what i thought as well, but if i load up the package in a pure nix-shell it seems to not be able to find it, even though it's on the PYTHONPATH

[FRidh]

but let's ignore it, it was broken already if I am correct. I recall having looked at it before

[jonringer]

I've also noticed that native bindings seem to be especially broken right now, similar to:

cytoolz/dicttoolz.c:11394:5: error: macro "__Pyx_PyCode_New" requires 16 arguments, but only 15 given

I'm assuming this is on the packages to update this?

I also noticed that the other python versions have cython patches, but py38 does not...

[FRidh]

Correct, packages need to fix that for 3.8.

[jonringer]

do you care for maintaining python2 support?

[FRidh]

I don't. I try to keep the core packages functioning but aside from that, no. It's really up to individual maintainers. Given that python2 is EOL by the time we release 20.03 it is even less relevant.

[jonringer]

That's what I thought, was just curious about freezing packages for python2, but I'll stop doing that as it is kind of time consuming

[jonringer]

also, it looks like wheel==41.4 is causing some issues:

      File "/nix/store/fam3daybpxbx7jnqxvi7hmr9zqqdf80v-python3.8-setuptools-41.4.0/lib/python3.8/site-packages/setuptools/_vendor/packaging/specifiers.py", line 594, in __init__
        specifiers = [s.strip() for s in specifiers.split(",") if s.strip()]
    AttributeError: 'SpecifierSet' object has no attribute 'split'

related issue ansible/molecule#2350

[FRidh]

About zope. I checked, and it appears they're migrating to the correct approach using __import__('pkg_resources').declare_namespace(__name__) but not all packages are using it yet it seems.

Don't know why they went straight for from pkgutil import extend_path __path__ = extend_path(__path__, __name__) instead.

[jonringer]

That's the correct approach for python2... but now i dont remember where I saw the error messages.

[jonringer]

i want to say the pkg_util is supposed to be python2 and python3 compatible:
https://packaging.python.org/guides/packaging-namespace-packages/

[FRidh]

zope_testrunner is not failing. I am now collecting for all our zope packages the methods they use. Also consider writing a zope hook that will force the pkg_util approach instead.

[dasj19]

this also broke the package syncserver as it is dependent on webtest which is no longer built using python2. There is a python2 webtest package in stable but none in unstable.