Improve import time of various stdlib modules

[layday]

Feature or enhancement

Proposal:

Following on from #109653, further improvements can be made to import times.

Links to previous discussion of this feature:

https://discuss.python.org/t/deferred-computation-evalution-for-toplevels-imports-and-dataclasses/34173

For example:

importlib.metadata is often used for tasks that need to happen at import, e.g. to enumerate/load entry point plug-ins, so it might be worth seeing if we can cut down its own import time a bit more.

importlib.metadata imports zipfile at the top for a function that won't be called in the vast majority of cases. It also imports importlib.abc, which in turn imports importlib.resources, to subclass an ABC with a single, non-abstract method - I assume redefining the method in importlib.metadata would be harmless. Some other less frequently-used imports which are only accessed once or twice, such as json, could also be tucked away in their calling functions.

Linked PRs

• gh-118761: Improve import time of pprint #122725
• gh-118761: Speedup pathlib import by deferring shutil #123520
• gh-121423: Improve import time of socket by writing socket.errorTab as a constant and lazy import modules #121424
• gh-118761: Improve import time of mimetypes #126979
• gh-118761: improve import time for pickle #128732
• gh-118761: substitute re import in base64.b16decode for a more efficient alternative #128736
• gh-118761: improve import time for secrets #128738
• gh-118761: Improve import time for csv #128858
• gh-118761: Reduce import time of gettext.py by delaying re import #128898
• gh-118761: improve optparse import time by delaying textwrap import #128899
• gh-118761: Improve import time of tomllib #128907
• gh-118761: Improve import time for pstats and zipfile by removing imports to typing #128981
• gh-118761: Improve import time of sqlite3 #129118
• gh-118761: Improve import time of subprocess #129427
• gh-118761: Always lazy import warnings in threading #129428
• [3.13] gh-118761: Improve import time of subprocess (GH-129427) #129447
• [3.12] gh-118761: Improve import time of subprocess (GH-129427) #129448

[hugovk]

@layday Is it okay if I repurpose this issue as an "Improve import time of various stdlib modules" like #109653 but for 3.14?

I've got some pprint improvements, and if we have importlib.metadata and some others, we can group them under the same umbrella issue like last time.

[layday]

Sure!

[danielhollas]

I've opened a PR over at the importlib_metadata repo that avoids importing inspect. python/importlib_metadata#499

importlib.metadata imports zipfile at the top for a function that won't be called in the vast majority of cases.
Some other less frequently-used imports which are only accessed once or twice, such as json, could also be tucked away in their calling functions.

@layday were you planning on tackling these?

It also imports importlib.abc, which in turn imports importlib.resources, to subclass an ABC with a single, non-abstract method

This seems to be solved on main, importlib.abc no longer imports importlib.resources.

[danielhollas]

I've opened a PR over at the importlib_metadata repo that avoids importing inspect. python/importlib_metadata#499

This has been merged and released in version 8.4 of importlib_metadata 🎉

importlib.metadata imports zipfile at the top for a function that won't be called in the vast majority of cases. It also imports importlib.abc, which in turn imports importlib.resources, to subclass an ABC with a single, non-abstract method - I assume redefining the method in importlib.metadata would be harmless. Some other less frequently-used imports which are only accessed once or twice, such as json, could also be tucked away in their calling functions.

I've submitted python/importlib_metadata#502 that defers zip import, and python/importlib_metadata#503 which defers json and platform.

[picnixz]

(removing the 3.14 label since features always target the main branch)

[hugovk]

Note for when documenting this in What's New in Python 3.14, can also include #128559 / #128560.

[vstinner]

@picnixz: I suggest to close this issue.

[hukkin]

@picnixz here's the tomllib import time improvements from Tomli! #128907

[hugovk]

enum has improved a lot recently, here's python3.12 -X importtime -c 'import enum' && tuna import.log and for 3.13 and 3.14 (using latest releases on macOS) visualised with https://github.com/nschloe/tuna: