-
-
Notifications
You must be signed in to change notification settings - Fork 31k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[C API] Move undocumented private _PyArg C API to pycore_modsupport.h internal C API #110964
Comments
Move the following private functions and structures to pycore_modsupport.h internal C API: * _PyArg_BadArgument() * _PyArg_CheckPositional() * _PyArg_NoKeywords() * _PyArg_NoPositional() * _PyArg_ParseStack() * _PyArg_ParseStackAndKeywords() * _PyArg_Parser structure * _PyArg_UnpackKeywords() * _PyArg_UnpackKeywordsWithVararg() * _PyArg_UnpackStack() * _Py_ANY_VARARGS() Changes: * Python/getargs.h now includes pycore_modsupport.h to export functions. * clinic.py now adds pycore_modsupport.h when one of these functions is used. * Add pycore_modsupport.h includes when a C extension uses one of these functions. * Define Py_BUILD_CORE_MODULE in C extensions which now include directly or indirectly (via code generated by Argument Clinic) pycore_modsupport.h: * _csv * _curses_panel * _dbm * _gdbm * _multiprocessing.posixshmem * _sqlite.row * _statistics * grp * resource * syslog * _testcapi: bad_get() no longer uses METH_FASTCALL calling convention but METH_VARARGS. Replace _PyArg_UnpackStack() with PyArg_ParseTuple(). * _testcapi: add PYTESTCAPI_NEED_INTERNAL_API macro which is defined by _testcapi sub-modules which need the internal C API (pycore_modsupport.h): exceptions.c, float.c, vectorcall.c, watchers.c. * Remove Include/cpython/modsupport.h header file. Include/modsupport.h no longer includes the removed header file.
Move the following private functions and structures to pycore_modsupport.h internal C API: * _PyArg_BadArgument() * _PyArg_CheckPositional() * _PyArg_NoKeywords() * _PyArg_NoPositional() * _PyArg_ParseStack() * _PyArg_ParseStackAndKeywords() * _PyArg_Parser structure * _PyArg_UnpackKeywords() * _PyArg_UnpackKeywordsWithVararg() * _PyArg_UnpackStack() * _Py_ANY_VARARGS() Changes: * Python/getargs.h now includes pycore_modsupport.h to export functions. * clinic.py now adds pycore_modsupport.h when one of these functions is used. * Add pycore_modsupport.h includes when a C extension uses one of these functions. * Define Py_BUILD_CORE_MODULE in C extensions which now include directly or indirectly (via code generated by Argument Clinic) pycore_modsupport.h: * _csv * _curses_panel * _dbm * _gdbm * _multiprocessing.posixshmem * _sqlite.row * _statistics * grp * resource * syslog * _testcapi: bad_get() no longer uses METH_FASTCALL calling convention but METH_VARARGS. Replace _PyArg_UnpackStack() with PyArg_ParseTuple(). * _testcapi: add PYTESTCAPI_NEED_INTERNAL_API macro which is defined by _testcapi sub-modules which need the internal C API (pycore_modsupport.h): exceptions.c, float.c, vectorcall.c, watchers.c. * Remove Include/cpython/modsupport.h header file. Include/modsupport.h no longer includes the removed header file. * Fix mypy clinic.py
Basically, these functions are only used by Argument Clinic. _PyArg_ParseTupleAndKeywordsFast(), _PyArg_ParseStackAndKeywords(), _PyArg_UnpackKeywords() and _PyArg_UnpackKeywordsWithVararg() have a _PyArg_Parser argument which is not really used without using the internal C API anyway. AC generates complex code using internal header files: #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE)
# include "pycore_gc.h" // PyGC_Head
# include "pycore_runtime.h" // _Py_ID()
#endif
(...)
#if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE)
#define NUM_KEYWORDS 1
static struct {
PyGC_Head _this_is_not_used;
PyObject_VAR_HEAD
PyObject *ob_item[NUM_KEYWORDS];
} _kwtuple = {
.ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS)
.ob_item = { &_Py_ID(loop), },
};
#undef NUM_KEYWORDS
#define KWTUPLE (&_kwtuple.ob_base.ob_base)
#else // !Py_BUILD_CORE
# define KWTUPLE NULL
#endif // !Py_BUILD_CORE
static const char * const _keywords[] = {"loop", NULL};
static _PyArg_Parser _parser = {
.keywords = _keywords,
.fname = "Future",
.kwtuple = KWTUPLE,
};
#undef KWTUPLE PyGC_Head and _Py_ID() API are part of the internal C API, and _Py_ID() must not be used outside Python code base, since we don't guarantee which identifiers are available or not. |
These helper functions are used by AC. I don't know if it would make sense to make them public or not. They should not be complicated to rewrite if they are no longer usable in the public C API:
|
Move the following private functions and structures to pycore_modsupport.h internal C API: * _PyArg_BadArgument() * _PyArg_CheckPositional() * _PyArg_NoKeywords() * _PyArg_NoPositional() * _PyArg_ParseStack() * _PyArg_ParseStackAndKeywords() * _PyArg_Parser structure * _PyArg_UnpackKeywords() * _PyArg_UnpackKeywordsWithVararg() * _PyArg_UnpackStack() * _Py_ANY_VARARGS() Changes: * Python/getargs.h now includes pycore_modsupport.h to export functions. * clinic.py now adds pycore_modsupport.h when one of these functions is used. * Add pycore_modsupport.h includes when a C extension uses one of these functions. * Define Py_BUILD_CORE_MODULE in C extensions which now include directly or indirectly (via code generated by Argument Clinic) pycore_modsupport.h: * _csv * _curses_panel * _dbm * _gdbm * _multiprocessing.posixshmem * _sqlite.row * _statistics * grp * resource * syslog * _testcapi: bad_get() no longer uses METH_FASTCALL calling convention but METH_VARARGS. Replace _PyArg_UnpackStack() with PyArg_ParseTuple(). * _testcapi: add PYTESTCAPI_NEED_INTERNAL_API macro which is defined by _testcapi sub-modules which need the internal C API (pycore_modsupport.h): exceptions.c, float.c, vectorcall.c, watchers.c. * Remove Include/cpython/modsupport.h header file. Include/modsupport.h no longer includes the removed header file. * Fix mypy clinic.py
Don't rely on the global 'clinic' argument: pass explicitly a 'clinic' argument.
They're vararg functions that return borrowed references, which makes them C-specific and dependent on CPython implementation details. They're quite handy in C, but IMO they'll need a good deal of design work if we're to expose them properly. |
Yeah, I agree with you. |
Don't rely on the global 'clinic' argument: pass explicitly a 'clinic' argument.
Move the following private functions and structures to pycore_modsupport.h internal C API: * _PyArg_BadArgument() * _PyArg_CheckPositional() * _PyArg_NoKeywords() * _PyArg_NoPositional() * _PyArg_ParseStack() * _PyArg_ParseStackAndKeywords() * _PyArg_Parser structure * _PyArg_UnpackKeywords() * _PyArg_UnpackKeywordsWithVararg() * _PyArg_UnpackStack() * _Py_ANY_VARARGS() Changes: * Python/getargs.h now includes pycore_modsupport.h to export functions. * clinic.py now adds pycore_modsupport.h when one of these functions is used. * Add pycore_modsupport.h includes when a C extension uses one of these functions. * Define Py_BUILD_CORE_MODULE in C extensions which now include directly or indirectly (via code generated by Argument Clinic) pycore_modsupport.h: * _csv * _curses_panel * _dbm * _gdbm * _multiprocessing.posixshmem * _sqlite.row * _statistics * grp * resource * syslog * _testcapi: bad_get() no longer uses METH_FASTCALL calling convention but METH_VARARGS. Replace _PyArg_UnpackStack() with PyArg_ParseTuple(). * _testcapi: add PYTESTCAPI_NEED_INTERNAL_API macro which is defined by _testcapi sub-modules which need the internal C API (pycore_modsupport.h): exceptions.c, float.c, vectorcall.c, watchers.c. * Remove Include/cpython/modsupport.h header file. Include/modsupport.h no longer includes the removed header file. * Fix mypy clinic.py
It's no longer available in the public headers [1] On Linux with GCC 11: error: there are no arguments to ‘_PyArg_NoKeywords’ that depend on a template parameter, so a declaration of ‘_PyArg_NoKeywords’ must be available [-fpermissive] 226 | if (!_PyArg_NoKeywords("function", iKw)) [1] python/cpython#110964
Move the following private functions and structures to pycore_modsupport.h internal C API: * _PyArg_BadArgument() * _PyArg_CheckPositional() * _PyArg_NoKeywords() * _PyArg_NoPositional() * _PyArg_ParseStack() * _PyArg_ParseStackAndKeywords() * _PyArg_Parser structure * _PyArg_UnpackKeywords() * _PyArg_UnpackKeywordsWithVararg() * _PyArg_UnpackStack() * _Py_ANY_VARARGS() Changes: * Python/getargs.h now includes pycore_modsupport.h to export functions. * clinic.py now adds pycore_modsupport.h when one of these functions is used. * Add pycore_modsupport.h includes when a C extension uses one of these functions. * Define Py_BUILD_CORE_MODULE in C extensions which now include directly or indirectly (via code generated by Argument Clinic) pycore_modsupport.h: * _csv * _curses_panel * _dbm * _gdbm * _multiprocessing.posixshmem * _sqlite.row * _statistics * grp * resource * syslog * _testcapi: bad_get() no longer uses METH_FASTCALL calling convention but METH_VARARGS. Replace _PyArg_UnpackStack() with PyArg_ParseTuple(). * _testcapi: add PYTESTCAPI_NEED_INTERNAL_API macro which is defined by _testcapi sub-modules which need the internal C API (pycore_modsupport.h): exceptions.c, float.c, vectorcall.c, watchers.c. * Remove Include/cpython/modsupport.h header file. Include/modsupport.h no longer includes the removed header file. * Fix mypy clinic.py
Python 3.13 moved some private C API functions to internal header files. See python/cpython#106320 for a general discussion on this (glad to see that gnureadline was at least listed as an affected PyPI package :-) ). These private functions affect us: - _Py_SetLocaleFromEnv -> pycore_pylifecycle.h (see python/cpython#106400) - _PyArg_CheckPositional, _PyArg_BadArgument -> pycore_modsupport.h (see python/cpython#110964) Since we can't include these anymore, patch the relevant declarations into the module code. The alternative (add the internal headers to this package) is much messier, as we would have to pull in most of those headers.
python/cpython#110964 Copied from pycore_modsupport.h
Add declarations for private functions from pycore_modsupport python/cpython#110964 Copied from pycore_modsupport.h
python/cpython#110964 Copied from pycore_modsupport.h
python/cpython#110964 Copied from pycore_modsupport.h
python/cpython#110964 Copied from pycore_modsupport.h remove PY_SSIZE_T_CLEAN definition python/cpython#104922
python/cpython#110964 Copied from pycore_modsupport.h remove PY_SSIZE_T_CLEAN definition python/cpython#104922
python/cpython#110964 Copied from pycore_modsupport.h remove PY_SSIZE_T_CLEAN definition python/cpython#104922
python/cpython#110964 Copied from pycore_modsupport.h remove PY_SSIZE_T_CLEAN definition python/cpython#104922
python/cpython#110964 Copied from pycore_modsupport.h remove PY_SSIZE_T_CLEAN definition python/cpython#104922
Add declarations for private functions from pycore_modsupport python/cpython#110964 Copied from pycore_modsupport.h remove PY_SSIZE_T_CLEAN definition python/cpython#104922
python/cpython#110964 Copied from pycore_modsupport.h remove PY_SSIZE_T_CLEAN definition python/cpython#104922
python/cpython#110964 Copied from pycore_modsupport.h remove PY_SSIZE_T_CLEAN definition python/cpython#104922
python/cpython#110964 Copied from pycore_modsupport.h remove PY_SSIZE_T_CLEAN definition python/cpython#104922
python/cpython#110964 Copied from pycore_modsupport.h remove PY_SSIZE_T_CLEAN definition python/cpython#104922
python/cpython#110964 Copied from pycore_modsupport.h remove PY_SSIZE_T_CLEAN definition python/cpython#104922
python/cpython#110964 Copied from pycore_modsupport.h remove PY_SSIZE_T_CLEAN definition python/cpython#104922
python/cpython#110964 Copied from pycore_modsupport.h remove PY_SSIZE_T_CLEAN definition python/cpython#104922
If a 3rd party C extension uses one of these functions, we should consider adding a clean, documented and tested public function to replace it.
Private functions and structures:
Linked PRs
The text was updated successfully, but these errors were encountered: