Merge branch 'main' into complex-cov

.github/CODEOWNERS

@@ -28,14 +28,18 @@ Objects/type*                 @markshannon
 Objects/codeobject.c          @markshannon
 Objects/frameobject.c         @markshannon
 Objects/call.c                @markshannon
-Python/ceval.c                @markshannon
+Python/ceval*.c               @markshannon @gvanrossum
+Python/ceval*.h               @markshannon @gvanrossum
 Python/compile.c              @markshannon @iritkatriel
 Python/assemble.c             @markshannon @iritkatriel
 Python/flowgraph.c            @markshannon @iritkatriel
 Python/ast_opt.c              @isidentical
+Python/bytecodes.c            @markshannon @gvanrossum
+Python/optimizer*.c           @markshannon @gvanrossum
 Lib/test/test_patma.py        @brandtbucher
 Lib/test/test_peepholer.py    @brandtbucher
 Lib/test/test_type_*.py       @JelleZijlstra
+Lib/test/test_capi/test_misc.py  @markshannon @gvanrossum
 
 # Exceptions
 Lib/traceback.py              @iritkatriel
@@ -102,6 +106,9 @@ Include/internal/pycore_time.h  @pganssle @abalkin
 /Lib/tokenize.py              @pablogsal @lysnikolaou
 /Lib/test/test_tokenize.py    @pablogsal @lysnikolaou
 
+# Code generator
+/Tools/cases_generator/        @gvanrossum
+
 # AST
 Python/ast.c                  @isidentical
 Parser/asdl.py                @isidentical
@@ -151,7 +158,7 @@ Doc/c-api/stable.rst          @encukou
 
 **/*idlelib*                  @terryjreedy
 
-**/*typing*                   @gvanrossum @JelleZijlstra @AlexWaygood
+**/*typing*                   @JelleZijlstra @AlexWaygood
 
 **/*ftplib                    @giampaolo
 **/*shutil                    @giampaolo

Doc/c-api/bool.rst

@@ -26,19 +26,19 @@ are available, however.
 .. c:var:: PyObject* Py_False
 
    The Python ``False`` object.  This object has no methods and is
-   `immortal <https://peps.python.org/pep-0683/>`_.
+   :term:`immortal`.
 
-.. versionchanged:: 3.12
-   :c:data:`Py_False` is immortal.
+   .. versionchanged:: 3.12
+      :c:data:`Py_False` is :term:`immortal`.
 
 
 .. c:var:: PyObject* Py_True
 
    The Python ``True`` object.  This object has no methods and is
-   `immortal <https://peps.python.org/pep-0683/>`_.
+   :term:`immortal`.
 
-.. versionchanged:: 3.12
-   :c:data:`Py_True` is immortal.
+   .. versionchanged:: 3.12
+      :c:data:`Py_True` is :term:`immortal`.
 
 
 .. c:macro:: Py_RETURN_FALSE

Doc/c-api/init.rst

@@ -1485,7 +1485,7 @@ otherwise immutable (e.g. ``None``, ``(1, 5)``) can't normally be shared
 because of the refcount.  One simple but less-efficient approach around
 this is to use a global lock around all use of some state (or object).
 Alternately, effectively immutable objects (like integers or strings)
-can be made safe in spite of their refcounts by making them "immortal".
+can be made safe in spite of their refcounts by making them :term:`immortal`.
 In fact, this has been done for the builtin singletons, small integers,
 and a number of other builtin objects.
 

Doc/c-api/init_config.rst

@@ -1170,7 +1170,7 @@ PyConfig
 
    .. c:member:: int show_ref_count
 
-      Show total reference count at exit (excluding immortal objects)?
+      Show total reference count at exit (excluding :term:`immortal` objects)?
 
       Set to ``1`` by :option:`-X showrefcount <-X>` command line option.
 

Doc/c-api/none.rst

@@ -16,10 +16,10 @@ same reason.
 .. c:var:: PyObject* Py_None
 
    The Python ``None`` object, denoting lack of value.  This object has no methods
-   and is `immortal <https://peps.python.org/pep-0683/>`_.
+   and is :term:`immortal`.
 
-.. versionchanged:: 3.12
-   :c:data:`Py_None` is immortal.
+   .. versionchanged:: 3.12
+      :c:data:`Py_None` is :term:`immortal`.
 
 .. c:macro:: Py_RETURN_NONE
 

Doc/c-api/refcounting.rst

@@ -17,7 +17,7 @@ of Python objects.
 
    Note that the returned value may not actually reflect how many
    references to the object are actually held.  For example, some
-   objects are "immortal" and have a very high refcount that does not
+   objects are :term:`immortal` and have a very high refcount that does not
    reflect the actual number of references.  Consequently, do not rely
    on the returned value to be accurate, other than a value of 0 or 1.
 
@@ -34,9 +34,7 @@ of Python objects.
 
    Set the object *o* reference counter to *refcnt*.
 
-   Note that this function has no effect on
-   `immortal <https://peps.python.org/pep-0683/>`_
-   objects.
+   This function has no effect on :term:`immortal` objects.
 
    .. versionadded:: 3.9
 
@@ -49,6 +47,8 @@ of Python objects.
    Indicate taking a new :term:`strong reference` to object *o*,
    indicating it is in use and should not be destroyed.
 
+   This function has no effect on :term:`immortal` objects.
+
    This function is usually used to convert a :term:`borrowed reference` to a
    :term:`strong reference` in-place. The :c:func:`Py_NewRef` function can be
    used to create a new :term:`strong reference`.
@@ -113,6 +113,8 @@ of Python objects.
    Release a :term:`strong reference` to object *o*, indicating the
    reference is no longer used.
 
+   This function has no effect on :term:`immortal` objects.
+
    Once the last :term:`strong reference` is released
    (i.e. the object's reference count reaches 0),
    the object's type's deallocation

Doc/c-api/set.rst

@@ -147,7 +147,7 @@ subtypes but not for instances of :class:`frozenset` or its subtypes.
 
    Return ``1`` if found and removed, ``0`` if not found (no action taken), and ``-1`` if an
    error is encountered.  Does not raise :exc:`KeyError` for missing keys.  Raise a
-   :exc:`TypeError` if the *key* is unhashable.  Unlike the Python :meth:`~set.discard`
+   :exc:`TypeError` if the *key* is unhashable.  Unlike the Python :meth:`~frozenset.discard`
    method, this function does not automatically convert unhashable sets into
    temporary frozensets. Raise :exc:`SystemError` if *set* is not an
    instance of :class:`set` or its subtype.

Doc/c-api/slice.rst

@@ -119,8 +119,7 @@ Ellipsis Object
 .. c:var:: PyObject *Py_Ellipsis
 
    The Python ``Ellipsis`` object.  This object has no methods.  Like
-   :c:data:`Py_None`, it is an `immortal <https://peps.python.org/pep-0683/>`_.
-   singleton object.
+   :c:data:`Py_None`, it is an :term:`immortal` singleton object.
 
    .. versionchanged:: 3.12
       :c:data:`Py_Ellipsis` is immortal.

Doc/extending/newtypes.rst

@@ -296,7 +296,7 @@ An interesting advantage of using the :c:member:`~PyTypeObject.tp_members` table
 descriptors that are used at runtime is that any attribute defined this way can
 have an associated doc string simply by providing the text in the table.  An
 application can use the introspection API to retrieve the descriptor from the
-class object, and get the doc string using its :attr:`__doc__` attribute.
+class object, and get the doc string using its :attr:`!__doc__` attribute.
 
 As with the :c:member:`~PyTypeObject.tp_methods` table, a sentinel entry with a :c:member:`~PyMethodDef.ml_name` value
 of ``NULL`` is required.

Doc/glossary.rst

@@ -579,6 +579,16 @@ Glossary
       :ref:`idle` is a basic editor and interpreter environment
       which ships with the standard distribution of Python.
 
+   immortal
+      If an object is immortal, its reference count is never modified, and
+      therefore it is never deallocated.
+
+      Built-in strings and singletons are immortal objects. For example,
+      :const:`True` and :const:`None` singletons are immmortal.
+
+      See `PEP 683 – Immortal Objects, Using a Fixed Refcount
+      <https://peps.python.org/pep-0683/>`_ for more information.
+
    immutable
       An object with a fixed value.  Immutable objects include numbers, strings and
       tuples.  Such an object cannot be altered.  A new object has to
@@ -1056,7 +1066,7 @@ Glossary
    reference count
       The number of references to an object.  When the reference count of an
       object drops to zero, it is deallocated.  Some objects are
-      "immortal" and have reference counts that are never modified, and
+      :term:`immortal` and have reference counts that are never modified, and
       therefore the objects are never deallocated.  Reference counting is
       generally not visible to Python code, but it is a key element of the
       :term:`CPython` implementation.  Programmers can call the

Doc/howto/descriptor.rst

@@ -521,11 +521,11 @@ everyday Python programs.
 Descriptor protocol
 -------------------
 
-``descr.__get__(self, obj, type=None) -> value``
+``descr.__get__(self, obj, type=None)``
 
-``descr.__set__(self, obj, value) -> None``
+``descr.__set__(self, obj, value)``
 
-``descr.__delete__(self, obj) -> None``
+``descr.__delete__(self, obj)``
 
 That is all there is to it.  Define any of these methods and an object is
 considered a descriptor and can override default behavior upon being looked up
@@ -1013,17 +1013,23 @@ here is a pure Python equivalent:
             if obj is None:
                 return self
             if self.fget is None:
-                raise AttributeError(f"property '{self._name}' has no getter")
+                raise AttributeError(
+                    f'property {self._name!r} of {type(obj).__name__!r} object has no getter'
+                 )
             return self.fget(obj)
 
         def __set__(self, obj, value):
             if self.fset is None:
-                raise AttributeError(f"property '{self._name}' has no setter")
+                raise AttributeError(
+                    f'property {self._name!r} of {type(obj).__name__!r} object has no setter'
+                 )
             self.fset(obj, value)
 
         def __delete__(self, obj):
             if self.fdel is None:
-                raise AttributeError(f"property '{self._name}' has no deleter")
+                raise AttributeError(
+                    f'property {self._name!r} of {type(obj).__name__!r} object has no deleter'
+                 )
             self.fdel(obj)
 
         def getter(self, fget):
@@ -1054,6 +1060,11 @@ here is a pure Python equivalent:
         def delx(self):
             del self.__x
         x = Property(getx, setx, delx, "I'm the 'x' property.")
+        no_getter = Property(None, setx, delx, "I'm the 'x' property.")
+        no_setter = Property(getx, None, delx, "I'm the 'x' property.")
+        no_deleter = Property(getx, setx, None, "I'm the 'x' property.")
+        no_doc = Property(getx, setx, delx, None)
+
 
     # Now do it again but use the decorator style
 
@@ -1092,6 +1103,32 @@ here is a pure Python equivalent:
     >>> hasattr(ccc, 'x')
     False
 
+    >>> cc = CC()
+    >>> cc.x = 33
+    >>> try:
+    ...     cc.no_getter
+    ... except AttributeError as e:
+    ...     e.args[0]
+    ...
+    "property 'no_getter' of 'CC' object has no getter"
+
+    >>> try:
+    ...     cc.no_setter = 33
+    ... except AttributeError as e:
+    ...     e.args[0]
+    ...
+    "property 'no_setter' of 'CC' object has no setter"
+
+    >>> try:
+    ...     del cc.no_deleter
+    ... except AttributeError as e:
+    ...     e.args[0]
+    ...
+    "property 'no_deleter' of 'CC' object has no deleter"
+
+    >>> CC.no_doc.__doc__ is None
+    True
+
 The :func:`property` builtin helps whenever a user interface has granted
 attribute access and then subsequent changes require the intervention of a
 method.

Doc/howto/isolating-extensions.rst

@@ -339,12 +339,44 @@ That is, heap types should:
 - Define a traverse function using ``Py_tp_traverse``, which
   visits the type (e.g. using :c:expr:`Py_VISIT(Py_TYPE(self))`).
 
-Please refer to the :ref:`the documentation <type-structs>` of
+Please refer to the the documentation of
 :c:macro:`Py_TPFLAGS_HAVE_GC` and :c:member:`~PyTypeObject.tp_traverse`
 for additional considerations.
 
-If your traverse function delegates to the ``tp_traverse`` of its base class
-(or another type), ensure that ``Py_TYPE(self)`` is visited only once.
+The API for defining heap types grew organically, leaving it
+somewhat awkward to use in its current state.
+The following sections will guide you through common issues.
+
+
+``tp_traverse`` in Python 3.8 and lower
+.......................................
+
+The requirement to visit the type from ``tp_traverse`` was added in Python 3.9.
+If you support Python 3.8 and lower, the traverse function must *not*
+visit the type, so it must be more complicated::
+
+   static int my_traverse(PyObject *self, visitproc visit, void *arg)
+   {
+       if (Py_Version >= 0x03090000) {
+           Py_VISIT(Py_TYPE(self));
+       }
+       return 0;
+   }
+
+Unfortunately, :c:data:`Py_Version` was only added in Python 3.11.
+As a replacement, use:
+
+* :c:macro:`PY_VERSION_HEX`, if not using the stable ABI, or
+* :py:data:`sys.version_info` (via :c:func:`PySys_GetObject` and
+  :c:func:`PyArg_ParseTuple`).
+
+
+Delegating ``tp_traverse``
+..........................
+
+If your traverse function delegates to the :c:member:`~PyTypeObject.tp_traverse`
+of its base class (or another type), ensure that ``Py_TYPE(self)`` is visited
+only once.
 Note that only heap type are expected to visit the type in ``tp_traverse``.
 
 For example, if your traverse function includes::
@@ -356,11 +388,70 @@ For example, if your traverse function includes::
     if (base->tp_flags & Py_TPFLAGS_HEAPTYPE) {
         // a heap type's tp_traverse already visited Py_TYPE(self)
     } else {
-        Py_VISIT(Py_TYPE(self));
+        if (Py_Version >= 0x03090000) {
+            Py_VISIT(Py_TYPE(self));
+        }
     }
 
-It is not necessary to handle the type's reference count in ``tp_new``
-and ``tp_clear``.
+It is not necessary to handle the type's reference count in
+:c:member:`~PyTypeObject.tp_new` and :c:member:`~PyTypeObject.tp_clear`.
+
+
+Defining ``tp_dealloc``
+.......................
+
+If your type has a custom :c:member:`~PyTypeObject.tp_dealloc` function,
+it needs to:
+
+- call :c:func:`PyObject_GC_UnTrack` before any fields are invalidated, and
+- decrement the reference count of the type.
+
+To keep the type valid while ``tp_free`` is called, the type's refcount needs
+to be decremented *after* the instance is deallocated. For example::
+
+   static void my_dealloc(PyObject *self)
+   {
+       PyObject_GC_UnTrack(self);
+       ...
+       PyTypeObject *type = Py_TYPE(self);
+       type->tp_free(self);
+       Py_DECREF(type);
+   }
+
+The default ``tp_dealloc`` function does this, so
+if your type does *not* override
+``tp_dealloc`` you don't need to add it.
+
+
+Not overriding ``tp_free``
+..........................
+
+The :c:member:`~PyTypeObject.tp_free` slot of a heap type must be set to
+:c:func:`PyObject_GC_Del`.
+This is the default; do not override it.
+
+
+Avoiding ``PyObject_New``
+.........................
+
+GC-tracked objects need to be allocated using GC-aware functions.
+
+If you use use :c:func:`PyObject_New` or :c:func:`PyObject_NewVar`:
+
+- Get and call type's :c:member:`~PyTypeObject.tp_alloc` slot, if possible.
+  That is, replace ``TYPE *o = PyObject_New(TYPE, typeobj)`` with::
+
+      TYPE *o = typeobj->tp_alloc(typeobj, 0);
+
+  Replace ``o = PyObject_NewVar(TYPE, typeobj, size)`` with the same,
+  but use size instead of the 0.
+
+- If the above is not possible (e.g. inside a custom ``tp_alloc``),
+  call :c:func:`PyObject_GC_New` or :c:func:`PyObject_GC_NewVar`::
+
+      TYPE *o = PyObject_GC_New(TYPE, typeobj);
+
+      TYPE *o = PyObject_GC_NewVar(TYPE, typeobj, size);
 
 
 Module State Access from Classes