Merge branch 'main' into complex-constructor

.github/workflows/build.yml

@@ -388,7 +388,7 @@ jobs:
       id: cache-hypothesis-database
       uses: actions/cache@v4
       with:
-        path: ./hypothesis
+        path: ${{ env.CPYTHON_BUILDDIR }}/.hypothesis/
         key: hypothesis-database-${{ github.head_ref || github.run_id }}
         restore-keys: |
           - hypothesis-database-
@@ -416,7 +416,7 @@ jobs:
       if: always()
       with:
         name: hypothesis-example-db
-        path: .hypothesis/examples/
+        path: ${{ env.CPYTHON_BUILDDIR }}/.hypothesis/examples/
 
 
   build_asan:

Doc/faq/library.rst

@@ -541,84 +541,6 @@ Thus, to read *n* bytes from a pipe *p* created with :func:`os.popen`, you need
 use ``p.read(n)``.
 
 
-.. XXX update to use subprocess. See the :ref:`subprocess-replacements` section.
-
-   How do I run a subprocess with pipes connected to both input and output?
-   ------------------------------------------------------------------------
-
-   Use the :mod:`popen2` module.  For example::
-
-      import popen2
-      fromchild, tochild = popen2.popen2("command")
-      tochild.write("input\n")
-      tochild.flush()
-      output = fromchild.readline()
-
-   Warning: in general it is unwise to do this because you can easily cause a
-   deadlock where your process is blocked waiting for output from the child
-   while the child is blocked waiting for input from you.  This can be caused
-   by the parent expecting the child to output more text than it does or
-   by data being stuck in stdio buffers due to lack of flushing.
-   The Python parent can of course explicitly flush the data it sends to the
-   child before it reads any output, but if the child is a naive C program it
-   may have been written to never explicitly flush its output, even if it is
-   interactive, since flushing is normally automatic.
-
-   Note that a deadlock is also possible if you use :func:`popen3` to read
-   stdout and stderr. If one of the two is too large for the internal buffer
-   (increasing the buffer size does not help) and you ``read()`` the other one
-   first, there is a deadlock, too.
-
-   Note on a bug in popen2: unless your program calls ``wait()`` or
-   ``waitpid()``, finished child processes are never removed, and eventually
-   calls to popen2 will fail because of a limit on the number of child
-   processes.  Calling :func:`os.waitpid` with the :const:`os.WNOHANG` option can
-   prevent this; a good place to insert such a call would be before calling
-   ``popen2`` again.
-
-   In many cases, all you really need is to run some data through a command and
-   get the result back.  Unless the amount of data is very large, the easiest
-   way to do this is to write it to a temporary file and run the command with
-   that temporary file as input.  The standard module :mod:`tempfile` exports a
-   :func:`~tempfile.mktemp` function to generate unique temporary file names. ::
-
-      import tempfile
-      import os
-
-      class Popen3:
-          """
-          This is a deadlock-safe version of popen that returns
-          an object with errorlevel, out (a string) and err (a string).
-          (capturestderr may not work under windows.)
-          Example: print(Popen3('grep spam','\n\nhere spam\n\n').out)
-          """
-          def __init__(self,command,input=None,capturestderr=None):
-              outfile=tempfile.mktemp()
-              command="( %s ) > %s" % (command,outfile)
-              if input:
-                  infile=tempfile.mktemp()
-                  open(infile,"w").write(input)
-                  command=command+" <"+infile
-              if capturestderr:
-                  errfile=tempfile.mktemp()
-                  command=command+" 2>"+errfile
-              self.errorlevel=os.system(command) >> 8
-              self.out=open(outfile,"r").read()
-              os.remove(outfile)
-              if input:
-                  os.remove(infile)
-              if capturestderr:
-                  self.err=open(errfile,"r").read()
-                  os.remove(errfile)
-
-   Note that many interactive programs (e.g. vi) don't work well with pipes
-   substituted for standard input and output.  You will have to use pseudo ttys
-   ("ptys") instead of pipes. Or you can use a Python interface to Don Libes'
-   "expect" library.  A Python extension that interfaces to expect is called
-   "expy" and available from https://expectpy.sourceforge.net.  A pure Python
-   solution that works like expect is :pypi:`pexpect`.
-
-
 How do I access the serial (RS232) port?
 ----------------------------------------
 

Doc/glossary.rst

@@ -425,11 +425,11 @@ Glossary
       An object that tries to find the :term:`loader` for a module that is
       being imported.
 
-      Since Python 3.3, there are two types of finder: :term:`meta path finders
+      There are two types of finder: :term:`meta path finders
       <meta path finder>` for use with :data:`sys.meta_path`, and :term:`path
       entry finders <path entry finder>` for use with :data:`sys.path_hooks`.
 
-      See :pep:`302`, :pep:`420` and :pep:`451` for much more detail.
+      See :ref:`importsystem` and :mod:`importlib` for much more detail.
 
    floor division
       Mathematical division that rounds down to nearest integer.  The floor

Doc/howto/pyporting.rst

@@ -18,9 +18,9 @@ please see :ref:`cporting-howto`.
 
 The archived python-porting_ mailing list may contain some useful guidance.
 
-Since Python 3.13 the original porting guide was discontinued.
+Since Python 3.11 the original porting guide was discontinued.
 You can find the old guide in the
-`archive <https://docs.python.org/3.12/howto/pyporting.html>`_.
+`archive <https://docs.python.org/3.10/howto/pyporting.html>`_.
 
 
 Third-party guides

Doc/library/ctypes.rst

@@ -661,14 +661,18 @@ for debugging because they can provide useful information::
    guaranteed by the library to work in the general case.  Unions and
    structures with bit-fields should always be passed to functions by pointer.
 
-Structure/union alignment and byte order
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-By default, Structure and Union fields are aligned in the same way the C
-compiler does it. It is possible to override this behavior by specifying a
-:attr:`~Structure._pack_` class attribute in the subclass definition.
-This must be set to a positive integer and specifies the maximum alignment for the fields.
-This is what ``#pragma pack(n)`` also does in MSVC.
+Structure/union layout, alignment and byte order
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+By default, Structure and Union fields are laid out in the same way the C
+compiler does it.  It is possible to override this behavior entirely by specifying a
+:attr:`~Structure._layout_` class attribute in the subclass definition; see
+the attribute documentation for details.
+
+It is possible to specify the maximum alignment for the fields by setting
+the :attr:`~Structure._pack_` class attribute to a positive integer.
+This matches what ``#pragma pack(n)`` does in MSVC.
+
 It is also possible to set a minimum alignment for how the subclass itself is packed in the
 same way ``#pragma align(n)`` works in MSVC.
 This can be achieved by specifying a ::attr:`~Structure._align_` class attribute
@@ -2540,6 +2544,31 @@ fields, or any other data types containing pointer type fields.
       the structure when being packed or unpacked to/from memory.
       Setting this attribute to 0 is the same as not setting it at all.
 
+   .. attribute:: _layout_
+
+      An optional string naming the struct/union layout. It can currently
+      be set to:
+
+      - ``"ms"``: the layout used by the Microsoft compiler (MSVC).
+        On GCC and Clang, this layout can be selected with
+        ``__attribute__((ms_struct))``.
+      - ``"gcc-sysv"``: the layout used by GCC with the System V or “SysV-like”
+        data model, as used on Linux and macOS.
+        With this layout, :attr:`~Structure._pack_` must be unset or zero.
+
+      If not set explicitly, ``ctypes`` will use a default that
+      matches the platform conventions. This default may change in future
+      Python releases (for example, when a new platform gains official support,
+      or when a difference between similar platforms is found).
+      Currently the default will be:
+
+      - On Windows: ``"ms"``
+      - When :attr:`~Structure._pack_` is specified: ``"ms"``
+      - Otherwise: ``"gcc-sysv"``
+
+      :attr:`!_layout_` must already be defined when
+      :attr:`~Structure._fields_` is assigned, otherwise it will have no effect.
+
    .. attribute:: _anonymous_
 
       An optional sequence that lists the names of unnamed (anonymous) fields.

Doc/library/dataclasses.rst

@@ -461,8 +461,8 @@ Module contents
 
 .. function:: is_dataclass(obj)
 
-   Return ``True`` if its parameter is a dataclass or an instance of one,
-   otherwise return ``False``.
+   Return ``True`` if its parameter is a dataclass (including subclasses of a
+   dataclass) or an instance of one, otherwise return ``False``.
 
    If you need to know if a class is an instance of a dataclass (and
    not a dataclass itself), then add a further check for ``not

Doc/library/logging.rst

@@ -109,11 +109,11 @@ The ``name`` is potentially a period-separated hierarchical value, like
 Loggers that are further down in the hierarchical list are children of loggers
 higher up in the list.  For example, given a logger with a name of ``foo``,
 loggers with names of ``foo.bar``, ``foo.bar.baz``, and ``foo.bam`` are all
-descendants of ``foo``.  The logger name hierarchy is analogous to the Python
-package hierarchy, and identical to it if you organise your loggers on a
-per-module basis using the recommended construction
-``logging.getLogger(__name__)``.  That's because in a module, ``__name__``
-is the module's name in the Python package namespace.
+descendants of ``foo``.  In addition, all loggers are descendants of the root
+logger. The logger name hierarchy is analogous to the Python package hierarchy,
+and identical to it if you organise your loggers on a per-module basis using
+the recommended construction ``logging.getLogger(__name__)``.  That's because
+in a module, ``__name__`` is the module's name in the Python package namespace.
 
 
 .. class:: Logger
@@ -1157,10 +1157,12 @@ functions.
 
 .. function:: getLogger(name=None)
 
-   Return a logger with the specified name or, if name is ``None``, return a
-   logger which is the root logger of the hierarchy. If specified, the name is
-   typically a dot-separated hierarchical name like *'a'*, *'a.b'* or *'a.b.c.d'*.
-   Choice of these names is entirely up to the developer who is using logging.
+   Return a logger with the specified name or, if name is ``None``, return the
+   root logger of the hierarchy. If specified, the name is typically a
+   dot-separated hierarchical name like *'a'*, *'a.b'* or *'a.b.c.d'*. Choice
+   of these names is entirely up to the developer who is using logging, though
+   it is recommended that ``__name__`` be used unless you have a specific
+   reason for not doing that, as mentioned in :ref:`logger`.
 
    All calls to this function with a given name return the same logger instance.
    This means that logger instances never need to be passed between different parts

Doc/library/subprocess.rst

@@ -52,7 +52,7 @@ underlying :class:`Popen` interface can be used directly.
 
    If *capture_output* is true, stdout and stderr will be captured.
    When used, the internal :class:`Popen` object is automatically created with
-   *stdout* and *stdin* both set to :data:`~subprocess.PIPE`.
+   *stdout* and *stderr* both set to :data:`~subprocess.PIPE`.
    The *stdout* and *stderr* arguments may not be supplied at the same time as *capture_output*.
    If you wish to capture and combine both streams into one,
    set *stdout* to :data:`~subprocess.PIPE`
@@ -1443,36 +1443,8 @@ Environment example::
 
 
 
-Replacing :func:`os.popen`, :func:`os.popen2`, :func:`os.popen3`
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-::
-
-   (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
-   ==>
-   p = Popen(cmd, shell=True, bufsize=bufsize,
-             stdin=PIPE, stdout=PIPE, close_fds=True)
-   (child_stdin, child_stdout) = (p.stdin, p.stdout)
-
-::
-
-   (child_stdin,
-    child_stdout,
-    child_stderr) = os.popen3(cmd, mode, bufsize)
-   ==>
-   p = Popen(cmd, shell=True, bufsize=bufsize,
-             stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
-   (child_stdin,
-    child_stdout,
-    child_stderr) = (p.stdin, p.stdout, p.stderr)
-
-::
-
-   (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
-   ==>
-   p = Popen(cmd, shell=True, bufsize=bufsize,
-             stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
-   (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
+Replacing :func:`os.popen`
+^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Return code handling translates as follows::
 
@@ -1489,44 +1461,6 @@ Return code handling translates as follows::
        print("There were some errors")
 
 
-Replacing functions from the :mod:`!popen2` module
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. note::
-
-   If the cmd argument to popen2 functions is a string, the command is executed
-   through /bin/sh.  If it is a list, the command is directly executed.
-
-::
-
-   (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
-   ==>
-   p = Popen("somestring", shell=True, bufsize=bufsize,
-             stdin=PIPE, stdout=PIPE, close_fds=True)
-   (child_stdout, child_stdin) = (p.stdout, p.stdin)
-
-::
-
-   (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
-   ==>
-   p = Popen(["mycmd", "myarg"], bufsize=bufsize,
-             stdin=PIPE, stdout=PIPE, close_fds=True)
-   (child_stdout, child_stdin) = (p.stdout, p.stdin)
-
-:class:`popen2.Popen3` and :class:`popen2.Popen4` basically work as
-:class:`subprocess.Popen`, except that:
-
-* :class:`Popen` raises an exception if the execution fails.
-
-* The *capturestderr* argument is replaced with the *stderr* argument.
-
-* ``stdin=PIPE`` and ``stdout=PIPE`` must be specified.
-
-* popen2 closes all file descriptors by default, but you have to specify
-  ``close_fds=True`` with :class:`Popen` to guarantee this behavior on
-  all platforms or past Python versions.
-
-
 Legacy Shell Invocation Functions
 ---------------------------------
 

Doc/tools/extensions/peg_highlight.py

@@ -16,7 +16,6 @@ class PEGLexer(RegexLexer):
         - Rule types
         - Rule options
         - Rules named `invalid_*` or `incorrect_*`
-        - Rules with `RAISE_SYNTAX_ERROR`
     """
 
     name = "PEG"
@@ -60,7 +59,6 @@ class PEGLexer(RegexLexer):
             (r"^(\s+\|\s+.*invalid_\w+.*\n)", bygroups(None)),
             (r"^(\s+\|\s+.*incorrect_\w+.*\n)", bygroups(None)),
             (r"^(#.*invalid syntax.*(?:.|\n)*)", bygroups(None),),
-            (r"^(\s+\|\s+.*\{[^}]*RAISE_SYNTAX_ERROR[^}]*\})\n", bygroups(None)),
         ],
         "root": [
             include("invalids"),

Doc/whatsnew/3.13.rst

@@ -656,6 +656,18 @@ copy
   any user classes which define the :meth:`!__replace__` method.
   (Contributed by Serhiy Storchaka in :gh:`108751`.)
 
+ctypes
+------
+
+* The layout of :ref:`bit fields <ctypes-bit-fields-in-structures-unions>` in
+  :class:`~ctypes.Structure` and :class:`~ctypes.Union` was improved to better
+  match platform defaults (GCC/Clang or MSC). In particular, fields no longer
+  overlap.
+  (Contributed by Matthias Görgens in :gh:`97702`.)
+* A :attr:`ctypes.Structure._layout_` class attribute can be set
+  to help match a non-default ABI.
+  (Contributed by Petr Viktorin in :gh:`97702`.)
+
 dbm
 ---
 
@@ -2169,6 +2181,10 @@ Build Changes
   The bundled mimalloc has custom changes, see :gh:`113141` for details.
   (Contributed by Dino Viehland in :gh:`109914`.)
 
+* On POSIX systems, the pkg-config (``.pc``) filenames now include the ABI
+  flags.  For example, the free-threaded build generates ``python-3.13t.pc``
+  and the debug build generates ``python-3.13d.pc``.
+
 
 Porting to Python 3.13
 ======================