Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-89279: In ceval.c, redefine some macros for speed #32387

Merged
merged 27 commits into from
Apr 22, 2022
Merged

gh-89279: In ceval.c, redefine some macros for speed #32387

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
0be9d62
TEMPORARY hacks to show inline decisions
gvanrossum Apr 6, 2022
abd9118
In ceval.c, redefine some macros for speed
gvanrossum Apr 7, 2022
1724056
Merge remote-tracking branch 'origin/main' into inline
gvanrossum Apr 7, 2022
50e3d7b
Victor's review
gvanrossum Apr 7, 2022
59da3bf
Merge branch 'python:main' into inline
gvanrossum Apr 7, 2022
ab6660e
Add _Py_atomic_load_32bit_impl and use cast macro
gvanrossum Apr 7, 2022
ae6b53c
Move _Py_atomic_load_32bit_impl out of '#if'
gvanrossum Apr 7, 2022
01cff81
Go back to full PGO test suite
gvanrossum Apr 8, 2022
69da380
Longer comment; improve _Py_atomic_load_relaxed
gvanrossum Apr 8, 2022
2894f6e
Only specialize atomic-load-relaxed if MSC
gvanrossum Apr 9, 2022
4b70976
Try _Py_atomic_load_relaxed_int32
gvanrossum Apr 9, 2022
5617fcd
MSC is better off with the default Py_UNREACHABLE()
gvanrossum Apr 11, 2022
a901b91
Update bpo numbers to GH issue numbers
gvanrossum Apr 11, 2022
cceb531
Split accidentally merged lines
gvanrossum Apr 13, 2022
26e8107
Delete temporary /d2inlinelogfull option
gvanrossum Apr 13, 2022
11084c0
Merge branch 'main' into inline
gvanrossum Apr 13, 2022
b7dfcdc
📜🤖 Added by blurb_it.
blurb-it[bot] Apr 13, 2022
9a5c57c
Use static_assert() in _Py_atomic_load_relaxed_int32
gvanrossum Apr 19, 2022
a8cba6e
Revert "MSC is better off with the default Py_UNREACHABLE()"
gvanrossum Apr 19, 2022
c43ec56
Add comment referring to GH-89279 for is_method()
gvanrossum Apr 19, 2022
9a15194
static_assert() is only a C++ feature
gvanrossum Apr 19, 2022
5c992d2
Merge remote-tracking branch 'origin/main' into inline
gvanrossum Apr 20, 2022
7b342e4
Also redefine _Py_DECREF_SPECIALIZED as a macro
gvanrossum Apr 20, 2022
3b21c52
Reformat macros for readability
gvanrossum Apr 21, 2022
41cb067
Try to silence warnings
gvanrossum Apr 21, 2022
87aaf81
Update Python/ceval.c
vstinner Apr 21, 2022
3508f45
Update Python/ceval.c
vstinner Apr 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion PCbuild/build.bat
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ set parallel=/m
set verbose=/nologo /v:m /clp:summary
set kill=
set do_pgo=
set pgo_job=-m test --pgo
set pgo_job=-c0

:CheckOpts
if "%~1"=="-h" goto Usage
Expand Down
2 changes: 1 addition & 1 deletion PCbuild/pyproject.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
<WholeProgramOptimization>true</WholeProgramOptimization>
<ControlFlowGuard Condition="$(EnableControlFlowGuard) != ''">$(EnableControlFlowGuard)</ControlFlowGuard>
<AdditionalOptions>/utf-8 %(AdditionalOptions)</AdditionalOptions>
<MultiProcessorCompilation>true</MultiProcessorCompilation>
<AdditionalOptions Condition="$(SupportPGO) and $(Configuration) == 'PGUpdate'">/d2inlinelogfull:_PyEval_EvalFrameDefault %(AdditionalOptions)</AdditionalOptions> <MultiProcessorCompilation>true</MultiProcessorCompilation>
</ClCompile>
<ClCompile Condition="$(Configuration) == 'Debug'">
<InlineFunctionExpansion>OnlyExplicitInline</InlineFunctionExpansion>
Expand Down
22 changes: 18 additions & 4 deletions Python/ceval.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,23 @@
# error "ceval.c must be build with Py_BUILD_CORE define for best performance"
#endif

#if !defined(Py_DEBUG)
// bpo-45116: The MSVC compiler fails to inline these in PGO build,
// and they're kind of important for performance.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might explain that MSVC can inline them, but it's just an arbitrary limit which cannot be configured.

neonene wrote "PR25244 told me the amount of code in _PyEval_EvalFrameDefault() is over the limit of PGO."

I suggest something like:

Suggested change
// bpo-45116: The MSVC compiler fails to inline these in PGO build,
// and they're kind of important for performance.
// bpo-45116: The MSVC compiler does not inline these 3 static inline functions in PGO build
// in _PyEval_EvalFrameDefault(), because this function is over the limit of PGO,
// and these limits which cannot be configured. Define them as macro to make sure that
// there are always inlined by the preprocessor.

Maybe tomorrow, MSVC PGO limits will change or become configurable, and it will become possible to remove these macros on more recent MSVC versions.


#undef Py_DECREF
#define Py_DECREF(arg) do { PyObject *op = _PyObject_CAST(arg); if (--op->ob_refcnt == 0) { destructor d = Py_TYPE(op)->tp_dealloc; (*d)(op); } } while (0)
Copy link
Member

@vstinner vstinner Apr 20, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Such long line is one of the reason why I wrote PEP 670 :-) I propose to make the code more readable by putting one statement per line, as done in static inline functions:

#ifndef Py_DEBUG
// GH-89279: The MSVC compiler does not inline these static inline functions
// in PGO build in _PyEval_EvalFrameDefault(), because this function is over
// the limit of PGO, and that limit cannot be configured.
// Define them as macros to make sure that they are always inlined by the
// preprocessor.

#undef Py_DECREF
#define Py_DECREF(arg) \
    do { \
        PyObject *op = _PyObject_CAST(arg); \
        if (--op->ob_refcnt == 0) { \
            destructor dealloc = Py_TYPE(op)->tp_dealloc; \
            (*dealloc)(op); \
        } \
    } while (0)

#undef Py_XDECREF
#define Py_XDECREF(arg) \
    do { \
        PyObject *xop = _PyObject_CAST(arg); \
        if (xop != NULL) { \
            Py_DECREF(xop); \
        } \
    } while (0)

#undef Py_IS_TYPE
#define Py_IS_TYPE(ob, type) \
    (_PyObject_CAST_CONST(ob)->ob_type == (type))

#endif

// GH-89279: Similar to above, force inlining by using a macro.
#if defined(_MSC_VER) && SIZEOF_INT == 4
#define _Py_atomic_load_relaxed_int32(ATOMIC_VAL) \
    (assert(sizeof((ATOMIC_VAL)->_value) == 4), \
     *((volatile int*)&((ATOMIC_VAL)->_value)))
#else
#define _Py_atomic_load_relaxed_int32(ATOMIC_VAL) \
    _Py_atomic_load_relaxed(ATOMIC_VAL)
#endif

I declared Py_XDECREF just after Py_DECREF. I also renamed "op1" to "xop" in Py_XDECREF.

UPDATE: Oh, and to copy/paste Py_IS_TYPE(), I used _PyObject_CAST_CONST().


#undef Py_IS_TYPE
#define Py_IS_TYPE(ob, type) (_PyObject_CAST(ob)->ob_type == (type))

#undef Py_XDECREF
#define Py_XDECREF(arg) do { PyObject *op1 = _PyObject_CAST(arg); if (op1 != NULL) { Py_DECREF(op1); } } while (0)

#endif

#define _Py_atomic_load_32bit_impl(value, order) (assert((order) == _Py_memory_order_relaxed), *(value))
sweeneyde marked this conversation as resolved.
Show resolved Hide resolved

/* Forward declarations */
static PyObject *trace_call_function(
PyThreadState *tstate, PyObject *callable, PyObject **stack,
Expand Down Expand Up @@ -1574,10 +1591,7 @@ typedef struct {
PyObject *kwnames;
} CallShape;

static inline bool
is_method(PyObject **stack_pointer, int args) {
return PEEK(args+2) != NULL;
}
#define is_method(stack_pointer, args) (PEEK((args)+2) != NULL)
gvanrossum marked this conversation as resolved.
Show resolved Hide resolved
gvanrossum marked this conversation as resolved.
Show resolved Hide resolved

#define KWNAMES_LEN() \
(call_shape.kwnames == NULL ? 0 : ((int)PyTuple_GET_SIZE(call_shape.kwnames)))
Expand Down