Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consistently spill the evaluation stack when making escaping calls #128682

Open
1 of 3 tasks
markshannon opened this issue Jan 9, 2025 · 0 comments
Open
1 of 3 tasks

Consistently spill the evaluation stack when making escaping calls #128682

markshannon opened this issue Jan 9, 2025 · 0 comments
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement

Comments

@markshannon
Copy link
Member

markshannon commented Jan 9, 2025

Currently, we spill the evaluation stack for most escaping calls, but not all.
For a few calls, notably Py_DECREF, PyStackRef_CLOSE and related calls, the stack isn't spilled.

Spilling in those cases would add a fair bit of complexity to the code generator and might produce a small slowdown, but it is worth it.

Knowing that all values are in memory and visible during escaping calls allows us to be much less conservative
when handling reference counts. We can potentially defer many, many more references as the GC is guaranteed to
be able to find all references on the stack. The free-threading build can avoid having to NULL out the stack
as will only see the valid part of the stack.

There are also benefits unrelated to performance: debuggers and other tools can reliably see values on the stack,
providing more information that just local variables. For example, having the full stack visible during sys.monitoring
INSTRUCTION events allows reversable debuggers to be implemented more simply.

Tasks

Preview Give feedback

Linked PRs

@picnixz picnixz added interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement labels Jan 10, 2025
markshannon added a commit that referenced this issue Jan 14, 2025
GH-128822)

* Mark Py_DECREF and Py_XDECREF as escaping

* Remove explicit loops for clearing array inputs
markshannon added a commit that referenced this issue Jan 22, 2025
…uccess. (GH-129132)

Change PyTuple_FromStackRefSteal and PyList_FromStackRefSteal to only steal on success to avoid escaping
markshannon added a commit that referenced this issue Feb 4, 2025
Expand out SETLOCAL so that code generator can see the decref. Mark Py_CLEAR as escaping
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this issue Feb 7, 2025
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this issue Feb 7, 2025
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this issue Feb 7, 2025
Expand out SETLOCAL so that code generator can see the decref. Mark Py_CLEAR as escaping
cmaloney pushed a commit to cmaloney/cpython that referenced this issue Feb 8, 2025
cmaloney pushed a commit to cmaloney/cpython that referenced this issue Feb 8, 2025
Expand out SETLOCAL so that code generator can see the decref. Mark Py_CLEAR as escaping
markshannon added a commit that referenced this issue Feb 12, 2025
* Handle escapes in DECREF_INPUTS

* Mark a few more functions as escaping

* Replace DECREF_INPUTS with PyStackRef_CLOSE where possible
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement
Projects
None yet
Development

No branches or pull requests

2 participants