Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[internal] Separate holding of GIL in create_digest_to_digest() #13586

Merged
merged 3 commits into from
Nov 11, 2021

Conversation

Eric-Arellano
Copy link
Contributor

Prework for #13526.

Before, we were parsing the Python objects (which requires the GIL) in the same iterator that created async blocks. That won't work when we port to PyO3 - iterating over the file_items collection uses &'py PyAny values, where the 'py lifetime corresponds to how long we are holding the GIL (as represented by the pyo3::Python type). The Python token is not safe in async blocks because it does not implement Send, so we can't create the futures in this iterator.

This fix makes the code much more understandable as well. In the future, we are unblocked from defining CreateDigest in Rust with PyO3 because parsing of the code is now completely decoupled from using it to create async futures.

[ci skip-build-wheels]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
@Eric-Arellano Eric-Arellano enabled auto-merge (squash) November 11, 2021 17:16
@Eric-Arellano Eric-Arellano merged commit a655638 into pantsbuild:main Nov 11, 2021
@Eric-Arellano Eric-Arellano deleted the refactor-create-digest branch November 11, 2021 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants