[internal] Separate holding of GIL in create_digest_to_digest()
#13586
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Prework for #13526.
Before, we were parsing the Python objects (which requires the GIL) in the same iterator that created async blocks. That won't work when we port to PyO3 - iterating over the
file_items
collection uses&'py PyAny
values, where the'py
lifetime corresponds to how long we are holding the GIL (as represented by thepyo3::Python
type). ThePython
token is not safe in async blocks because it does not implementSend
, so we can't create the futures in this iterator.This fix makes the code much more understandable as well. In the future, we are unblocked from defining
CreateDigest
in Rust withPyO3
because parsing of the code is now completely decoupled from using it to create async futures.[ci skip-build-wheels]