Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Made tokio initialization lazy #33

Merged
merged 1 commit into from
Aug 8, 2021
Merged

Made tokio initialization lazy #33

merged 1 commit into from
Aug 8, 2021

Conversation

awestlake87
Copy link
Owner

This PR makes tokio initialization lazy, which allows for more idiomatic module imports and should make Python multiprocessing less problematic. Motivations for these changes can be found in #32

Changes:

  • pyo3_asyncio::tokio::init now accepts a tokio::runtime::Builder instead of a tokio::runtime::Runtime
    • This function no longer panics when called twice, only the most recent builder config is used
    • init does not actually build the runtime, it instead stores the config so the runtime can be built upon the first call to pyo3_asyncio::tokio::get_runtime
  • Tokio's current thread scheduler now requires the user to manually spawn the thread:
    let mut builder = tokio::runtime::Builder::new_current_thread();
    builder.enable_all();
    
    pyo3_asyncio::tokio::init(builder);
    std::thread::spawn(move || {
        pyo3_asyncio::tokio::get_runtime().block_on(std::future::pending::<()>());
    });
    • This makes current thread initialization more explicit, but it also allows more control over when and where the worker thread is spawned. Spawning it immediately may not be desirable for Python multiprocessing since the fork() system call only clones the thread that called it. For this reason it's preferable to spawn this thread only after the process has been forked, whereas the Builder initialization can occur before the fork if desired.
    • I think current-thread is not the primary use-case right now so it probably won't affect too many people.
  • Removed most tokio initialization functions to reduce the API surface. Users of multi-threaded scheduler just need to remove their initialization calls. Any other configuration is supported, but more verbose.

This PR might be bundled into #30 for a single disruptive initialization change.

This was referenced Jul 14, 2021
@awestlake87 awestlake87 merged commit 0ffcea7 into master Aug 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant