Restrict CPU usage

[mleonhard]

I wish to use wasmtime in a server to run small untrusted code blocks. I need to limit the CPU time consumed by each running block. I looked through the docs and found only the ability to interrupt running code (Store::interrupt_handle). I don't want to kill badly-behaved blocks. I want to make the blocks behave nicely. I need the server to run hundreds of wasmtime VMs while also not starving the server's request processing code.

Ideally, wasmtime would let me configure a set of VMs with a shared CPU restriction of a certain number of cores, or fractional cores. The VMs would run concurrently and yield to each other so all can make progress. And all of the VMs in the set would not consume more than the set amount of CPU time per second.

Let's support or document how to restrict instance cpu time.

Related issue for restricting memory usage: #2273

Related discussion: https://users.rust-lang.org/t/limit-handler-to-cpu-memory-bounds/26429

[mleonhard]

A workaround is to use core_affinity crate to bind threads to a particular CPU core. If all calls into the VM occur on one core, then they can never exceed one core of CPU usage.

Other server workloads will run slowly if they get scheduled onto the VM's core. As long as workload threads did not set CPU affinity, the OS will run them smoothly on other cores.

I searched through the tokio::runtime source and found no indication that it sets thread CPU affinity. async-std and smol both use async_global_executor which does not set thread CPU affinity.

[alexcrichton]

Currently Wasmtime has no inherent method of limiting CPU usage. It also has no form of resumable interrupts natively. One option is to inject your own periodic calls into the host which can do stack-switching to give you a form of preemption, but it's true that fully supporting this feature would likely require native support in Wasmtime itself.

For now I'd recommend exploring the stack-switching option combined with host calls.

[alexcrichton]

Coming back to this, I think this is roughly implemented with the combination of async support in Wasmtime plus fuel now. That gives you the ability to relatively deterministically preempt executing WebAssembly code to allow an embedding to make otherwise-relevant scheduling decisions.

While that doesn't expose the native ability to limit CPU execution of WebAssembly in the manner that you're describing, I think it provides a solid building block for doing so. I'm not sure that it would make sense for wasmtime to manage thread pools or CPU resources in this regard, that sort of sounds like more of what an embedding of Wasmtime would want to do.

In that sense is the async/fuel support enough for you to unblock this issue?

[mleonhard]

I think the new 'fuel' mechanism is sufficient for making wasm threads behave nicely. Thank you.

I found relevant documentation under wasmtime::Config::async_support. I wish it did not require async.

I found a little issue in the docs:

Care must be taken for this because Wasmtime futures are not Send or Sync.

The binary examples/tokio/main.rs makes a future to call wasmtime::Func::call_async and passes the future to tokio::runtime::Runtime::spawn which accepts only Send futures. If this binary compiles then Wasmtime futures are Send.

Closing issue. Thanks again!

[alexcrichton]

Thanks for pointing that out, I've updated those docs in #3034