What are implications of small queue for Hystrix thread pool?

[mattrjacobs]

Question raised in https://groups.google.com/forum/#!topic/hystrixoss/XgAuwe2RxS8.

I added some unit tests in #911 to check. These have failed in CI fairly frequently, so I'd like to understand this scenario more thoroughly.

[mattrjacobs]

Here's a comment from the JDK implementation of ThreadPoolExecutor:

/*
* Proceed in 3 steps:
*
* 1. If fewer than corePoolSize threads are running, try to
* start a new thread with the given command as its first
* task. The call to addWorker atomically checks runState and
* workerCount, and so prevents false alarms that would add
* threads when it shouldn't, by returning false.
*
* 2. If a task can be successfully queued, then we still need
* to double-check whether we should have added a thread
* (because existing ones died since last checking) or that
* the pool shut down since entry into this method. So we
* recheck state and if necessary roll back the enqueuing if
* stopped, or start a new thread if there are none.
*
* 3. If we cannot queue task, then we try to add a new
* thread. If it fails, we know we are shut down or saturated
* and so reject the task.
*/

[mattrjacobs]

The above comment implies that if m tasks are submitted into a pool of size n, with m < n, then they should each create a new thread (or re-use an already idle one) without ever touching the work queue.

[mattrjacobs]

Just saw the unit test: HystrixCommandTest.testRejectedThreadUsingQueueSize, which demonstrates the opposite.

In this case, we set up a single-threaded threadpool, with a queue sized at 10 and a rejection threshold of 1. We put a latent Runnable into the queue (outside of any Hystrix work). Then we submit work to the pool via a Hystrix command execution. The task gets rejected, even though the threadpool has a thread available

[mattrjacobs]

After more research, the way this works is that all submitted work first goes to the queue in front of the threadpool, and then the threadpool picks it up at some later point. If using a SynchronousQueue, this happens synchronously, otherwise it happens asynchronously.

This implies that an infinitely big threadpool with a finite queue will still reject some tasks, if they get rejected upon insertion to the queue.