Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spawn worker before kill (max_requests) #2422

Closed
radoslav11 opened this issue Sep 16, 2020 · 3 comments
Closed

Spawn worker before kill (max_requests) #2422

radoslav11 opened this issue Sep 16, 2020 · 3 comments

Comments

@radoslav11
Copy link

Hi!

We have a latency sensitive application typically answering requests in 15ms with a worst case deadline of 200ms. To reduce the latency, we have disabled the garbage collection, hence we use the max_requests option.

Unfortunately, this way we might have some down time in our system (basically when we restart some of the workers). To avoid this, we want to change the worker restarts, so that we first spawn a worker and after it's initialised, kill the previous one. In a way this is similar to #2196 but for the max_requests restarts. I was wondering whether something like that is planned as a feature or if there is some work around?

I've attached a plot of the requests in which you can see the spikes that happen from the worker restarts. We have 2 workers right now and the small spikes happen when 1 worker is being restarted, while the large ones happen when both of them restart at the same time.

Screenshot 2020-09-16 at 10 05 49

@benoitc
Copy link
Owner

benoitc commented Jan 26, 2021

There is no way to spawn a worker before it exist in such case or it would require to lock on the arbiter. I would suggest rather to introduce another worker so you would reduce the number of chances 2 workers are killed at the same time and ensure you always have a spare worker to take the load when one is going off.

@tilgovi
Copy link
Collaborator

tilgovi commented Feb 16, 2021

Related discussion here: #2467 (comment)

@benoitc
Copy link
Owner

benoitc commented Aug 6, 2024

no activity since awhile. closing feel free to create a new ticket if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants