-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HTTP-Server: Graceful shutdown #97223
Conversation
Pinging @elastic/kibana-core (Team:Core) |
Pinging @elastic/kibana-operations (Team:Operations) |
This shouldn't be what's happening, when |
@spalger yeah! I see that in the code. I think the underlying issue is what we see when running it as I'll try to find a way to hold the UPDATE: In my tests, I noticed the parent Next up is for me to find out how to stop |
I found the reason for this! 🎉 Kibana (the child) is handling the |
@elasticmachine merge upstream |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CliDevMode gracefulTimeout change LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
several nits, LGTM for the Core changes
@elasticmachine merge upstream |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can be done in a separate PR, but we should also document the config option.
💛 Build succeeded, but was flaky
Test FailuresKibana Pipeline / general / Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/dashboard/sync_colors·ts.dashboard sync colors should sync colors on dashboard by defaultStandard Out
Stack Trace
Metrics [docs]
History
To update your PR or re-run it, just comment with: |
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
💚 Backport successful
This backport PR will be merged automatically after passing CI. |
The default setting for `server.shutdownTimeout` is 30s. Docker's default stop behavior is to send a SIGKILL after 10s. This sets the graceful shutdown behavior to be inline with Docker defaults. Related to elastic#97223. Closes elastic#97658
Summary
When shutting down, Kibana now waits up to
server.shutdownTimeout
(30 seconds by default) for all the ongoing HTTP requests to finish. During that period, it sets the response headerconnection
to'close'
(instead ofkeep-alive
), and rejects new connections (by responding503
).Resolves #84452. We might need a follow up to cover #83612.
NOTE: As shown in the HTTP Server tests, the statement in the description above is true. However, during manual tests, I noticed slightly different behaviours depending on how Kibana is run:
node scripts/kibana
): it works as mentioned above.yarn start
): The proxy that runs in front of it immediately closes the ongoing requests with502 Bad Gateway
.yarn start --no-base-path
): The dev watcher wrapping the process also kills the process immediately, leading to the ongoing requests to be immediately dropped due to the server closing the socket.I noticed that both dev mode behaviours are caused by the
CliDevMode
triggering a duplicate of theSIGINT
event. I've created #97439 to follow up on that issue.Checklist
Delete any items that are not applicable to this PR.
For maintainers