Enable Log Rotation on Serve #31844
Enable Log Rotation on Serve #31844
Conversation
Signed-off-by: Andrea Pisoni <andreapiso@gmail.com>
Signed-off-by: Andrea Pisoni <andreapiso@gmail.com>
|
@edoakes would you be willing to have a look? |
There was a problem hiding this comment.
@andreapiso thanks so much for the contribution, this looks great!
Left a small comment and would like to have @rkooo567 take a quick look because he worked on the Ray-level log rotation and may have a comment or two.
| max_bytes = ray._private.worker._global_node.max_bytes | ||
| backup_count = ray._private.worker._global_node.backup_count |
There was a problem hiding this comment.
nit: could you add these as parameters to this function and if they aren't provided pick them up from the ray variables?
There was a problem hiding this comment.
+1. Also we need to update a doc
There was a problem hiding this comment.
OK, will add these parameters to the function, defaulting to None (where if the params are None, they will be taken from the ray global node), and update the docs since the function signature will change.
There was a problem hiding this comment.
Let us know when this is done!
There was a problem hiding this comment.
Done! As per the doc, i made a brief addition on the Serve Logging documentaiton, pointing to the original Log Rotation page on Ray Core.
Signed-off-by: Andrea Pisoni <andreapiso@gmail.com>
Signed-off-by: Andrea Pisoni <andreapiso@gmail.com>
|
@ray-project/ray-docs please take a look at the docs changes here |
|
@edoakes there's a serve failure. can you see if this is relevant? |
I see the same timeout/OOM error on other PRs - so maybe the test is flaky? E.g. #31972 --> https://buildkite.com/ray-project/oss-ci-build-pr/builds/10466#0185efec-d772-4e18-8b51-d6547526c8d1 |
|
Thanks for the contribution! |
This PR adds log rotation for Ray Serve, letting it inherit rotation parameters (max_bytes, backup_count) from Ray Core, bringing a more consistent logging experience to Ray (as opposed to having the serve/ folder grow forever while the other logs rotate. Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Why are these changes needed?
This PR adds log rotation for Ray Serve, letting it inherit rotation parameters (max_bytes, backup_count) from Ray Core, bringing a more consistent logging experience to Ray (as opposed to having the serve/ folder grow forever while the other logs rotate.
Related issue number
Closes #31842
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.