Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log reason for unready model mesh container #100

Closed
Legion2 opened this issue Jun 26, 2023 · 2 comments · Fixed by #132
Closed

Log reason for unready model mesh container #100

Legion2 opened this issue Jun 26, 2023 · 2 comments · Fixed by #132
Labels
enhancement New feature or request

Comments

@Legion2
Copy link
Contributor

Legion2 commented Jun 26, 2023

I'm currently debugging constantly unready model mesh containers, which do not directly indicate the problem in the logs. Only through reading the code I discovered abortStartup, which fails the ready probe without logging the reason

if (abortStartup) {
return false;
}

ModelMesh.abortStartup indicates an unrecoverable failure, which can only be resolved with an restart of model mesh container.
If the readiness probe fails because of abortStartup it should be logged, to allow debugging the root cause of the issue.

Alternatively, the Liveness probe of the container should fail to indicate an unrecoverable application failure.

@rafvasq rafvasq added the enhancement New feature or request label Jul 19, 2023
@ckadner
Copy link
Member

ckadner commented Jan 19, 2024

@Legion2 -- would you like to propose a code change in a PR?

@Legion2
Copy link
Contributor Author

Legion2 commented Jan 21, 2024

@ckadner I created #132

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants