[Model Monitoring] Implementing model monitoring dashboards and datasource in Grafana #64
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5a039b3 to
37f7ec2
Compare
quaark
approved these changes
May 21, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As part of supporting model monitoring features in
mlrun/ce, in this PR we are adding the required dashboards and datasource to Grafana. Please note that real time metrics (such as drift metrics by time) will be included in the next release. Below you can find information about the new dashboards and datasource:dashboard
Model Monitoring - Overview: A general overview of the current model endpoints in the selected project. This dasbhaords includes the general KPIs of each model endpoint (e.g.drift statusand the number of errors).dashboard
Model Monitoring - Details: Detailed overview of one of the model endpoints in the selected project. This dashboard includes drift and feature analysis according the the results of the latest model monitoring drift analysis which runs each hour by default.datasource
iguazio: amysqldatasource that stores themodel-endpointstable with the model endpoints records. The user will need to supply the credentials for this datasource. He can do that via Grafana UI or through the helm chart installment.A related JIRA ticket: https://jira.iguazeng.com/browse/ML-3553