forked from opendatahub-io/odh-dashboard
-
Notifications
You must be signed in to change notification settings - Fork 37
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
b288b49
commit 2f1576c
Showing
5 changed files
with
186 additions
and
7 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,30 @@ | ||
| ## Subscribe to the operator on Market place | ||
| # Seldon Deploy | ||
|
|
||
| [Seldon Deploy](https://deploy.seldon.io) provides oversight and governance for machine learning deployments. | ||
|
|
||
| Seldon Deploy [builds on top of Seldon Core](https://deploy.seldon.io/docs/about/), an open source platform for deploying machine learning models on a Kubernetes cluster. | ||
|
|
||
| Deployments in Seldon Deploy can be SeldonDeployments, which are [Seldon Core resources](https://docs.seldon.io/en/latest/). | ||
|
|
||
| Seldon Core is an open source operator and custom resource (the SeldonDeployment) for deploying models. Models can be built in different toolkits, packaged by either serializing or dockerising and then deployed to kubernetes in SeldonDeployments. | ||
|
|
||
| Features include: | ||
|
|
||
| * Integrations to gateways for progressive rollout strategies such as canary and A/B test. | ||
| * Graphs for pre-process and post-process steps | ||
| * Integration to elasticsearch for audit trails of requests | ||
| * Integration to prometheus for monitoring | ||
| * Ways to deploy explainers, outlier detectors and drift detectors for advanced monitoring. | ||
|
|
||
| Seldon deploy adds a user interface for easily deploying, monitoring and managing models. It adds enterprise functionality such as user roles and permissions, overall resource usage and integrations such as gitops. | ||
|
|
||
| ## Installing Seldon Deploy | ||
|
|
||
| Seldon Deploy has a Red Hat marketplace listing. That install includes Red Hat versions of required sub-components such as Seldon Core. | ||
|
|
||
| Documentation on the Seldon Deploy Red Hat marketplace install is at [https://deploy-master.seldon.io/docs/getting-started/redhat-installation/](https://deploy-master.seldon.io/docs/getting-started/redhat-installation/) | ||
|
|
||
| ### Subscribe to the operator on Market place | ||
| - [https://marketplace.redhat.com/en-us/products/seldon-deploy](https://marketplace.redhat.com/en-us/products/seldon-deploy) | ||
| ## Install the operator and validate | ||
| ### Install the operator and validate | ||
| - [https://marketplace.redhat.com/en-us/documentation/operators](https://marketplace.redhat.com/en-us/documentation/operators) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| kind: ConsoleQuickStart | ||
| metadata: | ||
| name: seldon-deploy-model-drift | ||
| spec: | ||
| displayName: Monitor drift for deployed model | ||
| durationMinutes: 10 | ||
| icon: 'images/seldon.svg' | ||
| description: Monitor drift for deployed image classifier model | ||
| introduction: |- | ||
| ### This quick start shows you how to launch a tensorflow image classifier model and detect when requests to the model are drifting. | ||
| Seldon Deploy is a specialist set of tools designed to simplify and accelerate the process of deploying and managing your machine learning models. | ||
| This quick start is a short version of https://deploy-master.seldon.io/docs/demos/seldon-core/drift/ | ||
| tasks: | ||
| - title: Deploy a pretrained image classifier model | ||
| description: |- | ||
| ### Deploy a pretrained image classifier model | ||
| 1. Open Seldon Console and click create. Deployment creation wizard appears. | ||
| 2. Choose type SeldonDeployment and protocol tensorflow and click next. | ||
| 3. Choose runtime tensorflow and enter the model url gs://seldon-models/tfserving/cifar10/resnet32 | ||
| 4. Click through wizard to deploy the model | ||
| summary: | ||
| success: You have deployed a pretrained tensorflow image classifier | ||
| failed: Try the steps again or see https://deploy-master.seldon.io/docs/demos/seldon-core/drift/ | ||
| - title: Add a drift detector | ||
| description: |- | ||
| ### Deploy a drift detector for the image classifier | ||
| 1. From the image classifier model's page, go to the drift detector wizard | ||
| 2. Enter the model name cifar10 and url gs://seldon-models/alibi-detect/cd/ks/cifar10-0_4_3 for a drift detector. | ||
| 3. Set the batch size to 2, protocol to tensorflow and port to 8080 | ||
| 3. Click through wizard to deploy the drift detector | ||
| summary: | ||
| success: The drift detector shows as loaded | ||
| failed: Try the steps again or see https://deploy-master.seldon.io/docs/demos/seldon-core/drift/ | ||
| - title: Make predictions | ||
| description: |- | ||
| ### Make predictions: | ||
| 1. Go to the make predictions screen | ||
| 2. Run a single prediction using the tensorflow payload format of an image truck - file is https://deploy.seldon.io/docs/demos/seldon-core/outlier/cifar10_image.json | ||
| 3. Run a prediction payload format of an outlier/perturbed truck - file is https://deploy.seldon.io/docs/demos/seldon-core/outlier/outlier_image.json | ||
| summary: | ||
| success: You see ok responses for each of the predictions. | ||
| failed: Try the steps again or see https://deploy-master.seldon.io/docs/demos/seldon-core/drift/ | ||
| - title: Monitor Drift | ||
| description: |- | ||
| ### Monitor Drift: | ||
| 1. Go to the monitor section of the deployment | ||
| 2. Under the ‘Monitor’ section of your deployment, you can see a timeline revealing drift of requests | ||
| summary: | ||
| success: You see graphs revealing drift of predictions. | ||
| failed: Try the steps again or see https://deploy-master.seldon.io/docs/demos/seldon-core/drift/ | ||
| conclusion: You are now able to identify drift in requests sent to the model. | ||
| nextQuickStart: [] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| kind: ConsoleQuickStart | ||
| metadata: | ||
| name: seldon-deploy-model-explainer | ||
| spec: | ||
| displayName: See predictions and explanations for a deployed SKLearn model | ||
| durationMinutes: 10 | ||
| icon: 'images/seldon.svg' | ||
| description: See predictions and explanations for deployed income classifier model | ||
| introduction: |- | ||
| ### This quick start shows you how to launch an income classifier model and see explanations. | ||
| Seldon Deploy is a specialist set of tools designed to simplify and accelerate the process of deploying and managing your machine learning models. | ||
| This quick start is a short version of https://deploy.seldon.io/docs/demos/seldon-core/explainer-anchortabular/ | ||
| tasks: | ||
| - title: Deploy a pretrained income classifier model | ||
| description: |- | ||
| ### Deploy a pretrained income classifier model | ||
| 1. Open Seldon Console and click create. Deployment creation wizard appears. | ||
| 2. Choose type SeldonDeployment and click next. | ||
| 3. Choose runtime scikit-learn and enter the model url gs://seldon-models/sklearn/income/model-0.23.2 | ||
| 4. Click through wizard to deploy the model | ||
| summary: | ||
| success: You have deployed a pretrained sklearn iris model | ||
| failed: Try the steps again or see https://deploy.seldon.io/docs/demos/seldon-core/explainer-anchortabular/ | ||
| - title: Add an explainer | ||
| description: |- | ||
| ### Deploy a pretrained explainer model for the income classifier | ||
| 1. From the income classifier model's page, go to the alibi explainer wizard | ||
| 2. Enter the model url gs://seldon-models/sklearn/income/explainer-py36-0.5.2 for an Anchor Tabular explainer and use the defaults | ||
| 3. Click through wizard to deploy the explainer | ||
| summary: | ||
| success: The explainer shows as loaded | ||
| failed: Try the steps again or see https://deploy.seldon.io/docs/demos/seldon-core/explainer-anchortabular/ | ||
| - title: Make a prediction and see explanations | ||
| description: |- | ||
| ### Make a prediction: | ||
| 1. Use this request payload to make a prediction {"data": {"names": ["Age","Workclass","Education","Marital Status","Occupation","Relationship","Race","Sex","Capital Gain","Capital Loss","Hours per week","Country"], "ndarray": [[53,4,0,2,8,4,2,0,0,0,60,9]]}} | ||
| 2. You should get a successful result back from the prediction. | ||
| 3. Click explain under the prediction to understand why this result was given by seeing anchors of most influential features | ||
| summary: | ||
| success: You see anchor-style explanations for income classifier model. | ||
| failed: Try the steps again or see https://deploy.seldon.io/docs/demos/seldon-core/explainer-anchortabular/ | ||
| conclusion: You are now able to understand the predictions of the model. | ||
| nextQuickStart: [] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| kind: ConsoleQuickStart | ||
| metadata: | ||
| name: seldon-deploy-model-outlier | ||
| spec: | ||
| displayName: See outlier scores for predictions to deployed model | ||
| durationMinutes: 10 | ||
| icon: 'images/seldon.svg' | ||
| description: See outlier scores for predictions to a deployed image classifier model | ||
| introduction: |- | ||
| ### This quick start shows you how to launch a tensorflow image classifier model and detect requests which are outliers. | ||
| Seldon Deploy is a specialist set of tools designed to simplify and accelerate the process of deploying and managing your machine learning models. | ||
| This quick start is a short version of https://deploy.seldon.io/docs/demos/seldon-core/outlier/ | ||
| tasks: | ||
| - title: Deploy a pretrained image classifier model | ||
| description: |- | ||
| ### Deploy a pretrained image classifier model | ||
| 1. Open Seldon Console and click create. Deployment creation wizard appears. | ||
| 2. Choose type SeldonDeployment and protocol tensorflow and click next. | ||
| 3. Choose runtime tensorflow and enter the model url gs://seldon-models/tfserving/cifar10/resnet32 | ||
| 4. Click through wizard to deploy the model | ||
| summary: | ||
| success: You have deployed a pretrained tensorflow image classifier | ||
| failed: Try the steps again or see https://deploy.seldon.io/docs/demos/seldon-core/outlier/ | ||
| - title: Add an outlier detector | ||
| description: |- | ||
| ### Deploy a pretrained outlier detector for the image classifier | ||
| 1. From the image classifier model's page, go to the outlier detector wizard | ||
| 2. Enter the model name cifar10 and url gs://seldon-models/alibi-detect/od/OutlierVAE/cifar10 for an outlier detector and use the defaults | ||
| 3. Click through wizard to deploy the outlier detector | ||
| summary: | ||
| success: The outlier detector shows as loaded | ||
| failed: Try the steps again or see https://deploy.seldon.io/docs/demos/seldon-core/outlier/ | ||
| - title: Make predictions | ||
| description: |- | ||
| ### Make predictions: | ||
| 1. Go to the make predictions screen | ||
| 2. Run a single prediction using the tensorflow payload format of an image truck - file is https://deploy.seldon.io/docs/demos/seldon-core/outlier/cifar10_image.json | ||
| 3. Run a prediction payload format of an outlier/perturbed truck - file is https://deploy.seldon.io/docs/demos/seldon-core/outlier/outlier_image.json | ||
| summary: | ||
| success: You see ok responses for each of the predictions. | ||
| failed: Try the steps again or see https://deploy.seldon.io/docs/demos/seldon-core/outlier/ | ||
| - title: Review Outliers | ||
| description: |- | ||
| ### Review Predictions/Outliers: | ||
| 1. Go to the requests screen (request log) | ||
| 2. Observe the outlier value on each instance. | ||
| 3. Optionally highlight outliers based on this score and also use the filter to see only the outliers as needed. | ||
| summary: | ||
| success: You see outlier scores for each of the predictions. | ||
| failed: Try the steps again or see https://deploy.seldon.io/docs/demos/seldon-core/outlier/ | ||
| conclusion: You are now able to identify outliers in requests sent to the model. | ||
| nextQuickStart: [] |