diff --git a/README.md b/README.md index 888cb554ec0..50ef1b763c1 100644 --- a/README.md +++ b/README.md @@ -7,4 +7,6 @@ In the future, we hope to support more advanced use cases such as outlier detect This project is an evolution of the [original proposal in the Kubeflow repo](https://github.com/kubeflow/kubeflow/issues/2306). To know more about KFServing, please [read the docs](/docs) -For developers looking to contribute, please [follow this doc](/docs/DEVELOPER_GUIDE.md). \ No newline at end of file +For developers looking to contribute, please [follow this doc](/docs/DEVELOPER_GUIDE.md). + +![KFServing](./docs/diagrams/kfserving.png) \ No newline at end of file diff --git a/docs/DEVELOPER_GUIDE.md b/docs/DEVELOPER_GUIDE.md index 4d16fc0c60e..fc0a9bfdff4 100644 --- a/docs/DEVELOPER_GUIDE.md +++ b/docs/DEVELOPER_GUIDE.md @@ -193,6 +193,51 @@ controller is simply: make deploy-dev ``` +### Knative CLI (knctl): + +You can also use [Knative CLI (`knctl`)](https://github.com/cppforlife/knctl) to interact with models deployed on KFServing. It provides a simple set of commands to interact with a [Knative installation](https://github.com/knative/docs). You can grab pre-built binaries from the [Releases page](https://github.com/cppforlife/knctl/releases). Once downloaded, you can run the following commands to get it working. + +``` +# compare checksum output to what's included in the release notes +$ shasum -a 265 ~/Downloads/knctl-* + +# move binary to your system’s /usr/local/bin -- might require root password +$ mv ~/Downloads/knctl-* /usr/local/bin/knctl + +# make the newly copied file executable -- might require root password +$ chmod +x /usr/local/bin/knctl +``` + +You can then run a smoke test by running the following command to show the details of tensorflow sample revision. + +``` +knctl revision show -r flowers-sample-default-4s74r +Revision 'flowers-sample-default-4s74r' + +Name flowers-sample-default-4s74r +Tags - +Image digest index.docker.io/tensorflow/serving@sha256:df3c6fe1fbe5ccc3a916984ff313cc2d17e617f7b8782fc31e762c491325d813 +Log URL http://localhost:8001/api/v1/namespaces/knative-monitoring/services/kibana-logging/proxy/app/kibana#/discover?_a=(query:(match:(kubernetes.labels.knative-dev%2FrevisionUID:(query:'1135797e-8585-11e9-adbd-b680f8334647',type:phrase)))) +Annotations autoscaling.knative.dev/class: kpa.autoscaling.knative.dev + autoscaling.knative.dev/target: "1" +Age 1h + +Conditions + +Type Status Age Reason Message +Active False 59m NoTraffic The target is not receiving traffic. +BuildSucceeded True 1h - - +ContainerHealthy True 1h - - +Ready True 1h - - +ResourcesAvailable True 1h - - + +Pods conditions + +Pod Type Status Age Reason Message + +Succeeded +``` + ## Troubleshooting 1. When you run make deploy, you may encounter an error like this: diff --git a/docs/diagrams/kfserving.png b/docs/diagrams/kfserving.png new file mode 100644 index 00000000000..0e4ff3e2120 Binary files /dev/null and b/docs/diagrams/kfserving.png differ diff --git a/docs/samples/tensorflow/README.md b/docs/samples/tensorflow/README.md index a3c4a5730b1..f0d50cd06e8 100644 --- a/docs/samples/tensorflow/README.md +++ b/docs/samples/tensorflow/README.md @@ -57,4 +57,56 @@ Expected Output ] * Connection #0 to host 34.83.190.188 left intact }% -``` \ No newline at end of file +``` + +If you stop making requests to the application, you should eventually see that your application scales itself back down to zero. Watch the pod until you see that it is `Terminating`. This should take approximately 90 seconds. + +``` +kubectl get pods --watch +``` + +Note: To exit the watch, use `ctrl + c`. + +## Canary Rollout + +To test a canary rollout, you can use the tensorflow-canary.yaml + +Apply the CRD +``` +kubectl apply -f tensorflow-canary.yaml +``` + +To verify if your traffic split percenage is applied correctly, you can use the following command: + +``` +kubectl get kfservices +NAME URL DEFAULT TRAFFIC CANARY TRAFFIC AGE +flowers-sample flowers-sample.default.example.com 90 10 48s +``` + +If you are using the [Knative CLI (knctl)](#knative-cli), run the following command + +``` +knctl route show --route flowers-sample +Route 'flowers-sample' + +Name flowers-sample +Domain flowers-sample.default.example.com +Internal Domain flowers-sample.default.svc.cluster.local +Age 1m + +Targets + +Percent Revision Service Domain +90% flowers-sample-default-4s74r - flowers-sample.default.example.com +10% flowers-sample-canary-bjdkm - flowers-sample.default.example.com + +Conditions + +Type Status Age Reason Message +AllTrafficAssigned True 46s - - +IngressReady True 45s - - +Ready True 45s - - + +Succeeded +``` diff --git a/docs/samples/tensorflow/tensorflow-canary.yaml b/docs/samples/tensorflow/tensorflow-canary.yaml new file mode 100644 index 00000000000..e105f19fed2 --- /dev/null +++ b/docs/samples/tensorflow/tensorflow-canary.yaml @@ -0,0 +1,14 @@ +apiVersion: "serving.kubeflow.org/v1alpha1" +kind: "KFService" +metadata: + name: "flowers-sample" +spec: + default: + # 90% of traffic is sent to this model + tensorflow: + modelUri: "gs://kfserving-samples/models/tensorflow/flowers" + canaryTrafficPercent: 10 + canary: + # 10% of traffic is sent to this model + tensorflow: + modelUri: "gs://kfserving-samples/models/tensorflow/flowers-2" diff --git a/python/sklearnserver/model.joblib b/python/sklearnserver/model.joblib deleted file mode 100644 index 8638baa4a22..00000000000 Binary files a/python/sklearnserver/model.joblib and /dev/null differ diff --git a/python/xgbserver/model.bst b/python/xgbserver/model.bst deleted file mode 100644 index cebe9408374..00000000000 Binary files a/python/xgbserver/model.bst and /dev/null differ