add ONNX support (kubeflow#258)

* add readme for azure * add onnx as a fwk * add swagger gen files * delete extra reame * fix test args * fix test * regenerate files after merge * update with sample that works * use args correctly * add example * add instructions for own model * add onnx version * make notebook into a link * fix test * actually fix test * add newline * add clarification to readme * add requirements txt
magdalenakuhn17 · Sep 6, 2019 · 4d2cd70 · 4d2cd70
1 parent a697d83
commit 4d2cd70
Show file tree

Hide file tree

Showing 17 changed files with 2,322 additions and 4 deletions.
diff --git a/config/default/configmap/kfservice.yaml b/config/default/configmap/kfservice.yaml
@@ -9,6 +9,9 @@ data:
         "tensorflow": {
             "image": "tensorflow/serving"
         },
+        "onnx": {
+            "image": "mcr.microsoft.com/onnxruntime/server"
+        },
         "sklearn": {
             "image": "gcr.io/kfserving/sklearnserver"
         },

diff --git a/config/default/crds/serving_v1alpha2_kfservice.yaml b/config/default/crds/serving_v1alpha2_kfservice.yaml
@@ -119,6 +119,20 @@ spec:
                         to 0 in case of no traffic
                       format: int64
                       type: integer
+                    onnx:
+                      properties:
+                        modelUri:
+                          type: string
+                        resources:
+                          description: Defaults to requests and limits of 1CPU, 2Gb
+                            MEM.
+                          type: object
+                        runtimeVersion:
+                          description: Defaults to latest ONNX Version.
+                          type: string
+                      required:
+                      - modelUri
+                      type: object
                     pytorch:
                       properties:
                         modelClassName:
@@ -302,6 +316,20 @@ spec:
                         to 0 in case of no traffic
                       format: int64
                       type: integer
+                    onnx:
+                      properties:
+                        modelUri:
+                          type: string
+                        resources:
+                          description: Defaults to requests and limits of 1CPU, 2Gb
+                            MEM.
+                          type: object
+                        runtimeVersion:
+                          description: Defaults to latest ONNX Version.
+                          type: string
+                      required:
+                      - modelUri
+                      type: object
                     pytorch:
                       properties:
                         modelClassName:

diff --git a/docs/control-plane.md b/docs/control-plane.md
@@ -50,6 +50,7 @@ KFService offers a few high level specifications for common ML technologies. The
 | Field       | Value       | Description |
 | ----------- | ----------- | ----------- |
 | tensorflow  | [Tensorflow](#Tensorflow)   | A high level specification for Tensorflow models. |
+| onnx        | [ONNX](#ONNX)               | A high level specification for ONNX models. |
 | xgboost     | [XGBoost](#XGBoost)         | A high level specification for XGBoost models. |
 | scikitlearn | [ScikitLearn](#ScikitLearn) | A high level specification for ScikitLearn models. |
 | pytorch     | [Pytorch](#Pytorch)         | A high level specification for Pytorch models. |
@@ -143,6 +144,9 @@ Conditions provide realtime feedback to users on the underlying state of their d
 | runtimeVersion | String                                                                                             | Defaults to latest the version of Tensorflow. |
 | resources      | [Resources](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) | Defaults to requests and limits of 1CPU, 2Gb MEM. |
 
+### ONNX
+Currently, this is identical to [Tensorflow](#Tensorflow)
+
 ### XGBoost
 Currently, this is identical to [Tensorflow](#Tensorflow)
 

diff --git a/docs/samples/onnx/README.md b/docs/samples/onnx/README.md
@@ -0,0 +1,38 @@
+
+# Predict on a KFService using ONNX
+## Setup
+1. Your ~/.kube/config should point to a cluster with [KFServing installed](https://github.com/kubeflow/kfserving/blob/master/docs/DEVELOPER_GUIDE.md#deploy-kfserving).
+2. Your cluster's Istio Ingress gateway must be network accessible.
+3. Your cluster's Istio Egresss gateway must [allow Google Cloud Storage](https://knative.dev/docs/serving/outbound-network-access/)
+
+## Create the KFService
+Apply the CRD
+```
+kubectl apply -f onnx.yaml 
+```
+
+Expected Output
+```
+$ kfservice.serving.kubeflow.org/style-sample configured
+```
+
+## Run a sample inference
+1. Setup env vars
+```
+export SERVICE_URL=$(kubectl get kfservice ${MODEL_NAME} -o jsonpath='{.status.url}')
+```
+2. Verify the service is healthy
+```
+curl ${SERVICE_URL}
+```
+3. Install dependencies
+```
+pip install -r requirements.txt
+```
+4. Run the [sample notebook](mosaic-onnx.ipynb) in jupyter
+```
+jupyter notebook
+```
+
+## Uploading your own model
+The sample model for the example in this readme is already uploaded and available for use. However if you would like to modify the example to use your own ONNX model, all you need to do is to upload your model as `model.onnx` to S3, GCS or an Azure Blob.