Skip to content

Commit

Permalink
protocolVersion and old schema updated with onnx example.
Browse files Browse the repository at this point in the history
Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>
  • Loading branch information
andyi2it committed Nov 8, 2023
1 parent 44d819d commit 85c7002
Show file tree
Hide file tree
Showing 4 changed files with 28 additions and 11 deletions.
33 changes: 23 additions & 10 deletions docs/modelserving/v1beta1/onnx/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,37 @@
2. Your cluster's Istio Ingress gateway must be [network accessible](https://istio.io/latest/docs/tasks/traffic-management/ingress/ingress-control/).

## Create the InferenceService
Apply the CRD

```shell
kubectl apply -f - <<EOF
=== "New Schema"

```yaml
apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
name: "style-sample"
name: "style-sample"
spec:
predictor:
predictor:
model:
modelFormat:
protocolVersion: v2
modelFormat:
name: onnx
storageUri: "gs://kfserving-examples/models/onnx"
EOF
```
storageUri: "gs://kfserving-examples/models/onnx"
```

=== "Old Schema"

```yaml
apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
name: "style-sample"
spec:
predictor:
onnx:
storageUri: "gs://kfserving-examples/models/onnx"
```
!!! Note
For the default kserve installation, While using new schema, you must specify **protocolVersion** as v2 for onnx. Otherwise, you will get a no runtime found error.
Expected Output
```
$ inferenceservice.serving.kserve.io/style-sample configured
Expand All @@ -33,7 +47,6 @@ $ inferenceservice.serving.kserve.io/style-sample configured
The first step is to [determine the ingress IP and ports](https://kserve.github.io/website/master/get_started/first_isvc/#4-determine-the-ingress-ip-and-ports) and set `INGRESS_HOST` and `INGRESS_PORT`

```
export MODEL_NAME=onnx-model
export ISVC_NAME=style-sample
export SERVICE_HOSTNAME=$(kubectl get inferenceservice ${ISVC_NAME} -o jsonpath='{.status.url}' | cut -d "/" -f 3)
```
Expand Down
4 changes: 3 additions & 1 deletion docs/modelserving/v1beta1/onnx/mosaic-onnx.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,10 @@
]
},
{
"cell_type": "markdown",
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# load & resize image\n",
"image = Image.open(\"image.jpg\")\n",
Expand Down
1 change: 1 addition & 0 deletions docs/modelserving/v1beta1/onnx/onnx.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ metadata:
spec:
predictor:
model:
protocolVersion: v2
modelFormat:
name: onnx
storageUri: "gs://kfserving-examples/models/onnx"
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ nav:
- LightGBM: modelserving/v1beta1/lightgbm/README.md
- Paddle: modelserving/v1beta1/paddle/README.md
- MLFlow: modelserving/v1beta1/mlflow/v2/README.md
- ONNX: modelserving/v1beta1/onnx/README.md
- Multi-Framework Serving Runtimes:
- Nvidia Triton:
- Torchscript: modelserving/v1beta1/triton/torchscript/README.md
Expand Down

0 comments on commit 85c7002

Please sign in to comment.