From 77dbeab5e33de2d986a9c29647ae8141f13fe051 Mon Sep 17 00:00:00 2001 From: Nick Hill Date: Mon, 1 Aug 2022 17:33:17 -0700 Subject: [PATCH] docs: Indicate which model-runtime proto SPI rpcs are optional (#50) #### Motivation In the model server gRPC service provider interface, implementation of the `predictModelSize` RPC is optional, and `modelSize` doesn't need to be implemented if sizes are always returned in the `loadModel` response. This is documented in the [modelmesh docs](https://github.com/kserve/modelmesh-serving/blob/main/docs/runtimes/custom_runtimes.md#model-sizing) but not in the comments of the service .proto file. #### Modifications Add appropriate wording to doc comments in the `model-runtime.proto` file to convey the fact these rpcs aren't required. #### Result Clearer docs Signed-off-by: Nick Hill Signed-off-by: Nick Hill --- src/main/proto/current/model-runtime.proto | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/main/proto/current/model-runtime.proto b/src/main/proto/current/model-runtime.proto index efa1b2d1..a3856c4b 100644 --- a/src/main/proto/current/model-runtime.proto +++ b/src/main/proto/current/model-runtime.proto @@ -43,9 +43,14 @@ service ModelRuntime { // Predict size of not-yet-loaded model - must return almost immediately. // Should not perform expensive computation or remote lookups. // Should be a conservative estimate. + // NOTE: Implementation of this RPC is optional. rpc predictModelSize (PredictModelSizeRequest) returns (PredictModelSizeResponse) {} - // Calculate size (memory consumption) of currently-loaded model + // Calculate size (memory consumption) of currently-loaded model. + // NOTE: Implementation of this RPC is only required if models' size + // is not returned in the response to loadModel. If the size computation + // takes a nontrivial amount of time, it's better to return from loadModel + // immediately and implement this to perform the sizing separately. rpc modelSize (ModelSizeRequest) returns (ModelSizeResponse) {} // Provide basic runtime status and parameters; called only during startup.