Skip to content

Commit

Permalink
docs: Indicate which model-runtime proto SPI rpcs are optional (#50)
Browse files Browse the repository at this point in the history
#### Motivation

In the model server gRPC service provider interface, implementation of the `predictModelSize` RPC is optional, and `modelSize` doesn't need to be implemented if sizes are always returned in the `loadModel` response.

This is documented in the [modelmesh docs](https://github.com/kserve/modelmesh-serving/blob/main/docs/runtimes/custom_runtimes.md#model-sizing) but not in the comments of the service .proto file.

#### Modifications

Add appropriate wording to doc comments in the `model-runtime.proto` file to convey the fact these rpcs aren't required.

#### Result

Clearer docs

Signed-off-by: Nick Hill <nickhill@us.ibm.com>

Signed-off-by: Nick Hill <nickhill@us.ibm.com>
  • Loading branch information
njhill authored Aug 2, 2022
1 parent 69c002d commit 77dbeab
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion src/main/proto/current/model-runtime.proto
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,14 @@ service ModelRuntime {
// Predict size of not-yet-loaded model - must return almost immediately.
// Should not perform expensive computation or remote lookups.
// Should be a conservative estimate.
// NOTE: Implementation of this RPC is optional.
rpc predictModelSize (PredictModelSizeRequest) returns (PredictModelSizeResponse) {}

// Calculate size (memory consumption) of currently-loaded model
// Calculate size (memory consumption) of currently-loaded model.
// NOTE: Implementation of this RPC is only required if models' size
// is not returned in the response to loadModel. If the size computation
// takes a nontrivial amount of time, it's better to return from loadModel
// immediately and implement this to perform the sizing separately.
rpc modelSize (ModelSizeRequest) returns (ModelSizeResponse) {}

// Provide basic runtime status and parameters; called only during startup.
Expand Down

0 comments on commit 77dbeab

Please sign in to comment.