Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Indicate which model-runtime proto SPI rpcs are optional #50

Merged
merged 1 commit into from
Aug 2, 2022
Merged

docs: Indicate which model-runtime proto SPI rpcs are optional #50

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion src/main/proto/current/model-runtime.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,14 @@ service ModelRuntime {
// Predict size of not-yet-loaded model - must return almost immediately.
// Should not perform expensive computation or remote lookups.
// Should be a conservative estimate.
// NOTE: Implementation of this RPC is optional.
rpc predictModelSize (PredictModelSizeRequest) returns (PredictModelSizeResponse) {}

// Calculate size (memory consumption) of currently-loaded model
// Calculate size (memory consumption) of currently-loaded model.
// NOTE: Implementation of this RPC is only required if models' size
// is not returned in the response to loadModel. If the size computation
// takes a nontrivial amount of time, it's better to return from loadModel
// immediately and implement this to perform the sizing separately.
rpc modelSize (ModelSizeRequest) returns (ModelSizeResponse) {}

// Provide basic runtime status and parameters; called only during startup.
Expand Down