[ML] add deployed native models to inference_stats in trained model stats response #88187
Conversation
|
Hi @benwtrent, I've created a changelog YAML for you. |
| public InferenceStats getOverallInferenceStats() { | ||
| return new InferenceStats( | ||
| 0L, | ||
| nodeStats.stream().filter(n -> n.getInferenceCount().isPresent()).mapToLong(n -> n.getInferenceCount().get()).sum(), |
There was a problem hiding this comment.
inference_count is the only duplicate field between these two stats objects.
But, it seems weird to remove it from the assignment stats. Having just a single duplicate field doesn't seem that big of a deal to me.
@dimitris-athanasiou @davidkyle what say you?
There was a problem hiding this comment.
It actually isn't a duplicate as it is the sum of inferences across all nodes. I think that's useful
There was a problem hiding this comment.
It is a duplicate but I don't think that's a problem. AssignmentStats collects various counts across all nodes
|
Pinging @elastic/ml-core (Team:ML) |
…nwtrent/elasticsearch into feature/ml-fix-pytorch-stats-response
|
@elasticmachine update branch |
…nwtrent/elasticsearch into feature/ml-fix-pytorch-stats-response
|
Btw, I think a unit test that checks we sum the node inference counts and failures might be useful to add. |
…torch-stats-response
correct @davidkyle |
This adds a valid
inference_statssection for deployed native models.inference_statsis effectively a sub-set of thedeployment_stats. It's a high level view of the overall stats of the model, deployment_stats contains more detailed information around types of errors seen, throughput, etc.