-
Notifications
You must be signed in to change notification settings - Fork 120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ExecutedActionMetadata: Allow adding auxiliary metadata #154
ExecutedActionMetadata: Allow adding auxiliary metadata #154
Conversation
ExecutedActionMetadata provides a number of fields that help us to get insight in the worker that executed a build action, such as the name of the worker and some basic timing statistics. Buildbarn ships with a small local modification to ExecutedActionMetadata, adding a 'repeated google.protobuf.Any' field. This field is used by some of Buildbarn's components to embed even more metrics into ActionResult, such as POSIX getrusage(2) information and temporary space allocation stats: https://github.com/buildbarn/bb-remote-execution/blob/master/pkg/proto/resourceusage/resourceusage.proto By having this data embedded in the ActionResult, people can view this information by simply inspecting a build action through bb-browser. This makes it easier for people to diagnose build failures accurately. By using google.protobuf.Any, build clusters can expose their own metrics without requiring any further protocol extensions. It doesn't make sense to standardise those metrics, as they tend to be strongly implementation specific.
…zelbuild#154) Co-authored-by: jpoole <jpoole@thoughtmachine.net>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
General questions for followup from RE API monthly conversations. I'll loop back to comment more on these later.
-
Extensibility is good and desirable in general. It's possible this should be paired with some level of standardization, e.g., for POSIX-related data.
-
This message is fairly bloated and may not be very useful in the context of cache hits. We may want to consider moving it elsewhere or not returning it with cache hits.
-
Can probably approve this as-is and loop back on these questions later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While I prefer standardized fields for common statistics, I tend to agree with @bergsieker to not hold this up at this stage.
This allows us to get rid of a local patch. This one has been merged as part of PR #154: bazelbuild/remote-apis#154
Thanks for merging this @sstriker! @bergsieker If you want I can send a followup PR to add POSIXResourceUsage to remote_execution.proto. Or do we want to add this to some kind of separate proto file as part of this repository? |
I'm inclined to leave this as a generic field rather than breaking out POSIX data explicitly. (My comment above was a summary of discussion from the RE monthly, not my personal opinion.) So far we've managed to keep the API itself mostly free of platform-specific structures. I would anticipate that this data is generally used for logging purposes, not for active evaluation by the client, in which case I think there's not much point in overly standardizing it. |
Followup to bazelbuild#154, which forgot to update the bazel configuration. This makes hooks/pre-commit work again.
This change is missing a dependency which broke the bindings generation, fix here: #182 |
* Fix go bindings generation Followup to #154, which forgot to update the bazel configuration. This makes hooks/pre-commit work again. * Add a simple CI check that bindings generation works
* Fix go bindings generation Followup to bazelbuild#154, which forgot to update the bazel configuration. This makes hooks/pre-commit work again. * Add a simple CI check that bindings generation works
ExecutedActionMetadata provides a number of fields that help us to get insight in the worker that executed a build action, such as the name of the worker and some basic timing statistics.
Buildbarn ships with a small local modification to ExecutedActionMetadata, adding a 'repeated google.protobuf.Any' field. This field is used by some of Buildbarn's components to embed even more metrics into ActionResult, such as POSIX getrusage(2) information and temporary space allocation stats:
https://github.com/buildbarn/bb-remote-execution/blob/master/pkg/proto/resourceusage/resourceusage.proto
By having this data embedded in the ActionResult, people can view this information by simply inspecting a build action through bb-browser. This makes it easier for people to diagnose build failures accurately.
By using google.protobuf.Any, build clusters can expose their own metrics without requiring any further protocol extensions. It doesn't make sense to standardise those metrics, as they tend to be strongly implementation specific.