Inference autoscaling telemetry

[jan-elastic]

This PR add telemetry for inference adaptive autoscaling.

This feature needs gauge metrics labeled by deployment_id, comparable to the examples listed here:
https://opentelemetry.io/docs/specs/otel/metrics/api/#asynchronous-gauge-creation

Unfortunately, the existing telemetry framework in ES for gauges only allows single values, therefore I had to extend it.

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)

[elasticsearchmachine]

Hi @jan-elastic, I've created a changelog YAML for you.

[pgomulka]

I think this is something a broader team would like to discuss hence pinging @elastic/es-core-infra
My main worry would be if we want to change the signature of the methods to accept the collection of attributes or have a variant of methods

[jan-elastic]

I think this is something a broader team would like to discuss hence pinging @elastic/es-core-infra My main worry would be if we want to change the signature of the methods to accept the collection of attributes or have a variant of methods

Thanks for bringing them in the loop.

Note that the signatures of the public-facing methods (like registerDoubleGauge) have not changed, and I've added variants (like registerDoublesGauge (plural)).

Regarding the internal methods (like DoubleAsyncCounterAdapter), I think the current approach leads to less code duplication without disadvantages, but I'll leave that decision up to the @elastic/es-core-infra team.

[jan-elastic]

@pgomulka @elastic/es-core-infra Could you have a look at this PR? If possible, I'd like to wrap this up this week before I go on holiday. Thanks!

[mosche]

Sorry, for the delay on this @jan-elastic!
We've just discussed your addition to the metrics interface in core/infra, that part looks good to us 👍 Thanks for the contribution :)

[jan-elastic]

Thanks for the review!