[exporter/prometheus] Fix panic when mutating data

[crobert-1]

Description:

The prometheus exporter hit a panic when accumulating Delta metrics into Cumulative sums. This is because the exporter does not enable mutating data in its capability. This change enables the exporter to mutate data in a safe and supported way.

Link to tracking Issue:
Fixed #29574

Testing
There are existing tests that hit the logic that was panicking, but the metrics are set to StateMutable in testing (which is the only way they can be created and setup for testing). I believe that means that before this change the tests were invalid (didn't represent reality), but after this change they'll properly represent the exporter's functionality.

[bryan-aguilar]

I believe that means that before this change the tests were invalid (didn't represent reality)

Is there any way to format/change the tests so that this isn't the case?

[crobert-1]

I believe that means that before this change the tests were invalid (didn't represent reality)

Is there any way to format/change the tests so that this isn't the case?

I looked into it a bit, I couldn't find a way with existing tests. The way to test this is to create the exporter and be able to consume test data and ensure that the test data has its state set to Mutable. However, all test methods have to set the data to Mutable so they can actually create it.

I think the best way for this to work would be attempting to use the telemetrygen tool, if it can create metrics in a more realistic way, and then data can be passed through a collector pipeline with the Prometheus exporter. Then we could check metrics coming in to see if they're set to mutable or not.

I was able to test manually though and confirmed that metrics were ReadOnly before the change and Mutable after.

[atoulme]

Please rebase