Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[prometheusremotewriteexporter] Support conversion of OTel Exponential Histogram to Prometheus Native Histogram #17370

Merged
merged 3 commits into from
Jan 12, 2023

Conversation

kovrus
Copy link
Member

@kovrus kovrus commented Jan 4, 2023

Description:

Support conversion of OTel Exponential Histogram to Prometheus Native Histogram.

pmetric.ExponentialHistogramDataPoint translates to prompb.Histogram as follows:

  • Exp. Scale -> Nat. Schema: histogram is dropped if Scale -4 < and > 8.
    TODO: We can potentially downscale hist to 8 if Scale > 8? (as a separate PR)
  • Exp. Count -> Nat. Count: if Flags().NoRecordedValue() is false
  • Exp. Sum_ -> Nat. Sum: if it Sum is set and Flags().NoRecordedValue() is false
  • Exp. TimeUnixNano -> Nat. Timestamp: TimeUnixNano ns converted Timestamp ms
  • Exp. ZeroCount -> Nat. ZeroCount
  • Nat. ZeroThreshold is set to the default value 1e-128, once Introduce an optional zero_threshold field to ExponentialHistogramDataPoint opentelemetry-proto#441 is merged, we can use Exp. ZeroThreshold to set the value.
  • Exp. Positive -> Nat. PositiveSpans and Nat. PositiveDeltas.
  • Exp. Negative -> Nat. NegativeSpans and Nat. NegativeDeltas.
  • Exp. Min_ and Max_ are not used.
  • Exp. StartTimeUnixNano is not used.

Exp. Exemplars, Labels, and ExponentialHistogramDataPoint will be translated to native hist data structures and will make up one or more prompb.TimeSeries.

Link to tracking Issue:
Resolves #16207

Testing:
Unit tests in pkg/transalator/prometheusremote and exporter/prometheusremotewriteexporter

Documentation:

open-telemetry/opentelemetry-specification#3079

@runforesight
Copy link

runforesight bot commented Jan 4, 2023

Foresight Summary

    
Major Impacts

build-and-test-windows duration(5 seconds) has decreased 44 minutes 19 seconds compared to main branch avg(44 minutes 24 seconds).
View More Details

⭕  build-and-test-windows workflow has finished in 5 seconds (44 minutes 19 seconds less than main branch avg.) and finished at 10th Jan, 2023.


Job Failed Steps Tests
windows-unittest-matrix -     🔗  N/A See Details
windows-unittest -     🔗  N/A See Details

✅  check-links workflow has finished in 46 seconds (1 minute 59 seconds less than main branch avg.) and finished at 10th Jan, 2023.


Job Failed Steps Tests
changed files -     🔗  N/A See Details
check-links -     🔗  N/A See Details

✅  tracegen workflow has finished in 1 minute 9 seconds (2 minutes 2 seconds less than main branch avg.) and finished at 10th Jan, 2023.


Job Failed Steps Tests
build-dev -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details
publish-latest -     🔗  N/A See Details

✅  changelog workflow has finished in 1 minute 24 seconds (8 minutes 4 seconds less than main branch avg.) and finished at 10th Jan, 2023.


Job Failed Steps Tests
changelog -     🔗  N/A See Details

✅  prometheus-compliance-tests workflow has finished in 3 minutes 16 seconds (5 minutes 41 seconds less than main branch avg.) and finished at 10th Jan, 2023.


Job Failed Steps Tests
prometheus-compliance-tests -     🔗  ✅ 21  ❌ 0  ⏭ 0    🔗 See Details

✅  build-and-test workflow has finished in 37 minutes 15 seconds (12 minutes 32 seconds less than main branch avg.) and finished at 10th Jan, 2023.


Job Failed Steps Tests
unittest-matrix (1.19, internal) -     🔗  ✅ 618  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, extension) -     🔗  ✅ 528  ❌ 0  ⏭ 0    🔗 See Details
correctness-metrics -     🔗  ✅ 2  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, processor) -     🔗  ✅ 1481  ❌ 0  ⏭ 0    🔗 See Details
correctness-traces -     🔗  ✅ 17  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, processor) -     🔗  ✅ 1481  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, internal) -     🔗  ✅ 618  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, extension) -     🔗  ✅ 528  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-0) -     🔗  ✅ 2565  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, receiver-0) -     🔗  ✅ 2565  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, exporter) -     🔗  ✅ 2464  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, exporter) -     🔗  ✅ 2464  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, other) -     🔗  ✅ 4445  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-1) -     🔗  ✅ 1883  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, other) -     🔗  ✅ 4445  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, receiver-1) -     🔗  ✅ 1883  ❌ 0  ⏭ 0    🔗 See Details
integration-tests -     🔗  ✅ 55  ❌ 0  ⏭ 0    🔗 See Details
setup-environment -     🔗  N/A See Details
check-codeowners -     🔗  N/A See Details
build-examples -     🔗  N/A See Details
check-collector-module-version -     🔗  N/A See Details
checks -     🔗  N/A See Details
lint-matrix (receiver-0) -     🔗  N/A See Details
lint-matrix (receiver-1) -     🔗  N/A See Details
lint-matrix (processor) -     🔗  N/A See Details
lint-matrix (exporter) -     🔗  N/A See Details
lint-matrix (extension) -     🔗  N/A See Details
lint-matrix (internal) -     🔗  N/A See Details
lint-matrix (other) -     🔗  N/A See Details
lint -     🔗  N/A See Details
unittest (1.19) -     🔗  N/A See Details
unittest (1.18) -     🔗  N/A See Details
cross-compile (darwin, amd64) -     🔗  N/A See Details
cross-compile (darwin, arm64) -     🔗  N/A See Details
cross-compile (linux, 386) -     🔗  N/A See Details
cross-compile (linux, amd64) -     🔗  N/A See Details
cross-compile (linux, arm) -     🔗  N/A See Details
cross-compile (linux, arm64) -     🔗  N/A See Details
cross-compile (linux, ppc64le) -     🔗  N/A See Details
cross-compile (windows, 386) -     🔗  N/A See Details
cross-compile (windows, amd64) -     🔗  N/A See Details
build-package (deb) -     🔗  N/A See Details
build-package (rpm) -     🔗  N/A See Details
windows-msi -     🔗  N/A See Details
publish-check -     🔗  N/A See Details
publish-dev -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details

✅  load-tests workflow has finished in 7 minutes 57 seconds (8 minutes 11 seconds less than main branch avg.) and finished at 10th Jan, 2023.


Job Failed Steps Tests
loadtest (TestTraceAttributesProcessor) -     🔗  ✅ 3  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestIdleMode) -     🔗  ✅ 1  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceNoBackend10kSPS|TestTrace1kSPSWithAttrs) -     🔗  ✅ 8  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetricResourceProcessor|TestTrace10kSPS) -     🔗  ✅ 12  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceBallast1kSPSWithAttrs|TestTraceBallast1kSPSAddAttrs) -     🔗  ✅ 10  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetric10kDPS|TestMetricsFromFile) -     🔗  ✅ 6  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestBallastMemory|TestLog10kDPS) -     🔗  ✅ 19  ❌ 0  ⏭ 0    🔗 See Details
setup-environment -     🔗  N/A See Details

🔎 See details on Foresight

*You can configure Foresight comments in your organization settings page.

@kovrus kovrus force-pushed the issue-16207 branch 4 times, most recently from 84ef580 to a610f4f Compare January 4, 2023 14:16
@kovrus kovrus marked this pull request as ready for review January 4, 2023 14:19
@kovrus kovrus requested a review from a team January 4, 2023 14:19
@kovrus
Copy link
Member Author

kovrus commented Jan 4, 2023

cc: @fstab @gouthamve @codesome

@gouthamve
Copy link
Member

cc @jmacd

@Aneurysm9 Aneurysm9 added the ready to merge Code review completed; ready to merge by maintainers label Jan 5, 2023
Copy link
Member

@gouthamve gouthamve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@codeboten codeboten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rebase and we can get this merged

@kovrus
Copy link
Member Author

kovrus commented Jan 10, 2023

@codeboten rebased.

I had to skip the flaky prw exporter TestWALOnExporterRoundTrip test on Linux as well. It was already skipped for Windows, see #10142

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

prometheusremotewrite: support for converting OTLP exponential histogram to Prometheus Native Histograms
7 participants