Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OTLP support for exporting exponential histograms #4337

Merged
merged 22 commits into from
Mar 28, 2023

Conversation

alanwest
Copy link
Member

This PR makes the Base2ExponentialHistogramConfiguration public. Support is added to the OTLP exporter for exporting exponential histograms.

Follow ups for other exporters

Minimal work is done in this PR to the console and Prometheus exporters so that they don't blow up. Follow ups to consider:

Prometheus exporter

Currently, the exporter ignores exponential histograms. The collector exporter logs an error. We could do the same, though it may occur frequently?

Example:

2023-03-27T18:23:00.908Z error prometheusexporter@v0.74.0/accumulator.go:105 failed to translate metric {"kind": "exporter", "data_type": "metrics", "name": "prometheus", "data_type": "\u0004", "metric_name": "http.server.duration"}

Console exporter

With this PR only the sum, count, min, and max are displayed. Buckets are not yet displayed. Two options:

  1. Java's console exporter supports OTLP/JSON format, so the bucket counts just show up as an array of numbers. The JSON looks roughly like:

    "http.server.duration":{"dataPoints":[{"timeUnixNano":"10","count":"6","sum":10.0,"min":1.0,"max":5.0,"positive":{"offset":10,"bucketCounts":["1","2","3"]},"negative":{},"exemplars":[],"attributes":[],"aggregationTemporality":1}

  2. The collector's logging exporter calculates the bucket boundaries for display. This can become very lengthy

Collector logging exporter example
Metric #0
Descriptor:
     -> Name: http.server.duration
     -> Description: Measures the duration of inbound HTTP 
     -> Unit: ms
     -> DataType: ExponentialHistogram
     -> AggregationTemporality: Cumulative
ExponentialHistogramDataPoints #0
Data point attributes:
     -> http.flavor: Str(1.1)
     -> http.method: Str(GET)
     -> http.route: Str(WeatherForecast)
     -> http.scheme: Str(https)
     -> http.status_code: Int(200)
     -> net.host.name: Str(localhost)
     -> net.host.port: Int(5001)
StartTimestamp: 2023-03-27 18:20:00.912126 +0000 UTC
Timestamp: 2023-03-27 18:23:00.890708 +0000 UTC
Count: 9
Sum: 2854.493000
Min: 161.128000
Max: 523.119000
Bucket [160.688737, 162.438523), Count: 1
Bucket [162.438523, 164.207362), Count: 0
Bucket [164.207362, 165.995463), Count: 0
Bucket [165.995463, 167.803035), Count: 0
Bucket [167.803035, 169.630290), Count: 0
Bucket [169.630290, 171.477443), Count: 0
Bucket [171.477443, 173.344710), Count: 0
Bucket [173.344710, 175.232310), Count: 0
Bucket [175.232310, 177.140465), Count: 0
Bucket [177.140465, 179.069398), Count: 0
Bucket [179.069398, 181.019336), Count: 0
Bucket [181.019336, 182.990507), Count: 0
Bucket [182.990507, 184.983143), Count: 0
Bucket [184.983143, 186.997478), Count: 0
Bucket [186.997478, 189.033747), Count: 0
Bucket [189.033747, 191.092189), Count: 0
Bucket [191.092189, 193.173047), Count: 0
Bucket [193.173047, 195.276563), Count: 0
Bucket [195.276563, 197.402986), Count: 0
Bucket [197.402986, 199.552563), Count: 0
Bucket [199.552563, 201.725548), Count: 0
Bucket [201.725548, 203.922195), Count: 0
Bucket [203.922195, 206.142762), Count: 0
Bucket [206.142762, 208.387510), Count: 0
Bucket [208.387510, 210.656701), Count: 0
Bucket [210.656701, 212.950602), Count: 0
Bucket [212.950602, 215.269482), Count: 0
Bucket [215.269482, 217.613613), Count: 0
Bucket [217.613613, 219.983270), Count: 0
Bucket [219.983270, 222.378731), Count: 0
Bucket [222.378731, 224.800277), Count: 0
Bucket [224.800277, 227.248191), Count: 0
Bucket [227.248191, 229.722762), Count: 1
Bucket [229.722762, 232.224278), Count: 0
Bucket [232.224278, 234.753035), Count: 0
Bucket [234.753035, 237.309328), Count: 1
Bucket [237.309328, 239.893457), Count: 0
Bucket [239.893457, 242.505726), Count: 0
Bucket [242.505726, 245.146440), Count: 0
Bucket [245.146440, 247.815910), Count: 0
Bucket [247.815910, 250.514448), Count: 0
Bucket [250.514448, 253.242371), Count: 0
Bucket [253.242371, 256.000000), Count: 0
Bucket [256.000000, 258.787657), Count: 0
Bucket [258.787657, 261.605670), Count: 0
Bucket [261.605670, 264.454369), Count: 1
Bucket [264.454369, 267.334088), Count: 0
Bucket [267.334088, 270.245166), Count: 0
Bucket [270.245166, 273.187943), Count: 0
Bucket [273.187943, 276.162764), Count: 0
Bucket [276.162764, 279.169980), Count: 1
Bucket [279.169980, 282.209941), Count: 0
Bucket [282.209941, 285.283006), Count: 0
Bucket [285.283006, 288.389534), Count: 0
Bucket [288.389534, 291.529890), Count: 0
Bucket [291.529890, 294.704443), Count: 0
Bucket [294.704443, 297.913564), Count: 0
Bucket [297.913564, 301.157630), Count: 0
Bucket [301.157630, 304.437021), Count: 0
Bucket [304.437021, 307.752123), Count: 0
Bucket [307.752123, 311.103324), Count: 0
Bucket [311.103324, 314.491017), Count: 0
Bucket [314.491017, 317.915600), Count: 0
Bucket [317.915600, 321.377474), Count: 0
Bucket [321.377474, 324.877045), Count: 0
Bucket [324.877045, 328.414724), Count: 0
Bucket [328.414724, 331.990926), Count: 0
Bucket [331.990926, 335.606070), Count: 0
Bucket [335.606070, 339.260581), Count: 0
Bucket [339.260581, 342.954886), Count: 0
Bucket [342.954886, 346.689420), Count: 1
Bucket [346.689420, 350.464620), Count: 0
Bucket [350.464620, 354.280930), Count: 0
Bucket [354.280930, 358.138796), Count: 1
Bucket [358.138796, 362.038672), Count: 0
Bucket [362.038672, 365.981015), Count: 0
Bucket [365.981015, 369.966287), Count: 0
Bucket [369.966287, 373.994955), Count: 0
Bucket [373.994955, 378.067493), Count: 0
Bucket [378.067493, 382.184378), Count: 0
Bucket [382.184378, 386.346093), Count: 0
Bucket [386.346093, 390.553127), Count: 0
Bucket [390.553127, 394.805971), Count: 0
Bucket [394.805971, 399.105126), Count: 0
Bucket [399.105126, 403.451096), Count: 0
Bucket [403.451096, 407.844391), Count: 0
Bucket [407.844391, 412.285525), Count: 0
Bucket [412.285525, 416.775020), Count: 0
Bucket [416.775020, 421.313402), Count: 0
Bucket [421.313402, 425.901205), Count: 0
Bucket [425.901205, 430.538965), Count: 0
Bucket [430.538965, 435.227227), Count: 0
Bucket [435.227227, 439.966540), Count: 0
Bucket [439.966540, 444.757462), Count: 0
Bucket [444.757462, 449.600553), Count: 0
Bucket [449.600553, 454.496382), Count: 0
Bucket [454.496382, 459.445523), Count: 0
Bucket [459.445523, 464.448557), Count: 0
Bucket [464.448557, 469.506070), Count: 1
Bucket [469.506070, 474.618656), Count: 0
Bucket [474.618656, 479.786914), Count: 0
Bucket [479.786914, 485.011451), Count: 0
Bucket [485.011451, 490.292880), Count: 0
Bucket [490.292880, 495.631819), Count: 0
Bucket [495.631819, 501.028896), Count: 0
Bucket [501.028896, 506.484743), Count: 0
Bucket [506.484743, 512.000000), Count: 0
Bucket [512.000000, 517.575314), Count: 0
Bucket [517.575314, 523.211340), Count: 1

@alanwest alanwest requested a review from a team March 27, 2023 19:26
@codecov
Copy link

codecov bot commented Mar 27, 2023

Codecov Report

Merging #4337 (78ac7b4) into main (87a3313) will increase coverage by 0.02%.
The diff coverage is 54.71%.

❗ Current head 78ac7b4 differs from pull request most recent head d6cddeb. Consider uploading reports for the commit d6cddeb to get more accurate results

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4337      +/-   ##
==========================================
+ Coverage   84.67%   84.70%   +0.02%     
==========================================
  Files         298      298              
  Lines       11937    11970      +33     
==========================================
+ Hits        10108    10139      +31     
- Misses       1829     1831       +2     
Impacted Files Coverage Δ
...elemetry.Exporter.Console/ConsoleMetricExporter.cs 0.00% <0.00%> (ø)
...cs/Base2ExponentialBucketHistogramConfiguration.cs 83.33% <ø> (ø)
...tryProtocol/Implementation/MetricItemExtensions.cs 85.24% <96.42%> (+2.12%) ⬆️
...s.HttpListener/Internal/PrometheusSerializerExt.cs 100.00% <100.00%> (ø)

... and 3 files with indirect coverage changes

…pentelemetry-dotnet into alanwest/exp-histogram-export
@alanwest alanwest merged commit 410532a into open-telemetry:main Mar 28, 2023
@alanwest alanwest deleted the alanwest/exp-histogram-export branch March 28, 2023 04:23
@alanwest alanwest mentioned this pull request Apr 26, 2023
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants