Make prometheus serializer update timestamps and expiration time as new data arrives

[jakemcc]

• Updated associated README.md.
• Wrote appropriate unit tests.

resolves #8170

Replaces #8257

#8257 partially implements a fix for #8170 but this PR expands it to also resolve the expiration issue to the Summary type.

It also updates both the Histogram and Summary timestampMs field when new data arrives for a particular Histogram or Summary. I talk about this problem in this comment on #8170. Below expands on it.

If you have configured [[outputs.prometheus_client]] to export_timestamps = true then without the updating of timestampMs on new data coming in you end up in a situation like below.

Say you are pushing prometheus data to telegraf. The first batch might look something like below with a timestamp of 1617722541063

metric_name{application="the-application",quantile="0.5"} 1 1617722540000
metric_name{application="the-application",quantile="0.75"} 2 1617722540000
metric_name{application="the-application",quantile="0.95"} 3 1617722540000
metric_name{application="the-application",quantile="0.98"} 3.3 1617722540000
metric_name{application="the-application",quantile="0.99"} 3.4 1617722540000
metric_name{application="the-application",quantile="0.999"} 10 1617722540000
metric_name_sum{application="the-application"} 207.501625198 1617722540000
metric_name_count{application="the-application"} 18000 1617722540000

About 10 seconds later, you publish an update to telegraf.

metric_name{application="the-application",quantile="0.5"} 0.9  1617722550000
metric_name{application="the-application",quantile="0.75"} 1.5 1617722550000
metric_name{application="the-application",quantile="0.95"} 2 1617722550000
metric_name{application="the-application",quantile="0.98"} 3 1617722550000
metric_name{application="the-application",quantile="0.99"} 3.5 1617722550000
metric_name{application="the-application",quantile="0.999"} 10 1617722550000
metric_name_sum{application="the-application"} 310 1617722550000
metric_name_count{application="the-application"} 18281 1617722550000

If telegraf is then scrapped by prometheus and export_timestamp = true, the initial first received timestamp is exported instead of the correct second timestamp. Updating timestampMs as new data comes in resolves this issue.

[telegraf-tiger]

🤝 ✅ CLA has been signed. Thank you!

[ivorybilled]

Makes sense to me

[reimda]

It makes sense to me for expiration to be relative to to the last time a histogram or summary is gathered instead of the first time. Thanks @jakemcc for adding comprehensive tests for this.

[reimda]

Hey @ssoroka, you commented on #8170 a few months ago, would you like to review this fix?

[jakemcc]

We've been running with a fork that includes these changes since a little before I submitted this PR. It has been working well for us and, as expected, has resolved the gaps mentioned in the referenced issues.

[telegraf-tiger]

Looks like new artifacts were built from this PR. Get them here!

Artifact URLs

• aarch64.rpm

• armel.rpm

• armv6hl.rpm

• i386.rpm

• ppc64le.rpm

• s390x.rpm

• x86_64.rpm

• darwin_amd64.tar.gz

• freebsd_amd64.tar.gz

• freebsd_i386.tar.gz

• linux_amd64.tar.gz

• linux_arm64.tar.gz

• linux_armel.tar.gz

• linux_armhf.tar.gz

• linux_i386.tar.gz

• linux_mips.tar.gz

• linux_mipsel.tar.gz

• linux_ppc64le.tar.gz

• linux_s390x.tar.gz

• static_linux_amd64.tar.gz

• windows_amd64.zip

• windows_i386.zip

• amd64.deb

• arm64.deb

• armel.deb

• armhf.deb

• i386.deb

• mips.deb

• mipsel.deb

• ppc64el.deb

• s390x.deb

[jjh74]

@reimda / @ssoroka Any updates on this ?
I've been running a fork with subset/previous version of fix(#8257) for nearly a year. Without this PR prometheus histograms are not useful.