Prometheus serializer: remove HELP metadata and make TYPE metadata optional to cut down on traffic costs

[vpedosyuk]

Currently, the Prometheus serializer always adds HELP and TYPE metadata:

# HELP node_netstat_UdpLite6_InErrors Telegraf collected metric
# TYPE node_netstat_UdpLite6_InErrors untyped
node_netstat_UdpLite6_InErrors{host="vagrant",url="http://localhost:9100/metrics"} 0 1656092055020
# HELP node_power_supply_capacity Telegraf collected metric
# TYPE node_power_supply_capacity gauge
node_power_supply_capacity{host="vagrant",power_supply="BAT0",url="http://localhost:9100/metrics"} 77 1656092055020
# HELP go_memstats_alloc_bytes_total Telegraf collected metric
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total{host="vagrant",url="http://localhost:9100/metrics"} 3.810568e+06 1656092055020
# HELP node_memory_WritebackTmp_bytes Telegraf collected metric
# TYPE node_memory_WritebackTmp_bytes gauge
node_memory_WritebackTmp_bytes{host="vagrant",url="http://localhost:9100/metrics"} 0 1656092055020

The issue is HELP and TYPE info just adds useless paylod and increases traffic costs, especially, when operating with massive scale. According to the Prometheus exposition format, these fields are completely optional:
https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#text-format-example

Test

I generated ~1kk metrics of node-exporter in a file and tried to do some calculations:

$ wc -l metrics.prom
1008324 metrics.prom

$ grep -v 'TYPE' metrics.prom > no_type_metrics.prom
$ grep -v 'HELP' metrics.prom > no_help_metrics.prom
$ grep -v -E 'HELP|TYPE' metrics.prom > no_both_metrics.prom

$ ls -lh *.prom
-rw-r--r-- 1 vpedosyuk 72M Jun 24 20:58 metrics.prom
-rw-r--r-- 1 vpedosyuk 39M Jun 24 21:01 no_both_metrics.prom
-rw-r--r-- 1 vpedosyuk 52M Jun 24 21:00 no_help_metrics.prom
-rw-r--r-- 1 vpedosyuk 58M Jun 24 20:59 no_type_metrics.prom

$ gzip *.prom

$ ls -lh *.prom.gz
-rw-r--r-- 1 vpedosyuk 5.7M Jun 24 20:58 metrics.prom.gz
-rw-r--r-- 1 vpedosyuk 3.6M Jun 24 21:01 no_both_metrics.prom.gz
-rw-r--r-- 1 vpedosyuk 4.5M Jun 24 21:00 no_help_metrics.prom.gz
-rw-r--r-- 1 vpedosyuk 4.7M Jun 24 20:59 no_type_metrics.prom.gz

So the "no metadata" file was roughly 45% smaller in plain text format and 35% smaller in gzip. It'd be great to have the metadata disabled.

HELP metadata

It's always a kind of "dummy" string so probably it's worth removing. It's quite easy to do since expfmt can skip it if it's nil:
https://github.com/prometheus/common/blob/main/expfmt/text_create.go#L96
I've tried it and it worked fine for me locally.

TYPE metadata

As I see, it's harder to have it disabled as expfmt doesn't support it out-of-the-box:
https://github.com/prometheus/common/blob/main/expfmt/text_create.go#L123-L151
Not sure what would be the best option here.

[reimda]

I suspect some clients expect the help and type metadata even though they are officially optional. I think we could add a setting so users can choose to turn them off but I don't want to make it the default.

Are you able to put together a PR to add a setting that removes them? Thanks!

[vpedosyuk]

@reimda thanks for looking into it. I opened a PR with such an option, disabled by default. I'm not 100% sure that's the best way of implementing it, any suggestions are welcome.