influxdbexporer creates metric exports that are not accepted by influx

[otbe]

Component(s)

exporter/influxdb

What happened?

Description

After #27084 was fixed I run into the next issue with a v1 influxdb exporter. Every metric which is exported via the javaagent micrometer instrumentation fails to be exported via the collector influxdb exporter with a similar message:

unable to parse 'http.server.duration,aws.ecs.container.arn=arn:aws:ecs:eu-central-1:foo:container/bar/baz/boo,aws.ecs.container.image.id=sha256:foo,aws.ecs.launchtype=fargate,aws.ecs.task.arn=arn:aws:ecs:eu-central-1:foo:task/foo/bar,aws.ecs.task.family=bar,aws.ecs.task.revision=1,cloud.platform=aws_ecs,cloud.provider=aws,container.id=baz-bar,container.image.name=my-rep,container.image.tag=my-tgag,container.name=app,host.arch=amd64,host.name=host,http.method=GET,http.route=/actuator/health,http.scheme=http,http.status_code=200,http.target=/actuator/health,net.host.name=localhost,net.host.port=8080,net.protocol.name=http,net.protocol.version=1.1,net.sock.host.addr=127.0.0.1,net.sock.peer.addr=127.0.0.1,net.sock.peer.port=44578,os.description=Linux\\\\\\\\ 5.10.201-191.748.amzn2.x86_64,os.type=linux,otel.library.name=io.opentelemetry.tomcat-10.0,otel.library.version=1.31.0-alpha,process.command_args=[\\\\\\\"/usr/lib/jvm/java-17-amazon-corretto/bin/java\\\\\\\"\\\\\\\\,\\\\\\\"-Dotel.instrumentation.logback-appender.experimental-log-attributes\\\\\\\\=true\\\\\\\"\\\\\\\\,\\\\\\\"-Dotel.instrumentation.logback-appender.experimental.capture-mdc-attributes\\\\\\\\=*\\\\\\\"\\\\\\\\,\\\\\\\"-Dotel.instrumentation.logback-appender.experimental.capture-logger-context-attributes\\\\\\\\=true\\\\\\\"\\\\\\\\,\\\\\\\"-Dotel.instrumentation.logback-appender.experimental.capture-logger-context-attributes\\\\\\\\=true\\\\\\\"\\\\\\\\,\\\\\\\"-javaagent:/home/service/aws-opentelemetry-agent.jar\\\\\\\"\\\\\\\\,\\\\\\\"-XX:MaxRAMPercentage\\\\\\\\=80.0\\\\\\\"\\\\\\\\,\\\\\\\"-jar\\\\\\\"\\\\\\\\,\\\\\\\"-Djava.security.egd\\\\\\\\=file:/dev/./urandom\\\\\\\"\\\\\\\\,\\\\\\\"/home/service/my-service.jar\\\\\\\"],process.executable.path=/usr/lib/jvm/java-17-amazon-corretto/bin/java,process.pid=42,process.runtime.description=Amazon.com\\\\\\\\ Inc.\\\\\\\\ OpenJDK\\\\\\\\ 64-Bit\\\\\\\\ Server\\\\\\\\ VM\\\\\\\\ 17.0.9+8-LTS,process.runtime.name=OpenJDK\\\\\\\\ Runtime\\\\\\\\ Environment,process.runtime.version=17.0.9+8-LTS,service.name=my-service,telemetry.auto.version=1.31.1-aws,telemetry.sdk.language=java,telemetry.sdk.name=opentelemetry,telemetry.sdk.version=1.31.0,user_agent.original=curl/7.64.0 7500=1,flags=0u,max=2.833621,0=0,250=1,500=1,5=1,750=1,min=2.833621,50=1,5000=1,25=1,10=1,start_time_unix_nano=1702566995156664494i,100=1,+Inf=1,sum=2.833621,1000=1,count=1,75=1,2500=1,10000=1 1702589255161648942': invalid number

Steps to Reproduce

Similar to #27084 but with a SB3.2 app producing the metrics via otel javaagent.

Expected Result

In general I would love if it just works. When I put the line from above into a line protocol parser it fails, it might be related to the escaping. However somehow Im not able to properly filter for tags to be added. More tags increase the cardinality in influx which is in general a bad thing. I cant really filter for each tag by name just to get more tags over time at the end. So is there a way to not send any tags apart from some that are explicitly whitelisted? I was expecting something similar to span_dimensions and log_record_dimensions to make it.

Collector version

0.91.0

Environment information

Environment

OS: otel/opentelemetry-collector-contrib:0.91.0

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

[github-actions]

Pinging code owners:

• exporter/influxdb: @jacobmarble

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[padraic-padraic]

Encountering the same bug on v0.91.0 and InfluxDB 1.8, with data from the dockerstatsreceiver plugin...

[padraic-padraic]

As an update: The regression seems to occur in the 0.85.0 release of otel-contrib. With 0.84.0, my config works fine. Not that I encounter the issue whether I set the token to use basic auth, or if I explicitly set the v1_compatibility block.

receivers:
  hostmetrics:
    collection_interval: 30s
    root_path: /rootfs
    scrapers:
      cpu:
      memory:
      filesystem:
        metrics:
            system.filesystem.utilization:
                enabled: true
      process:
        metrics:
          process.memory.utilization:
            enabled: true
  docker_stats:

processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 10
    spike_limit_percentage: 7
  attributes/host:
    actions:
      - key: host
        action: insert
        value: "dummyhost"
  batch:
    timeout: 10s

exporters:
  influxdb:
    endpoint: "dummyendpoint:8086"
    bucket: "v1_database_name"
    token: "username:password"
    org: "dummy_val"

service:
  pipelines:
    metrics:
      receivers: [hostmetrics, docker_stats]
      processors: [memory_limiter, attributes/host, batch]
      exporters: [influxdb]

[ngergis]

I also encounter the same bug on v0.91.0 and InfluxDB 1.8, I am using javaagent auto instrumentation.
Further investigation: The problem occurs because flags=0u is added to request of influxdb although there is no attribute named flags coming from the receiver, so it is added by influxdbexporer.
I tested same request manually and I got same error, once deleting flags=0u or changing it to flags=0, influxdb write request works.

Thanks @padraic-padraic I have tested it on 0.84.0 and it works

[crobert-1]

I was able to look at some InfluxDB documentation, it looks like u is appended to field values in the InfluxDB line protocol to denote unsigned values. This is valid for InfluxDB v2, but documentation for v1 shows that uints are not supported. I can't tell at this point why it just started breaking in >v0.85.0 of the collector.

The solution on the collector side of things would be to update the write logic in the writer to account for the added u suffix, and remove it when running in v1-compatibility mode.

I'm not familiar with InfluxDB, but is there a requirement to continue using v1? Looking from the outside, there haven't been any v1 releases for more than 2 years now, so I'm wondering how much longer it makes sense to support it on the collector side of things.

Related issues: influxdata/influxdb#17781, influxdata/influxdb-client-go#143

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/influxdb: @jacobmarble

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[tjkozakar]

Hi. Is there any update on this issue, or a potential workaround? Is v1 compatibility now totally off the table?

Trying to insert some StatsD metrics into InfluxDB v1.8 & seeing a similar problem.

[yasha145]

Hello there,
I am getting the same issue, influxdb exporting data but it is only visible at Influx CLI and not on InfluxDB UI (I tried changing the time range as well), only the measurement is getting creating and not the fields in UI while in InfluxDB CLI eveything is been displayed properly.

[padraic-padraic]

Also checking in to see if there's any progress on this issue. Despite being an 'old' version, the InfluxDB v1 API is still relevant, especially as InfluxDB themselves are advising using v1.x in preparation for the transition to the v3 open source release.

[jacobmarble]

I can see that using unsigned integers in the OpenTelemetry conversion is not a good idea. Fortunately, within the Metrics signal, only the otel flags field has type uint, and that field is safe to drop.