Large number of OTel instrumentation dependencies installed by Sentry

[nwalters512]

In recent releases, the number of OpenTelemetry instrumentation packages installed by @sentry/node has ballooned to 21:

    "@opentelemetry/instrumentation-amqplib": "npm:^0.42.0"
    "@opentelemetry/instrumentation-connect": "npm:0.39.0"
    "@opentelemetry/instrumentation-dataloader": "npm:0.12.0"
    "@opentelemetry/instrumentation-express": "npm:0.42.0"
    "@opentelemetry/instrumentation-fastify": "npm:0.39.0"
    "@opentelemetry/instrumentation-fs": "npm:0.15.0"
    "@opentelemetry/instrumentation-generic-pool": "npm:0.39.0"
    "@opentelemetry/instrumentation-graphql": "npm:0.43.0"
    "@opentelemetry/instrumentation-hapi": "npm:0.41.0"
    "@opentelemetry/instrumentation-http": "npm:0.53.0"
    "@opentelemetry/instrumentation-ioredis": "npm:0.43.0"
    "@opentelemetry/instrumentation-kafkajs": "npm:0.3.0"
    "@opentelemetry/instrumentation-koa": "npm:0.43.0"
    "@opentelemetry/instrumentation-mongodb": "npm:0.47.0"
    "@opentelemetry/instrumentation-mongoose": "npm:0.42.0"
    "@opentelemetry/instrumentation-mysql": "npm:0.41.0"
    "@opentelemetry/instrumentation-mysql2": "npm:0.41.0"
    "@opentelemetry/instrumentation-nestjs-core": "npm:0.40.0"
    "@opentelemetry/instrumentation-pg": "npm:0.44.0"
    "@opentelemetry/instrumentation-redis-4": "npm:0.42.0"
    "@opentelemetry/instrumentation-undici": "npm:0.6.0"

These comprise many megabytes of dependencies that increase install times and bloat the size of our container images with absolutely zero value for our team, application, or users.

Is there any chance that we'll get a @sentry/node-core or similar package that doesn't include this OpenTelemetry bloat? This would mirror what OpenTelemetry itself does: @opentelemetry/auto-instrumentations-node can be used by anyone who wants all possible instrumentation, but one can still piece together their own setup by manually installing and configuring just the constituent packages that they need.

If not: what'll happen in the long run here? The official OTel autoinstrumentation package is up to 39 @opentelemetry/instrumentation-* dependencies, with more being periodically added. Will @sentry/node just keep adding more and more dependencies to keep up?

[lforst]

Your concern is very valid. Up until now we just kept adding deps for the simple reason that we want to make it dead simple to set up Sentry. I am starting to think that this could go south long term when thinking about, maintenance, reliability, and security.

I don't have a specific answer for you but I will take this to the team and discuss.

[nwalters512]

Thanks! FWIW, in the past couple of weeks, two more instrumentation packages have made it onto the list. Here's the current collection on develop:

    "@opentelemetry/instrumentation-amqplib": "^0.42.0",
    "@opentelemetry/instrumentation-connect": "0.40.0",
    "@opentelemetry/instrumentation-dataloader": "0.12.0",
    "@opentelemetry/instrumentation-express": "0.44.0",
    "@opentelemetry/instrumentation-fastify": "0.40.0",
    "@opentelemetry/instrumentation-fs": "0.16.0",
    "@opentelemetry/instrumentation-generic-pool": "0.39.0",
    "@opentelemetry/instrumentation-graphql": "0.43.0",
    "@opentelemetry/instrumentation-hapi": "0.41.0",
    "@opentelemetry/instrumentation-http": "0.53.0",
    "@opentelemetry/instrumentation-ioredis": "0.43.0",
    "@opentelemetry/instrumentation-kafkajs": "0.4.0",
    "@opentelemetry/instrumentation-koa": "0.43.0",
    "@opentelemetry/instrumentation-lru-memoizer": "0.40.0",
    "@opentelemetry/instrumentation-mongodb": "0.47.0",
    "@opentelemetry/instrumentation-mongoose": "0.42.0",
    "@opentelemetry/instrumentation-mysql": "0.41.0",
    "@opentelemetry/instrumentation-mysql2": "0.41.0",
    "@opentelemetry/instrumentation-nestjs-core": "0.40.0",
    "@opentelemetry/instrumentation-pg": "0.44.0",
    "@opentelemetry/instrumentation-redis-4": "0.42.0",
    "@opentelemetry/instrumentation-undici": "0.6.0",

[lforst]

It's on our radar. Fwiw this will not be changed all too quickly. We need to figure out a way that aligns with how we want setup to work and generally find a story around how a less heavy node SDK can look.

[klippx]

This fact is the root cause for pain and issues in our mono repo as well.

$ yarn why @opentelemetry/instrumentation-http
├─ @sentry/node@npm:8.51.0
│  └─ @opentelemetry/instrumentation-http@npm:0.56.0 [8db7d] (via npm:0.56.0 [8db7d])
│
└─ @vcc/shared@workspace:shared
   └─ @opentelemetry/instrumentation-http@npm:0.57.1 [766b1] (via npm:0.57.1 [766b1])

Before 9.x this used to be kept in sync quite well by Sentry maintainers. But now that 9.x alpha is being worked on, the 8.x branch is no longer kept up to date with these dependencies. We have our own explicit dependency to @opentelemetry/instrumentation-http since we use it in our instrumentation, and it is being kept on latest version at all times using Renovate.

Combining all these facts amounts to a version diff which is causing problems with duplicate metrics in our OTEL exporters, and this issue can only be fixed if we manually downgrade our @opentelemetry/instrumentation-http dependency from 0.57.1 to 0.56.0 which is super inconvenient! Not to mention it took as a few days to figure out what was going on! Using the debug log exporter in the otel sidecar helped us crack the case.

PLEASE make these peer dependencies or drop them, this is not working. It needs to be fixed in 9.x this is not sustainable. I'd rather drop @sentry/node entirely than to be forced to manually keep versions in sync.

[mydea]

Hey, I understand how this is painful. I think for the time being, the best solution for this is to override the instrumentation you care about (e.g. @opentelemetry/instrumentation-http) to force the version you want to be used by Sentry as well. This should be safe to do mostly. E.g. if you use npm:

"overrides": {
  "@opentelemetry/instrumentation-http": "0.57.1"
}

This should fix this for now.

I understand that this is not a great solution. Really, one of the core problems here is that all of these instrumentations are 0.x versions, which makes deduping them across minors impossible. I hope that this will improve in the near future when v2 of OpenTelemetry JS is released.

We may also possibly look into a node-core SDK in the future that includes as little dependencies as possible.

[mydea]

See: #15213

[klippx]

Thanks for the tips, yea the 0.x versions are painful for us all. There are many separate things stewing here which all play a part (opentelemetry versioning together with sentry node pinned versioning + the fact that 8.x track is not getting regular updates any more)

I created a new issue and intended to groom my comment here a bit to remove duplication, but since you were so fast (and customer focused) I will leave it as it is. I did not try to post in several threads as an annoying person or an overly demanding customer would do, just that I thought I found a very important issue that I wanted to share as it cost us a lot of time to debug it was worth sharing.

Thanks again!