Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OTE-1506] Loadbalancer exporter: Add resource_keys routing for the traces #35970

Conversation

foadnh
Copy link

@foadnh foadnh commented Oct 23, 2024

Ticket

Context

Resource-based load balancing is required for Canva reliability metrics.
Currently, we maintain a copy of the load balancer in the otel-platform repo. This copy is based on the old version (v0.79.0) and never updated with the upstream.
There are improvements and new features in the upstream version that potentially can resolve our public gateway migration issues. See this thread.

In this PR, we add resource_keys based load balancing to the opentelemetry-collector-contrib fork.

Notes

  • Why didn't we simply copy the implementation from the old copy? In this PR, we used a different approach to add resource_keys based load balancing. Because it is simpler and more aligned with the rest of the load balancer, it enables us to keep our fork in sync with upstream easily. Also, it allows us to contribute changes upstream and eliminate this fork.
  • Does upstream accept this new feature? Yes, there is an open request for a limited version of this feature. And the last time we tried to contribute this feature to the upstream, the plugin owner liked the feature but needed a more robust implementation.
  • Is this branch ready for a PR to upstream? No, not yet
    • We will follow the Otel collector contribution guideline. This means we will start by raising an issue and describing the feature requirements. Once we get LGTM on that one, then we will raise a PR.
    • To minimise changes on our side, we used our old config namings (routing_key: resource) in this PR, which conflicts with the new versions' features.
    • To keep this PR small, we only added a resource_keys based load balancer to traces. We must add it to metrics and logs before raising it to the upstream.
    • This is based on v0.102.0 and not the upstream master (Preparing for v0.112.0). We need to rebase with the upstream master to prevent conflicts.
    • We skipped README.md and CHANGELOG.md updates.

@foadnh foadnh requested review from jpkrohling and a team as code owners October 23, 2024 21:22
@foadnh foadnh closed this Oct 23, 2024
@foadnh foadnh deleted the foadnh-loadbalancer-resource-keys-routing branch October 24, 2024 02:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants