Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Only watch metadata for ReplicaSets in K8s #41100

Merged
merged 5 commits into from
Oct 14, 2024
Merged

Only watch metadata for ReplicaSets in K8s #41100

merged 5 commits into from
Oct 14, 2024

Conversation

swiatekm
Copy link
Contributor

@swiatekm swiatekm commented Oct 3, 2024 •
edited
Loading

Proposed commit message

Use metadata watchers for ReplicaSets in components that need them. The only data we need from ReplicaSets are their name and OwnerReferences, which are used to connect Pods to Deployments and DaemonSets.

This PR makes the change in:

  • autodiscovery
  • the add_kubernetes_metadata processor

For more details see elastic/elastic-agent#5623.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

You need a full K8s cluster, unfortunately. I've tested it using the default elastic-agent standalone manifest, by building a custom container image and loading it into a local kind cluster.

Related issues

@swiatekm swiatekm added enhancement backport-8.15 Automated backport to the 8.15 branch with mergify backport-8.x Automated backport to the 8.x branch with mergify labels Oct 3, 2024
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Oct 3, 2024
@swiatekm swiatekm added the Team:Elastic-Agent Label for the Agent team label Oct 3, 2024
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Oct 3, 2024
This was referenced Oct 3, 2024
Closed
Merged
@swiatekm swiatekm force-pushed the feat/k8s-metadata-replicaset-partial branch 2 times, most recently from a789817 to 99a610d Compare October 4, 2024 11:53
@swiatekm swiatekm marked this pull request as ready for review October 4, 2024 12:47
@swiatekm swiatekm requested review from a team as code owners October 4, 2024 12:47
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

@swiatekm swiatekm requested review from belimawr and rdner October 4, 2024 12:47
@swiatekm
Copy link
Contributor Author

swiatekm commented Oct 4, 2024

Not sure what the lint failure is about, or why it only happens on Windows.

belimawr
belimawr reviewed Oct 4, 2024
View reviewed changes
@swiatekm swiatekm force-pushed the feat/k8s-metadata-replicaset-partial branch from 99a610d to ed19c44 Compare October 4, 2024 14:06
@swiatekm swiatekm requested a review from belimawr October 4, 2024 15:06
rdner
rdner reviewed Oct 7, 2024
View reviewed changes
Copy link
Member

@rdner rdner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changelog has some incorrect extra entries.

We should also address all the linter issues reported in this PR.

According to our team policy, we fix linter issues in the files we touch.

@swiatekm swiatekm force-pushed the feat/k8s-metadata-replicaset-partial branch from ed19c44 to b78ec48 Compare October 7, 2024 16:39
@mergify Mergify
Copy link
Contributor

mergify bot commented Oct 8, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b feat/k8s-metadata-replicaset-partial upstream/feat/k8s-metadata-replicaset-partial
git merge upstream/main
git push upstream feat/k8s-metadata-replicaset-partial

This was referenced Oct 8, 2024
Merged
Merged
@swiatekm
Copy link
Contributor Author

swiatekm commented Oct 8, 2024

I've dropped the metricbeat changes from this PR for now. There's some kind of subtle problem with them that I'm still debugging.

@swiatekm swiatekm requested a review from rdner October 8, 2024 08:48
@swiatekm swiatekm force-pushed the feat/k8s-metadata-replicaset-partial branch from cd8a9cf to fd27cca Compare October 8, 2024 10:49
@mergify Mergify
Copy link
Contributor

mergify bot commented Oct 10, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b feat/k8s-metadata-replicaset-partial upstream/feat/k8s-metadata-replicaset-partial
git merge upstream/main
git push upstream feat/k8s-metadata-replicaset-partial

@swiatekm swiatekm force-pushed the feat/k8s-metadata-replicaset-partial branch from fd27cca to 301fe55 Compare October 10, 2024 14:46
pkoutsovasilis
pkoutsovasilis approved these changes Oct 11, 2024
View reviewed changes
Copy link
Contributor

@pkoutsovasilis pkoutsovasilis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🙂

@swiatekm swiatekm force-pushed the feat/k8s-metadata-replicaset-partial branch from 301fe55 to c313f0c Compare October 14, 2024 09:11
@jlind23
Copy link
Collaborator

jlind23 commented Oct 14, 2024 •
edited
Loading

@gizas @tetianakravchenko @MichaelKatsoulis @constanca-m Can we get your eyes on this as codeowner?

@swiatekm swiatekm enabled auto-merge (squash) October 14, 2024 10:02
@swiatekm swiatekm merged commit ee780d2 into main Oct 14, 2024
142 checks passed
@swiatekm swiatekm deleted the feat/k8s-metadata-replicaset-partial branch October 14, 2024 10:50
mergify bot pushed a commit that referenced this pull request Oct 14, 2024
@swiatekm @mergify
Only watch metadata for ReplicaSets in K8s (#41100)
163a929 
* Bump github.com/elastic/elastic-agent-autodiscover to v0.9.0

* Only watch metadata for ReplicaSets in k8s autodiscovery

* Only watch metadata for ReplicaSets in add_kubernetes_metadata processor

* Fix linter warnings

* Merge changelog entries

(cherry picked from commit ee780d2)

# Conflicts:
#	NOTICE.txt
#	go.mod
#	go.sum
#	libbeat/autodiscover/providers/kubernetes/pod.go
#	libbeat/processors/add_kubernetes_metadata/kubernetes.go
@mergify mergify bot mentioned this pull request Oct 14, 2024
Merged
4 tasks
mergify bot pushed a commit that referenced this pull request Oct 14, 2024
@swiatekm @mergify
Only watch metadata for ReplicaSets in K8s (#41100)
d75902b 
* Bump github.com/elastic/elastic-agent-autodiscover to v0.9.0

* Only watch metadata for ReplicaSets in k8s autodiscovery

* Only watch metadata for ReplicaSets in add_kubernetes_metadata processor

* Fix linter warnings

* Merge changelog entries

(cherry picked from commit ee780d2)
@mergify mergify bot mentioned this pull request Oct 14, 2024
Merged
4 tasks
swiatekm added a commit that referenced this pull request Oct 14, 2024
@swiatekm
Only watch metadata for ReplicaSets in K8s (#41100)
1dcf4fb 
* Bump github.com/elastic/elastic-agent-autodiscover to v0.9.0

* Only watch metadata for ReplicaSets in k8s autodiscovery

* Only watch metadata for ReplicaSets in add_kubernetes_metadata processor

* Fix linter warnings

* Merge changelog entries

(cherry picked from commit ee780d2)

# Conflicts:
#	NOTICE.txt
#	go.mod
#	go.sum
#	libbeat/autodiscover/providers/kubernetes/pod.go
#	libbeat/processors/add_kubernetes_metadata/kubernetes.go
swiatekm added a commit that referenced this pull request Oct 14, 2024
@mergify @swiatekm
Only watch metadata for ReplicaSets in K8s (#41100) (#41215)
8048cb4 
* Bump github.com/elastic/elastic-agent-autodiscover to v0.9.0

* Only watch metadata for ReplicaSets in k8s autodiscovery

* Only watch metadata for ReplicaSets in add_kubernetes_metadata processor

* Fix linter warnings

* Merge changelog entries

(cherry picked from commit ee780d2)

Co-authored-by: Mikołaj Świątek <mail@mikolajswiatek.com>
swiatekm added a commit that referenced this pull request Oct 14, 2024
@mergify @swiatekm
Only watch metadata for ReplicaSets in K8s (#41100) (#41214)
2861b61 
* Bump github.com/elastic/elastic-agent-autodiscover to v0.9.0

* Only watch metadata for ReplicaSets in k8s autodiscovery

* Only watch metadata for ReplicaSets in add_kubernetes_metadata processor

* Fix linter warnings

* Merge changelog entries

(cherry picked from commit ee780d2)

# Conflicts:
#	NOTICE.txt
#	go.mod
#	go.sum
#	libbeat/autodiscover/providers/kubernetes/pod.go
#	libbeat/processors/add_kubernetes_metadata/kubernetes.go

Co-authored-by: Mikołaj Świątek <mail@mikolajswiatek.com>
@swiatekm swiatekm mentioned this pull request Oct 17, 2024
Merged
4 tasks
This was referenced Oct 18, 2024
Merged
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@belimawr belimawr belimawr approved these changes

@pkoutsovasilis pkoutsovasilis pkoutsovasilis approved these changes

@MichaelKatsoulis MichaelKatsoulis MichaelKatsoulis approved these changes

@jlind23 jlind23 jlind23 approved these changes

@constanca-m constanca-m Awaiting requested review from constanca-m constanca-m was automatically assigned from elastic/obs-cloudnative-monitoring

@rdner rdner Awaiting requested review from rdner

Assignees

@swiatekm swiatekm

Labels
backport-8.x Automated backport to the 8.x branch with mergify backport-8.15 Automated backport to the 8.15 branch with mergify enhancement Team:Elastic-Agent Label for the Agent team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@swiatekm @elasticmachine @jlind23 @belimawr @rdner @pkoutsovasilis @MichaelKatsoulis