-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FileBeat] GCP module enhancement - Populate orchestrator.* fields for K8S logs #25368
Conversation
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
Trends 🧪💚 Flaky test reportTests succeeded. Expand to view the summary
Test stats 🧪
|
A discussion is opened on discuss.elastic.co regarding this PR: https://discuss.elastic.co/t/gcp-module-add-cluster-name-for-gke-k8s-io-logs/271278 |
This should also be done for the other modules that can get K8S logs. |
You also need to add sample K8S logs to the
|
Add sample logs
I tried to do that but got an error |
Agreed but I do not have access to K8S systems from other providers (Azure Kubernetes Service, Amazon EKS, on premise K8S...) to do that. |
Thats a good question. for the development team. I've never tried to use fields in a future ECS spec version. |
Pinging @elastic/integrations (Team:Integrations) |
Pinging @elastic/security-external-integrations (Team:Security-External Integrations) |
This pull request is now in conflicts. Could you fix it? 🙏
|
cc @kaiyan-sheng, @jsoriano, @exekias. Looks like we need to align with the Inventory schema work here. |
@sorantis Thanks for pinging here! |
This change will also need applied to elastic/integrations. |
Thanks for your contribution @TonioRyo! Regarding the ECS version how about trying to update it at
@kaiyan-sheng @andrewkroh this change looks good to me regarding the |
This pull request is now in conflicts. Could you fix it? 🙏
|
@TonioRyo Would you be able to merge with master because I added the ECS fields to the beats definition today. If you could also bump the ECS version to 1.10.0 that would be great! |
Thanks @P1llus, I was able to generate the expected results based on sample logs thanks to your work! |
Removing integration sync as I added the same changes to packages here: elastic/integrations#1045 |
Awesome @TonioRyo . Il check to see if someone can merge it then! |
jenkins run tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks!
…r K8S logs (elastic#25368) (cherry picked from commit 2d04bf7)
…te orchestrator.* fields for K8S logs (#26111)
What does this PR do?
This PR add the ability to populate orchestrator.* fields for GCP K8S logs. The
cluster_name
field from the original message was not kept in the logs, this field is important to analyze security events. I also tried to populate other orchestrator.* fields with information available in other fields of the original message.Why is it important?
The
cluster_name
field from the original message was not kept in the logs, this field is important to analyze security events.Checklist