Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fleet] Support dataset with multiple level like system.process.summary #99852

Merged
merged 3 commits into from
May 12, 2021

Conversation

nchaulet
Copy link
Member

@nchaulet nchaulet commented May 11, 2021

Summary

Resolve #92636

Currently when a datastream use a dataset with multiple dots we are not using the correct agent template.
We where looking to find the dastream by splitting the dataset name on . instead of using the datastream path property
That PR fix that.

How to test

You can try that PR elastic/integrations#778 run the package registry and you will get the correct
agent template for system.process.summary

Screen Shot 2021-05-11 at 6 01 20 PM

@nchaulet nchaulet added bug Fixes for quality problems that affect the customer experience v8.0.0 release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team v7.14.0 labels May 11, 2021
@nchaulet nchaulet self-assigned this May 11, 2021
@nchaulet nchaulet requested a review from a team as a code owner May 11, 2021 21:44
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@nchaulet nchaulet requested a review from jen-huang May 11, 2021 22:02
@nchaulet
Copy link
Member Author

Should we backport this to 7.13?

Copy link
Contributor

@jen-huang jen-huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Yes, this is a good 7.13 candidate.
  2. Would this be fixed if Attempt to fix process summary name integrations#778 also exported path: process_summary? (It's still good to have a fallback in Kibana, aka this PR)

@nchaulet
Copy link
Member Author

nchaulet commented May 11, 2021

Would this be fixed if elastic/integrations#778 also exported path: process_summary? (It's still good to have a fallback in Kibana, aka this PR)

Actually the package registry populate the path elastic/package-registry#176 so I think we probably should only rely on that.

Just did a quick check against the snapshot registry and the dataset we computed before and the path match for all packages expect apm endpoint and synthetics where there is no agent data stream template.

✅ /package/apache/0.5.0
app_metrics app
error_logs error
internal_metrics internal
profile_metrics profiling
sampled_traces
traces
❌ /package/apm/0.2.0
✅ /package/auditd/0.1.1
✅ /package/aws/0.5.4
✅ /package/azure/0.2.3
✅ /package/barracuda/0.2.4
✅ /package/bluecoat/0.1.4
✅ /package/cef/0.1.0
✅ /package/checkpoint/0.5.2
✅ /package/cisco/0.8.1
✅ /package/crowdstrike/0.4.1
✅ /package/cyberark/0.1.4
✅ /package/cylance/0.1.4
✅ /package/docker/0.1.1
✅ /package/elastic_agent/0.0.7
collection diagnostic_collection
file events_file
library events_library
network events_network
process events_process
registry events_registry
security events_security
❌ /package/endpoint/0.19.1
✅ /package/f5/0.2.4
✅ /package/fleet_server/0.2.3
✅ /package/fortinet/0.8.1
✅ /package/gcp/0.0.2
✅ /package/google_workspace/0.2.4
✅ /package/haproxy/0.2.8
✅ /package/iis/0.3.2
✅ /package/imperva/0.1.4
✅ /package/infoblox/0.1.4
✅ /package/iptables/0.1.0
✅ /package/juniper/0.5.1
✅ /package/kafka/0.3.7
✅ /package/kubernetes/0.5.1
✅ /package/linux/0.3.10
✅ /package/log/0.4.6
✅ /package/microsoft/0.4.3
✅ /package/mongodb/0.2.9
✅ /package/mysql/0.3.7
✅ /package/nats/0.1.4
✅ /package/netflow/0.3.8
✅ /package/netscout/0.1.4
✅ /package/nginx/0.5.0
✅ /package/nginx_ingress_controller/0.0.3
✅ /package/o365/0.3.2
✅ /package/okta/0.5.0
✅ /package/osquery/0.1.2
✅ /package/osquery_manager/0.2.2
✅ /package/panw/0.7.1
✅ /package/postgresql/0.3.1
✅ /package/prometheus/0.3.4
✅ /package/proofpoint/0.1.4
✅ /package/rabbitmq/0.2.7
✅ /package/radware/0.1.4
✅ /package/redis/0.3.7
✅ /package/santa/0.0.3
✅ /package/security_detection_engine/0.0.2
✅ /package/snapshot/0.0.1
✅ /package/sonicwall/0.1.5
✅ /package/sophos/0.2.1
✅ /package/squid/0.1.4
✅ /package/staging/0.0.1
✅ /package/stan/0.1.3
✅ /package/suricata/0.6.0
✅ /package/symantec/0.1.2
http
icmp
tcp
❌ /package/synthetics/0.0.4
✅ /package/system/0.12.3
✅ /package/tomcat/0.1.4
✅ /package/traefik/0.1.1
✅ /package/windows/0.7.0
✅ /package/winlog/0.2.1
✅ /package/zeek/0.7.0
✅ /package/zookeeper/0.2.6
✅ /package/zoom/0.3.1
✅ /package/zscaler/0.1.4

@nchaulet nchaulet added the auto-backport Deprecated - use backport:version if exact versions are needed label May 11, 2021
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Unknown metric groups

References to deprecated APIs

id before after diff
infra 5 3 -2
monitoring 109 56 -53
total -55

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @nchaulet

@nchaulet
Copy link
Member Author

/run-fleet-e2e-tests

@kibanamachine
Copy link
Contributor

💚 Backport successful

Status Branch Result
7.13
7.x

The backport PRs will be merged automatically after passing CI.

kibanamachine added a commit that referenced this pull request May 12, 2021
…ry (#99852) (#99874)

Co-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>
kibanamachine added a commit that referenced this pull request May 12, 2021
…ry (#99852) (#99875)

Co-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Deprecated - use backport:version if exact versions are needed bug Fixes for quality problems that affect the customer experience release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team v7.13.0 v7.14.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Metrics UI] Summary of process information is different between Metricbeat and Agent.
4 participants