Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Logs+] Add default ECS mappings to logs-*-* #95538

Closed
felixbarny opened this issue Apr 25, 2023 · 1 comment · Fixed by #96171
Closed

[Logs+] Add default ECS mappings to logs-*-* #95538

felixbarny opened this issue Apr 25, 2023 · 1 comment · Fixed by #96171
Assignees
Labels
:Data Management/Data streams Data streams and their lifecycles Team:Data Management Meta label for data/management team

Comments

@felixbarny
Copy link
Member

felixbarny commented Apr 25, 2023

This is required in order to be able to route sink-type integrations to logs-<service.name>-<namespace>, as this will match the logs-*-* index template instead of the one from the integration.

We should be able to re-use this mapping: https://github.com/elastic/elastic-package/blob/main/internal/builder/_static/ecs_mappings.yaml and convert it to a component template that is imported by the logs-*-* index template. With #95558, we can simplify the dynamic template.

An open question is how to keep the upstream ECS mappings, the elastic-package ECS mappings and the ECS mappings in Elasticsearch in sync.

Considerations

  • Should we only include field definitions from ECS core? Some logging fields wouldn't fall into this, however.
  • Split core and extended into different component templates?
  • Exclude fields with the nested field type from the component templates so that they'll work with subobjects: false?
  • Avoid mapping field types that are the default types, such as keyword for string fields
  • As much as possible, map on common naming patterns, such as name, message, *_ip, ip, etc.
  • Minimize the possibility of breaking changes where a new version of the mapping would change the field type

Related:

@felixbarny felixbarny added the :Data Management/Data streams Data streams and their lifecycles label Apr 25, 2023
@elasticsearchmachine elasticsearchmachine added the Team:Data Management Meta label for data/management team label Apr 25, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Data streams Data streams and their lifecycles Team:Data Management Meta label for data/management team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants