Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/filelog] containerd operator fails on windows hosts #35558

Closed
squ1d123 opened this issue Oct 2, 2024 · 4 comments
Closed

[receiver/filelog] containerd operator fails on windows hosts #35558

squ1d123 opened this issue Oct 2, 2024 · 4 comments
Labels

Comments

@squ1d123
Copy link
Contributor

squ1d123 commented Oct 2, 2024

Component(s)

receiver/filelog

What happened?

Description

The containerd operator whithin the filelog reciever fails to generate attributes on Windows hosts.

This looks to be because the regex parsing does not cater for windows file path seperator.

Steps to Reproduce

  1. Running on AWS EKS cluster with a windows node
  2. Deploy OpenTelemetryCollector CRD with below config
apiversion: opentelemetry.io/v1beta1
kind: opentelemetrycollector
metadata:
  name: otel-windows
  namespace: olly-system
spec:
  config:
    exporters:
      debug:
        verbosity: detailed
    processors:
      batch/logs:
        send_batch_max_size: 5000
        send_batch_size: 2500
        timeout: 10s
    receivers:
      filelog:
        include:
        - c:\var\log\pods\*\*\*.log
        include_file_name: false
        include_file_path: true
        include_file_record_number: true
        operators:
        - add_metadata_from_filepath: true
          format: containerd
          type: container
        retry_on_failure:
          enabled: true
          initial_interval: 1s
          max_elapsed_time: 10m
          max_interval: 30s
        start_at: beginning
        storage: file_storage
    service:
      pipelines:
        logs:
          exporters:
          - debug
          processors:
          - batch/logs
          receivers:
          - filelog
      telemetry:
        logs:
          level: debug
  nodeselector:
    kubernetes.io/os: windows

Expected Result

Collector to parse the log and add attributes.

  • k8s.pod.name
  • k8s.pod.uid
  • k8s.container.name
  • k8s.container.restart_count
  • k8s.namespace.name

Actual Result

Receive error

2024-10-01T14:57:41.620+0100    error   reader/reader.go:140    process: %w     {"kind": "receiver", "name": "filelog", "data_type": "logs", "component": "fileconsumer", "path": "C:\\var\\log\\pods\\my-namespace_some-pod-lksd23el3_041f33f9-34ac-4b6a-8df9-68dc27f4bd27\\main\\0.log", "error": "failed to handle attribute mappings: failed to detect a valid log path"}

Collector version

v0.110.0

Environment information

Environment

OS: Windows EKS Node

OpenTelemetry Collector configuration

exporters:
  debug:
    verbosity: detailed
processors:
  batch/logs:
    send_batch_max_size: 5000
    send_batch_size: 2500
    timeout: 10s
receivers:
  filelog:
    include:
    - c:\var\log\pods\*\*\*.log
    include_file_name: false
    include_file_path: true
    include_file_record_number: true
    operators:
    - add_metadata_from_filepath: true
      format: containerd
      type: container
    retry_on_failure:
      enabled: true
      initial_interval: 1s
      max_elapsed_time: 10m
      max_interval: 30s
    start_at: beginning
    storage: file_storage
service:
  pipelines:
    logs:
      exporters:
      - debug
      processors:
      - batch/logs
      receivers:
      - filelog
  telemetry:
    logs:
      level: debug

Log output

2024-10-01T14:57:41.620+0100    error   reader/reader.go:140    process: %w     {"kind": "receiver", "name": "filelog", "data_type": "logs", "component": "fileconsumer", "path": "C:\\var\\log\\pods\\my-namespace_some-pod-lksd23el3_041f33f9-34ac-4b6a-8df9-68dc27f4bd27\\main\\0.log", "error": "failed to handle attribute mappings: failed to detect a valid log path"}

Additional context

No response

@squ1d123 squ1d123 added bug Something isn't working needs triage New item requiring triage labels Oct 2, 2024
Copy link
Contributor

github-actions bot commented Oct 2, 2024

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@crobert-1
Copy link
Member

crobert-1 commented Oct 4, 2024

A code owner has approved the PR that resolves this issue, removing needs triage.

@crobert-1 crobert-1 removed the needs triage New item requiring triage label Oct 4, 2024
djaglowski pushed a commit that referenced this issue Oct 10, 2024
…35559)

seperator

**Description:** <Describe what has changed.>
This PR updates add support to parse container filepaths on windows
hosts

**Link to tracking Issue:** 
#35558

**Testing:** 
Additional unit test to test regex parsing for windows filepaths

**Documentation:**
N/A
Copy link
Contributor

github-actions bot commented Dec 4, 2024

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Dec 4, 2024
@djaglowski
Copy link
Member

Resolved by #35559

sbylica-splunk pushed a commit to sbylica-splunk/opentelemetry-collector-contrib that referenced this issue Dec 17, 2024
…pen-telemetry#35559)

seperator

**Description:** <Describe what has changed.>
This PR updates add support to parse container filepaths on windows
hosts

**Link to tracking Issue:** 
open-telemetry#35558

**Testing:** 
Additional unit test to test regex parsing for windows filepaths

**Documentation:**
N/A
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants