feat(secretstores): Add Docker Secrets

[shantanoo-desai]

This feature adds the secret-store for Docker Secrets to Telegraf. This PR is a resultant effort based on discussion for a feature-request in #12848.

In a nutshell, Docker can inject secrets to a container during boot into files under /run/secrets/<secret_file_x> depending on the configuration mentioned within a docker-compose.yml file. This plugin uses these _injected_f files to read credentials and passes them to the respective resolver strings wherever they are mentioned in plugins.

Changes

Following files are introduced with this PR, which complies with other secret-store plugins file structure:

• plugins/secretstores/all/docker.go
• plugins/secretstores/docker/docker.go
• plugins/secretstores/docker/docker_test.go
• plugins/secretstores/docker/README.md
• plugins/secretstores/docker/sample.conf

Required for all PRs

• Updated associated README.md.
• Wrote appropriate unit tests.
• Pull request title or commits are in conventional commit format

resolves #12848

[telegraf-tiger]

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

[shantanoo-desai]

Tests

One of the major concern with introducing tests for List and Get as well as ResolverFunc is the fact that Docker defaults to injecting secrets under /run directory. Assuming the tests can be written similar to jose plugin, creating temporary directories and files within them will fail because the Get method relies on the hard-coded path to /run/secrets. Another caveat is /run directory required privilege escalation which may not be desirable for CI tests.

I would like some guidance on how to proceed with testing this plugin which is inherently meant to be run in a docker container. Is there some provision to achieve the testing for /run directory?

cc @srebhan , @Hipska

[shantanoo-desai]

!signed-cla

[Hipska]

Hi, there is the testutil.Container, not sure if it can be used for this purpose as usually that's for running other apps.

[shantanoo-desai]

I think test-containers won't work directly here because I realize now that secrets injection from Docker is actually via docker compose so I might have to perform a test with a docker-compose.yml file.

[srebhan]

Even though this is still marked as a draft, this PR looks quite solid. I added some comments and would love to see a basic test with a dummy directory to get some more test coverage.

[srebhan]

Thanks @shantanoo-desai for the awesome plugin. A few comments from my side...

[shantanoo-desai]

Change Summary

• Use testdata directory for testing
• Refactor docker_test.go to adapt request changes (better error capturing)
• Add more information in README.md
• add dynamic option in sample.conf and adapt the resolver to pick this value as opposed to hard-coded boolean
• add a Error check for permissions for d.Path with os.IsPermission(err) in addition to os.IsNotExist(err) in initialization of plugin

[shantanoo-desai]

Local Test

Perform tests for plugin with a local docker container using the following files:

Dockerfile

FROM ubuntu:latest

WORKDIR /app

COPY https://<CIRCLECI_URL>_deb /app/telegraf_amd64.deb
COPY entrypoint.sh /app/entrypoint.sh
RUN  chmod +x /app/entrypoint.sh

RUN dpkg -i /app/telegraf_amd64.deb

ENTRYPOINT ["/app/entrypoint.sh"]
CMD ["telegraf"]

docker-compose.yml

services:
  telegraf:
    image: test-telegraf:latest
    container_name: telegraf-opcua-docker
    user: "1000"
    network_mode: host
    env_file:
      - .env
    volumes:
      - ./telegraf.conf:/etc/telegraf/telegraf.conf:ro
    secrets:
      - opcua_password
secrets:
  opcua_password:
    environment: OPCUA_PASSWORD

telegraf.conf

[agent]
    interval = "20s"
    round_interval = true
    metric_batch_size = 1000
    metric_buffer_limit = 10000
    collection_jitter = "0s"
    flush_interval = "10s"
    flush_jitter = "0s"
    precision = ""
    debug = true
    quiet = false
    hostname = ""
    omit_hostname = true

[[secretstores.docker]]
  id = "dockersecret_test"

[[outputs.file]]
  data_format = "json"

[[inputs.opcua]]
  ## Metric name
  name = "secrettest"
  endpoint = "opc.tcp://192.168.4.201:4840"
  connect_timeout = "60s"


  request_timeout = "60s"

  security_policy = "None"
  security_mode = "None"
  username = "OpcUser"

  password = "@{dockersecret_test:opcua_password}"
  timestamp = "source"

  nodes = [ {name="test", namespace="2", identifier_type="s", identifier="IX_SSC_LightBarrierOutsource_I4"}  ]

  data_format = "json"

entrypoint.sh

#!/bin/bash
set -e

if [ "${1:0:1}" = '-' ]; then
    set -- telegraf "$@"
fi

if [ $EUID -ne 0 ]; then
    exec "$@"
else
    # Allow telegraf to send ICMP packets and bind to privliged ports
    setcap cap_net_raw,cap_net_bind_service+ep /usr/bin/telegraf || echo "Failed to set additional capabilities on /usr/bin/telegraf"

    exec setpriv --reuid telegraf --init-groups "$@"
fi

[srebhan]

@shantanoo-desai thanks for the update, we are almost there... Just have one comment regarding the error handling in Init.

[telegraf-tiger]

Download PR build artifacts for linux_amd64.tar.gz, darwin_amd64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

☺️ This pull request doesn't significantly change the Telegraf binary size (less than 1%)

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB	RPM	TAR GZ	ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_arm64.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz

[srebhan]

Perfect. Thanks a lot @shantanoo-desai for the nice contribution!

[powersj]

Thank you for taking the time to both propose this and work through the process of creating a PR! This will go out in v1.27.0 in June.