Skip to content

Commit

Permalink
Agent sidecar injection support via Admission Controller (#1348)
Browse files Browse the repository at this point in the history
* Add basic baseline tests for agent chart

* baselines for AC sidecar injection

* Agent sidecar injection support via Admission Controller

* Update charts/datadog/values.yaml

Co-authored-by: Celene <celene@datadoghq.com>

* Update charts/datadog/values.yaml

Co-authored-by: Celene <celene@datadoghq.com>

* Update charts/datadog/values.yaml

Co-authored-by: Celene <celene@datadoghq.com>

* Update charts/datadog/values.yaml

Co-authored-by: Celene <celene@datadoghq.com>

* Update charts/datadog/values.yaml

Co-authored-by: Celene <celene@datadoghq.com>

* Update charts/datadog/values.yaml

Co-authored-by: Celene <celene@datadoghq.com>

* Update charts/datadog/values.yaml

Co-authored-by: Celene <celene@datadoghq.com>

* Update charts/datadog/values.yaml

Co-authored-by: Celene <celene@datadoghq.com>

* Update charts/datadog/README.md

Co-authored-by: Celene <celene@datadoghq.com>

* PR review updates

* Update charts/datadog/values.yaml

Co-authored-by: Celene <celene@datadoghq.com>

* Update charts/datadog/values.yaml

Co-authored-by: Celene <celene@datadoghq.com>

* PR review updates 2

* Update charts/datadog/values.yaml

Co-authored-by: Celene <celene@datadoghq.com>

* doc generate

* Set chart version to 3.59.1

---------

Co-authored-by: Celene <celene@datadoghq.com>
  • Loading branch information
levan-m and celenechang authored Mar 21, 2024
1 parent b11447b commit 433211e
Show file tree
Hide file tree
Showing 19 changed files with 3,604 additions and 5 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/go-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ name: Go Test
on:
push:
paths:
- 'test/**'
- 'test/datadog-operator/**'
- 'charts/datadog-operator/**'
pull_request:
paths:
- 'test/**'
- 'test/datadog-operator/**'
- 'charts/datadog-operator/**'
env:
GO111MODULE: "on"
Expand Down
4 changes: 4 additions & 0 deletions charts/datadog/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Datadog changelog

## 3.59.1

* Add support for configuring Agent sidecar injection using Admission Controller.

## 3.59.0

* Set default `Agent` and `Cluster-Agent` version to `7.52.0`.
Expand Down
2 changes: 1 addition & 1 deletion charts/datadog/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
apiVersion: v1
name: datadog
version: 3.59.0
version: 3.59.1
appVersion: "7"
description: Datadog Agent
keywords:
Expand Down
10 changes: 9 additions & 1 deletion charts/datadog/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Datadog

![Version: 3.59.0](https://img.shields.io/badge/Version-3.59.0-informational?style=flat-square) ![AppVersion: 7](https://img.shields.io/badge/AppVersion-7-informational?style=flat-square)
![Version: 3.59.1](https://img.shields.io/badge/Version-3.59.1-informational?style=flat-square) ![AppVersion: 7](https://img.shields.io/badge/AppVersion-7-informational?style=flat-square)

[Datadog](https://www.datadoghq.com/) is a hosted infrastructure monitoring platform. This chart adds the Datadog Agent to all nodes in your cluster via a DaemonSet. It also optionally depends on the [kube-state-metrics chart](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-state-metrics). For more information about monitoring Kubernetes with Datadog, please refer to the [Datadog documentation website](https://docs.datadoghq.com/agent/basic_agent_usage/kubernetes/).

Expand Down Expand Up @@ -544,6 +544,14 @@ helm install <RELEASE_NAME> \
| agents.volumeMounts | list | `[]` | Specify additional volumes to mount in all containers of the agent pod |
| agents.volumes | list | `[]` | Specify additional volumes to mount in the dd-agent container |
| clusterAgent.additionalLabels | object | `{}` | Adds labels to the Cluster Agent deployment and pods |
| clusterAgent.admissionController.agentSidecarInjection.clusterAgentCommunicationEnabled | bool | `true` | Enable communication between Agent sidecars and the Cluster Agent. |
| clusterAgent.admissionController.agentSidecarInjection.containerRegistry | string | `nil` | |
| clusterAgent.admissionController.agentSidecarInjection.enabled | bool | `false` | Enables Datadog Agent sidecar injection. |
| clusterAgent.admissionController.agentSidecarInjection.imageName | string | `nil` | |
| clusterAgent.admissionController.agentSidecarInjection.imageTag | string | `nil` | |
| clusterAgent.admissionController.agentSidecarInjection.profiles | list | `[]` | Defines the sidecar configuration override, currently only one profile is supported. |
| clusterAgent.admissionController.agentSidecarInjection.provider | string | `nil` | Used by the admission controller to add infrastructure provider-specific configurations to the Agent sidecar. |
| clusterAgent.admissionController.agentSidecarInjection.selectors | list | `[]` | Defines the pod selector for sidecar injection, currently only one rule is supported. |
| clusterAgent.admissionController.configMode | string | `nil` | The kind of configuration to be injected, it can be "hostip", "service", or "socket". |
| clusterAgent.admissionController.enabled | bool | `true` | Enable the admissionController to be able to inject APM/Dogstatsd config and standard tags (env, service, version) automatically into your pods |
| clusterAgent.admissionController.failurePolicy | string | `"Ignore"` | Set the failure policy for dynamic admission control.' |
Expand Down
50 changes: 50 additions & 0 deletions charts/datadog/templates/_ac-agent-sidecar-env.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
{{- define "ac-agent-sidecar-env" -}}
{{- if and .Values.clusterAgent.admissionController.enabled .Values.clusterAgent.admissionController.agentSidecarInjection.enabled }}
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_ENABLED
value: "true"
{{- if .Values.clusterAgent.admissionController.agentSidecarInjection.clusterAgentCommunicationEnabled }}
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_CLUSTER_AGENT_ENABLED
value: "true"
{{- else }}
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_CLUSTER_AGENT_ENABLED
value: "false"
{{- end }}
{{- if .Values.clusterAgent.admissionController.agentSidecarInjection.provider }}
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_PROVIDER
value: {{ .Values.clusterAgent.admissionController.agentSidecarInjection.provider }}
{{- end }}

{{- if .Values.clusterAgent.admissionController.agentSidecarInjection.containerRegistry }}
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_CONTAINER_REGISTRY
value: {{ .Values.clusterAgent.admissionController.agentSidecarInjection.containerRegistry }}
{{- else if .Values.registry }}
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_CONTAINER_REGISTRY
value: {{ .Values.registry }}
{{- end }}

{{- if .Values.clusterAgent.admissionController.agentSidecarInjection.imageName }}
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_IMAGE_NAME
value: {{ .Values.clusterAgent.admissionController.agentSidecarInjection.imageName }}
{{- else if .Values.agents.image.name}}
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_IMAGE_NAME
value: {{ .Values.agents.image.name }}
{{- end }}

{{- if .Values.clusterAgent.admissionController.agentSidecarInjection.imageTag }}
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_IMAGE_TAG
value: {{ .Values.clusterAgent.admissionController.agentSidecarInjection.imageTag }}
{{- else if .Values.agents.image.tag}}
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_IMAGE_TAG
value: {{ .Values.agents.image.tag }}
{{- end }}

{{- if .Values.clusterAgent.admissionController.agentSidecarInjection.selectors }}
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_SELECTORS
value: '{{ toJson .Values.clusterAgent.admissionController.agentSidecarInjection.selectors }}'
{{- end }}
{{- if .Values.clusterAgent.admissionController.agentSidecarInjection.profiles }}
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_PROFILES
value: '{{ toJson .Values.clusterAgent.admissionController.agentSidecarInjection.profiles }}'
{{- end }}
{{- end }}
{{- end }}
1 change: 1 addition & 0 deletions charts/datadog/templates/cluster-agent-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,7 @@ spec:
- name: DD_ADMISSION_CONTROLLER_AUTO_INSTRUMENTATION_PATCHER_ENABLED
value: "true"
{{- end }}
{{ include "ac-agent-sidecar-env" . | nindent 10 }}
- name: DD_REMOTE_CONFIGURATION_ENABLED
value: {{ include "clusterAgent-remoteConfiguration-enabled" . | quote }}
{{- if .Values.datadog.apm.instrumentation.enabled }}
Expand Down
50 changes: 50 additions & 0 deletions charts/datadog/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1061,6 +1061,56 @@ clusterAgent:
# clusterAgent.admissionController.port -- Set port of cluster-agent admission controller service
port: 8000

agentSidecarInjection:
# clusterAgent.admissionController.agentSidecarInjection.enabled -- Enables Datadog Agent sidecar injection.

## When enabled, the admission controller mutating webhook will inject an Agent sidecar with minimal configuration in every pod meeting the configured criteria.
enabled: false

# clusterAgent.admissionController.agentSidecarInjection.provider -- Used by the admission controller to add infrastructure provider-specific configurations to the Agent sidecar.

## Currently only "fargate" is supported. To use the feature in other environments (including local testing) omit the config.
## ref: https://docs.datadoghq.com/integrations/eks_fargate
provider:

# clusterAgent.admissionController.agentSidecarInjection.clusterAgentCommunicationEnabled -- Enable communication between Agent sidecars and the Cluster Agent.
clusterAgentCommunicationEnabled: true

# clusterAgent.admissionController.containerRegistry -- Override the default registry for the sidecar Agent.
containerRegistry:

# clusterAgent.admissionController.imageName -- Override the default agents.image.name for the Agent sidecar.
imageName:

# clusterAgent.admissionController.imageTag -- Override the default agents.image.tag for the Agent sidecar.
imageTag:

# clusterAgent.admissionController.agentSidecarInjection.selectors -- Defines the pod selector for sidecar injection, currently only one rule is supported.
selectors: []
# - objectSelector:
# matchLabels:
# "podlabelKey1": podlabelValue1
# "podlabelKey2": podlabelValue2
# namespaceSelector:
# matchLabels:
# "nsLabelKey1": nsLabelValue1
# "nsLabelKey2": nsLabelValue2

# clusterAgent.admissionController.agentSidecarInjection.profiles -- Defines the sidecar configuration override, currently only one profile is supported.

## This setting allows overriding the sidecar Agent configuration by adding environment variables and providing resource settings.
profiles: []
# - env:
# - name: DD_ORCHESTRATOR_EXPLORER_ENABLED
# value: "true"
# resources:
# requests:
# cpu: "1"
# memory: "512Mi"
# limits:
# cpu: "2"
# memory: "1024Mi"

# clusterAgent.confd -- Provide additional cluster check configurations. Each key will become a file in /conf.d.

## ref: https://docs.datadoghq.com/agent/autodiscovery/
Expand Down
8 changes: 7 additions & 1 deletion test/common/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ func RenderChart(t *testing.T, cmd HelmCommand) (string, error) {
ValuesFiles: cmd.Values,
}

output, err := helm.RenderTemplateE(t, options, chartPath, cmd.ReleaseName, cmd.ShowOnly)
output, err := helm.RenderTemplateE(t, options, chartPath, cmd.ReleaseName, cmd.ShowOnly, "--debug")

return output, err
}
Expand Down Expand Up @@ -80,6 +80,12 @@ func CreateSecretFromEnv(t *testing.T, kubectlOptions *k8s.KubectlOptions, apiKe
}
}

func ReadFile(t *testing.T, filepath string) string {
fileContent, err := os.ReadFile(filepath)
require.NoError(t, err, "can't load manifest from file", "path", filepath)
return string(fileContent)
}

func LoadFromFile[T any](t *testing.T, filepath string, destObj *T) string {
fileContent, err := os.ReadFile(filepath)
require.NoError(t, err, "can't load manifest from file", "path", filepath)
Expand Down
181 changes: 181 additions & 0 deletions test/datadog/baseline/agent-clusterchecks-deployment_default.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
---
# Source: datadog/templates/agent-clusterchecks-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: datadog-clusterchecks
namespace: datadog-agent
labels:
helm.sh/chart: 'datadog-3.58.2'
app.kubernetes.io/name: "datadog"
app.kubernetes.io/instance: "datadog"
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/version: "7"
app.kubernetes.io/component: clusterchecks-agent

spec:
replicas: 2
revisionHistoryLimit: 10
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
type: RollingUpdate
selector:
matchLabels:
app: datadog-clusterchecks
template:
metadata:
labels:
app.kubernetes.io/name: "datadog"
app.kubernetes.io/instance: "datadog"
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: clusterchecks-agent
admission.datadoghq.com/enabled: "false"
app: datadog-clusterchecks

name: datadog-clusterchecks
annotations:
checksum/clusteragent_token: 999b326e98e9596150bcbfd45becfdc4695634b0d8198c59d43ce7043ac9a611
checksum/install_info: 3c5d7a2732f453d72b241f37b74f59319bcbf51e387a8fc35dc47bc4a1a7a390
spec:
serviceAccountName: datadog-cluster-checks
automountServiceAccountToken: true
imagePullSecrets:
[]
initContainers:
- name: init-volume
image: "gcr.io/datadoghq/agent:7.51.0"
imagePullPolicy: IfNotPresent
command: ["bash", "-c"]
args:
- cp -r /etc/datadog-agent /opt
volumeMounts:
- name: config
mountPath: /opt/datadog-agent
readOnly: false # Need RW for writing agent config files
resources:
{}
- name: init-config
image: "gcr.io/datadoghq/agent:7.51.0"
imagePullPolicy: IfNotPresent
command: ["bash", "-c"]
args:
- for script in $(find /etc/cont-init.d/ -type f -name '*.sh' | sort) ; do bash $script ; done
volumeMounts:
- name: config
mountPath: /etc/datadog-agent
readOnly: false # Need RW for writing datadog.yaml config file
resources:
{}
containers:
- name: agent
image: "gcr.io/datadoghq/agent:7.51.0"
command: ["bash", "-c"]
args:
- rm -rf /etc/datadog-agent/conf.d && touch /etc/datadog-agent/datadog.yaml && exec agent run
imagePullPolicy: IfNotPresent
env:
- name: DD_API_KEY
valueFrom:
secretKeyRef:
name: "datadog-secret"
key: api-key
- name: KUBERNETES
value: "yes"
- name: DD_LOG_LEVEL
value: "INFO"
- name: DD_EXTRA_CONFIG_PROVIDERS
value: "clusterchecks"
- name: DD_HEALTH_PORT
value: "5557"
# Cluster checks (cluster-agent communication)
- name: DD_CLUSTER_AGENT_ENABLED
value: "true"
- name: DD_CLUSTER_AGENT_KUBERNETES_SERVICE_NAME
value: datadog-cluster-agent
- name: DD_CLUSTER_AGENT_AUTH_TOKEN
valueFrom:
secretKeyRef:
name: datadog-cluster-agent
key: token
# Safely run alongside the daemonset
- name: DD_ENABLE_METADATA_COLLECTION
value: "false"
# Expose CLC stats
- name: DD_CLC_RUNNER_ENABLED
value: "true"
- name: DD_CLC_RUNNER_HOST
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: DD_CLC_RUNNER_ID
valueFrom:
fieldRef:
fieldPath: metadata.name
# Remove unused features
- name: DD_USE_DOGSTATSD
value: "false"
- name: DD_PROCESS_AGENT_ENABLED
value: "false"
- name: DD_LOGS_ENABLED
value: "false"
- name: DD_APM_ENABLED
value: "false"
- name: DD_REMOTE_CONFIGURATION_ENABLED
value: "false"
- name: DD_HOSTNAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName


resources:
{}
volumeMounts:
- name: installinfo
subPath: install_info
mountPath: /etc/datadog-agent/install_info
readOnly: true
- name: config
mountPath: /etc/datadog-agent
readOnly: false # Need RW for config path
livenessProbe:
failureThreshold: 6
httpGet:
path: /live
port: 5557
scheme: HTTP
initialDelaySeconds: 15
periodSeconds: 15
successThreshold: 1
timeoutSeconds: 5
readinessProbe:
failureThreshold: 6
httpGet:
path: /ready
port: 5557
scheme: HTTP
initialDelaySeconds: 15
periodSeconds: 15
successThreshold: 1
timeoutSeconds: 5
volumes:
- name: installinfo
configMap:
name: datadog-installinfo
- name: config
emptyDir: {}
affinity:
# Prefer scheduling the runners on different nodes if possible
# for better checks stability in case of node failure.
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 50
podAffinityTerm:
labelSelector:
matchLabels:
app: datadog-clusterchecks
topologyKey: kubernetes.io/hostname
nodeSelector:
kubernetes.io/os: linux
Loading

0 comments on commit 433211e

Please sign in to comment.