Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial (alpha) documentation about sidecar containers #41802

Merged
merged 1 commit into from
Aug 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions content/en/docs/concepts/workloads/pods/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,17 @@ that updates those files from a remote source, as in the following diagram:

{{< figure src="/images/docs/pod.svg" alt="Pod creation diagram" class="diagram-medium" >}}

matthyx marked this conversation as resolved.
Show resolved Hide resolved
Some Pods have {{< glossary_tooltip text="init containers" term_id="init-container" >}} as well as {{< glossary_tooltip text="app containers" term_id="app-container" >}}. Init containers run and complete before the app containers are started.
Some Pods have {{< glossary_tooltip text="init containers" term_id="init-container" >}}
as well as {{< glossary_tooltip text="app containers" term_id="app-container" >}}.
By default, init containers run and complete before the app containers are started.

{{< feature-state for_k8s_version="v1.28" state="alpha" >}}
matthyx marked this conversation as resolved.
Show resolved Hide resolved

Enabling the [SidecarContainers feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit

Suggested change
Enabling the [SidecarContainers feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
Enabling the SidecarContainers` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)

allows specifying a <code>restartPolicy=Always</code> to init containers, making sure they are
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
allows specifying a <code>restartPolicy=Always</code> to init containers, making sure they are
allows you to specify `restartPolicy: Always` for init containers. Setting the `Always` restart policy
ensures that the init containers where you set it are

kept running during the entire lifetime of the Pod.
See [Sidecar containers and restartPolicy](/docs/concepts/workloads/pods/init-containers/#sidecar-containers-and-restartpolicy)
for more details.

Pods natively provide two kinds of shared resources for their constituent containers:
[networking](#pod-networking) and [storage](#pod-storage).
Expand Down Expand Up @@ -366,4 +376,4 @@ To understand the context for why Kubernetes wraps a common Pod API in other res
* [Borg](https://research.google.com/pubs/pub43438.html)
* [Marathon](https://mesosphere.github.io/marathon/docs/rest-api.html)
* [Omega](https://research.google/pubs/pub41684/)
* [Tupperware](https://engineering.fb.com/data-center-engineering/tupperware/).
* [Tupperware](https://engineering.fb.com/data-center-engineering/tupperware/).
43 changes: 42 additions & 1 deletion content/en/docs/concepts/workloads/pods/init-containers.md
Original file line number Diff line number Diff line change
Expand Up @@ -289,7 +289,47 @@ The Pod which is already running correctly would be killed by `activeDeadlineSec
The name of each app and init container in a Pod must be unique; a
validation error is thrown for any container sharing a name with another.

### Resources
#### API for sidecar containers

{{< feature-state for_k8s_version="v1.28" state="alpha" >}}

Starting with Kubernetes 1.28 in alpha, a feature gate named `SidecarContainers`
allows to specify a `restartPolicy` for init containers which is independent of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit)

Suggested change
allows to specify a `restartPolicy` for init containers which is independent of
allows you to specify a `restartPolicy` for init containers which is independent of

the Pod and other init containers. Container [probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe)
can also be added to control their lifecycle.

If an init container is created with its `restartPolicy` set to `Always`, it will
start and remain running during the entire life of the Pod, which is useful for
running supporting services separated from the main application containers.

If a `readinessProbe` is specified for this init container, its result will be used
to determine the `ready` state of the Pod.

Since these containers are defined as init containers, they benefit from the same
ordering and sequential guarantees as other init containers, allowing them to
be mixed with other init containers into complex Pod initialization flows.

The only difference is that they are not required to complete before the next
init container starts, so a next init container will start after the current
container status has been set to `Pod.status.containerStatuses[id of container].started=true`
Comment on lines +312 to +314
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The only difference is that they are not required to complete before the next
init container starts, so a next init container will start after the current
container status has been set to `Pod.status.containerStatuses[id of container].started=true`
Compared to regular init containers, sidecar-style init containers continue to
run and the next init container can being starting once the kubet has set
the `started` container status for the sidecar-style init container to true.
That status either becomes true because there is a process running in the
container and no startup probe defined, or

as a result of its `startupProbe` succeeding.

This feature can be used to implement the sidecar container pattern in a more
robust way, as the kubelet always restarts a sidecar container if it fails.

matthyx marked this conversation as resolved.
Show resolved Hide resolved
Here's an example of a Deployment with two containers, one of which is a sidecar:

{{% codenew language="yaml" file="application/deployment-sidecar.yaml" %}}

This feature is also useful for running Jobs with sidecars, as the sidecar
container will not prevent the Job from completing after the main container
has finished.

Here's an example of a Job with two containers, one of which is a sidecar:

{{% codenew language="yaml" file="application/job/job-sidecar.yaml" %}}

#### Resource sharing within containers

Given the ordering and execution for init containers, the following rules
for resource usage apply:
Expand Down Expand Up @@ -335,3 +375,4 @@ Kubernetes, consult the documentation for the version you are using.
* Learn how to [debug init containers](/docs/tasks/debug/debug-application/debug-init-containers/)
* Read about an overview of [kubelet](/docs/reference/command-line-tools-reference/kubelet/) and [kubectl](/docs/reference/kubectl/)
* Learn about the [types of probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe): liveness, readiness, startup probe.
*
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,7 @@ For a reference to old feature gates that are removed, please refer to
| `SecurityContextDeny` | `false` | Alpha | 1.27 | |
| `ServiceNodePortStaticSubrange` | `false` | Alpha | 1.27 | 1.27 |
| `ServiceNodePortStaticSubrange` | `true` | Beta | 1.28 | |
| `SidecarContainers` | `false` | Alpha | 1.28 | |
matthyx marked this conversation as resolved.
Show resolved Hide resolved
| `SizeMemoryBackedVolumes` | `false` | Alpha | 1.20 | 1.21 |
| `SizeMemoryBackedVolumes` | `true` | Beta | 1.22 | |
| `StableLoadBalancerNodeGet` | `true` | Beta | 1.27 | |
Expand Down Expand Up @@ -701,6 +702,11 @@ Each feature gate is designed for enabling/disabling a specific feature:
risk of collision. See
[Avoiding collisions](/docs/concepts/services-networking/service/#avoiding-collisions)
for more details.
- `SidecarContainers`: Allow setting the `restartPolicy` of an init container to
`Always` so that the container becomes a sidecar container (restartable init containers).
See
[Sidecar containers and restartPolicy](/docs/concepts/workloads/pods/init-containers/#sidecar-containers-and-restartpolicy)
for more details.
- `SizeMemoryBackedVolumes`: Enable kubelets to determine the size limit for
memory-backed volumes (mainly `emptyDir` volumes).
- `StableLoadBalancerNodeGet`: Enables less load balancer re-configurations by
Expand Down Expand Up @@ -749,4 +755,4 @@ Each feature gate is designed for enabling/disabling a specific feature:
feature, you will also need to enable any associated API resources.
For example, to enable a particular resource like
`storage.k8s.io/v1beta1/csistoragecapacities`, set `--runtime-config=storage.k8s.io/v1beta1/csistoragecapacities`.
See [API Versioning](/docs/reference/using-api/#api-versioning) for more details on the command line flags.
See [API Versioning](/docs/reference/using-api/#api-versioning) for more details on the command line flags.
34 changes: 34 additions & 0 deletions content/en/examples/application/deployment-sidecar.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
labels:
app: myapp
spec:
replicas: 1
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: alpine:latest
command: ['sh', '-c', 'echo "logging" > /opt/logs.txt']
volumeMounts:
- name: data
mountPath: /opt
initContainers:
- name: logshipper
image: alpine:latest
restartPolicy: Always
command: ['sh', '-c', 'tail /opt/logs.txt']
volumeMounts:
- name: data
mountPath: /opt
volumes:
- name: data
emptyDir: {}
26 changes: 26 additions & 0 deletions content/en/examples/application/job/job-sidecar.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
apiVersion: batch/v1
kind: Job
metadata:
name: myjob
spec:
template:
spec:
containers:
- name: myjob
image: alpine:latest
command: ['sh', '-c', 'echo "logging" > /opt/logs.txt']
volumeMounts:
- name: data
mountPath: /opt
initContainers:
- name: logshipper
image: alpine:latest
restartPolicy: Always
command: ['sh', '-c', 'tail /opt/logs.txt']
volumeMounts:
- name: data
mountPath: /opt
restartPolicy: Never
volumes:
- name: data
emptyDir: {}