Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved Documentation Regarding CNI Custom Network re 1.4.0 release #34

Merged
merged 2 commits into from
Jun 26, 2019
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 35 additions & 2 deletions doc_source/cni-custom-network.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@
By default, when new network interfaces are allocated for pods, [ipamD](https://github.com/aws/amazon-vpc-cni-k8s/blob/master/docs/cni-proposal.md) uses the worker node's primary elastic network interface's security groups and subnet\. However, there are use cases where your pod network interfaces should use a different security group or subnet, within the same VPC as your control plane security group\. For example:
+ There are a limited number of IP addresses available in a subnet\. This limits the number of pods can be created in the cluster\. Using different subnets for pod groups allows you to increase the number of available IP addresses\.
+ For security reasons, your pods must use different security groups or subnets than the node's primary network interface\.
+ The worker nodes are configured in public subnets and you wish for your pods to be placed in private subnets using a NAT Gateway\. In this situation, please also read about [External Source Network Address Translation](external-snat.md)\.

**Note**
This feature requires [Amazon VPC CNI plugin for Kubernetes](https://github.com/aws/amazon-vpc-cni-k8s) version 1\.2\.1 or later\. To check your CNI version, and upgrade if necessary, see [Amazon VPC CNI Plugin for Kubernetes Upgrades](cni-upgrades.md)\.
The use cases discussed in this topic require [Amazon VPC CNI plugin for Kubernetes](https://github.com/aws/amazon-vpc-cni-k8s) version 1\.4\.0 or later\. To check your CNI version, and upgrade if necessary, see [Amazon VPC CNI Plugin for Kubernetes Upgrades](cni-upgrades.md)\.

Enabling this feature effectively removes an available elastic network interface \(and all of its available IP addresses for pods\) from each worker node that uses it\. The primary network interface for the worker node is not used for pod placement when this feature is enabled\. You should choose larger instance types with more available elastic network interfaces if you choose to enable this feature\.

Expand Down Expand Up @@ -117,4 +118,36 @@ Each subnet and security group combination requires its own custom resource\.
subnet: subnet-017b472c2f79fdf96
```

1. If you have any worker nodes in your cluster that had pods placed on them before you completed this procedure, you should terminate them\. Only new nodes that are registered with the `k8s.amazonaws.com/eniConfig` label will use the new custom networking feature\.
1. If you have any worker nodes in your cluster that had pods placed on them before you completed this procedure, you should terminate them\. Only new nodes that are registered with the `k8s.amazonaws.com/eniConfig` label will use the new custom networking feature\.

**To automatically apply an ENIConfig to a node based on its Availability Zone**

By default, Kubernetes applies the availability zone of a node to the _failure-domain.beta.kubernetes.io/zone_ label. You can name your `ENIConfig` custom resources after each Availability Zone in your VPC, and then specify this label in the `ENI_CONFIG_LABEL_DEF` of the `aws-node` container spec for your worker nodes.

```
...
spec:
containers:
- env:
- name: AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG
value: "true"
- name: ENI_CONFIG_LABEL_DEF
value: failure-domain.beta.kubernetes.io/zone
- name: AWS_VPC_K8S_CNI_LOGLEVEL
value: DEBUG
- name: MY_NODE_NAME
...
```

For example, if `subnet-0c4678ec01ce68b24` is in the **us-east-1a** Availability Zone, you could use the following `ENIConfig` for the **us-east-1a** availability zone:

```
apiVersion: crd.k8s.amazonaws.com/v1alpha1
kind: ENIConfig
metadata:
name: us-east-1a
spec:
securityGroups:
- sg-08052d900a2c7fb0a
subnet: subnet-0c4678ec01ce68b24
```