-
Notifications
You must be signed in to change notification settings - Fork 386
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add support for NodeNetworkPolicy datapath
This PR introduces support for the NodeNetworkPolicy datapath, extending Antrea
ClusterNetworkPolicy (ACNP). The implementation leverages iptables and ipset for
enforcing rules, safeguarding Kubernetes Nodes.
There are four key components to implement the data path:
- Core iptables rule
- Integrated into static chains ANTREA-POL-INGRESS-RULES (ingress) or
ANTREA-POL-EGRESS-RULES (egress).
- Matches an ipset that includes NodeNetworkPolicy rule source or
destination IPs, or directly matches a single IP.
- Targets an action or a service chain created for NodeNetworkPolicy
rule with multiple services.
- Service iptables chain
- Created for NodeNetworkPolicy rule with multiple services.
- Service iptables rules:
- Added to the service chain for NodeNetworkPolicy rule, constructed from
rule services.
- From/To ipset:
- Created for a NodeNetworkPolicy rule, containing source (ingress) or
destination (egress) IPs.
Example ingress or egress core iptables rules organized by priorities:
```
:ANTREA-POL-INGRESS-RULES
-A ANTREA-POL-INGRESS-RULES -m set --match-set ANTREA-POL-RULE1-4 src -j ANTREA-POL-RULE1 -m comment --comment "Antrea: for rule RULE1, policy AntreaClusterNetworkPolicy:name1"
-A ANTREA-POL-INGRESS-RULES -m set --match-set ANTREA-POL-RULE2-4 src -p tcp --dport 8080 -j ACCEPT -m comment --comment "Antrea: for rule RULE2, policy AntreaClusterNetworkPolicy:name2"
-A ANTREA-POL-INGRESS-RULES -s 3.3.3.3/32 src -j ANTREA-POL-RULE3 -m comment --comment "Antrea: for rule RULE3, policy AntreaClusterNetworkPolicy:name3"
-A ANTREA-POL-INGRESS-RULES -s 4.4.4.4/32 -p tcp --dport 80 -j ACCEPT -m comment --comment "Antrea: for rule RULE4, policy AntreaClusterNetworkPolicy:name4"
```
Example service chain (for rule with multiple services)::
```
:ANTREA-POL-RULE1
-A ANTREA-POL-RULE1 -j ACCEPT -p tcp --dport 80
-A ANTREA-POL-RULE1 -j ACCEPT -p tcp --dport 443
```
Example ipset (for rule with multiple source or destination IPs)
```
Name: ANTREA-POL-RULE1-4
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 472
References: 1
Number of entries: 2
Members:
1.1.1.1
1.1.1.2
```
Signed-off-by: Hongliang Liu <lhongliang@vmware.com>- Loading branch information
1 parent
fed260a
commit 32e091b
Showing
38 changed files
with
4,579 additions
and
376 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| # Antrea Node NetworkPolicy | ||
|
|
||
| ## Table of Contents | ||
|
|
||
| <!-- toc --> | ||
| - [Introduction](#introduction) | ||
| - [Prerequisites](#prerequisites) | ||
| - [Usage](#usage) | ||
| - [Limitations](#limitations) | ||
| <!-- /toc --> | ||
|
|
||
| ## Introduction | ||
|
|
||
| Node NetworkPolicy is designed to secure the Kubernetes Nodes traffic. It is supported by Antrea starting with Antrea | ||
| v1.15. This guide demonstrates how to configure Node NetworkPolicy. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Node NetworkPolicy was introduced in v1.15 as an alpha feature and is disabled by default. A feature gate, | ||
| `NodeNetworkPolicy`, must be enabled in antrea-agent.conf in the `antrea-config` ConfigMap. | ||
|
|
||
| ```yaml | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: antrea-config | ||
| namespace: kube-system | ||
| data: | ||
| antrea-agent.conf: | | ||
| featureGates: | ||
| NodeNetworkPolicy: true | ||
| ``` | ||
|
|
||
| Alternatively, you can use the following helm installation command to enable the feature gate: | ||
|
|
||
| ```bash | ||
| helm install antrea antrea/antrea --namespace kube-system --set featureGates.NodeNetworkPolicy=true | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| Node NetworkPolicy is an extension of Antrea ClusterNetworkPolicy (ACNP). By specifying a `nodeSelector` in the | ||
| policy-level `appliedTo` without other selectors, an ACNP is applied to the selected Kubernetes Nodes. | ||
|
|
||
| An example Node NetworkPolicy that blocks ingress traffic from Pods with label `app=client` to Nodes with label | ||
| `kubernetes.io/hostname: k8s-node-control-plane`: | ||
|
|
||
| ```yaml | ||
| apiVersion: crd.antrea.io/v1beta1 | ||
| kind: ClusterNetworkPolicy | ||
| metadata: | ||
| name: ingress-drop-pod-to-node | ||
| spec: | ||
| priority: 5 | ||
| tier: application | ||
| appliedTo: | ||
| - nodeSelector: | ||
| matchLabels: | ||
| kubernetes.io/hostname: k8s-node-control-plane | ||
| ingress: | ||
| - name: drop-80 | ||
| action: Drop | ||
| from: | ||
| - podSelector: | ||
| matchLabels: | ||
| app: client | ||
| ports: | ||
| - protocol: TCP | ||
| port: 80 | ||
| ``` | ||
|
|
||
| An example Node NetworkPolicy that blocks egress traffic from Nodes with the label | ||
| `kubernetes.io/hostname: k8s-node-control-plane` to Nodes with the label `kubernetes.io/hostname: k8s-node-worker-1` | ||
| and some IP blocks: | ||
|
|
||
| ```yaml | ||
| apiVersion: crd.antrea.io/v1beta1 | ||
| kind: ClusterNetworkPolicy | ||
| metadata: | ||
| name: egress-drop-node-to-node | ||
| spec: | ||
| priority: 5 | ||
| tier: application | ||
| appliedTo: | ||
| - nodeSelector: | ||
| matchLabels: | ||
| kubernetes.io/hostname: k8s-node-control-plane | ||
| egress: | ||
| - name: drop-22 | ||
| action: Drop | ||
| to: | ||
| - nodeSelector: | ||
| matchLabels: | ||
| kubernetes.io/hostname: k8s-node-worker-1 | ||
| - ipBlock: | ||
| cidr: 192.168.77.0/24 | ||
| - ipBlock: | ||
| cidr: 10.10.0.0/24 | ||
| ports: | ||
| - protocol: TCP | ||
| port: 22 | ||
| ``` | ||
|
|
||
| ## Limitations | ||
|
|
||
| - This feature is currently only supported for Linux Nodes. | ||
| - Be cautious when you configure policies to Nodes, in particular, when configuring a default-deny policy applied to | ||
| Nodes. You should ensure Kubernetes and Antrea control-plane communication is exempt from the deny rules, otherwise | ||
| the cluster may go out-of-service and you may lose connectivity to the Nodes. | ||
| - Only ACNPs can be applied to Nodes. ANPs cannot be applied to Nodes. | ||
| - `nodeSelector` can only be specified in the policy-level `appliedTo` field, not in the rule-level `appliedTo`, and not | ||
| in a `Group` or `ClusterGroup`. | ||
| - ACNPs applied to Nodes cannot be applied to Pods at the same time. | ||
| - FQDN is not supported for ACNPs applied to Nodes. | ||
| - Layer 7 NetworkPolicy is not supported yet. | ||
| - For UDP or SCTP, when the `Reject` action is specified in an egress rule, it behaves identical to the `Drop` action. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.