-
Notifications
You must be signed in to change notification settings - Fork 386
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for NodeNetworkPolicy datapath #5658
Merged
+4,579
−376
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| # Antrea Node NetworkPolicy | ||
|
|
||
| ## Table of Contents | ||
|
|
||
| <!-- toc --> | ||
| - [Introduction](#introduction) | ||
| - [Prerequisites](#prerequisites) | ||
| - [Usage](#usage) | ||
| - [Limitations](#limitations) | ||
| <!-- /toc --> | ||
|
|
||
| ## Introduction | ||
|
|
||
| Node NetworkPolicy is designed to secure the Kubernetes Nodes traffic. It is supported by Antrea starting with Antrea | ||
| v1.15. This guide demonstrates how to configure Node NetworkPolicy. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Node NetworkPolicy was introduced in v1.15 as an alpha feature and is disabled by default. A feature gate, | ||
| `NodeNetworkPolicy`, must be enabled in antrea-agent.conf in the `antrea-config` ConfigMap. | ||
|
|
||
| ```yaml | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: antrea-config | ||
| namespace: kube-system | ||
| data: | ||
| antrea-agent.conf: | | ||
| featureGates: | ||
| NodeNetworkPolicy: true | ||
| ``` | ||
|
|
||
| Alternatively, you can use the following helm installation command to enable the feature gate: | ||
|
|
||
| ```bash | ||
| helm install antrea antrea/antrea --namespace kube-system --set featureGates.NodeNetworkPolicy=true | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| Node NetworkPolicy is an extension of Antrea ClusterNetworkPolicy (ACNP). By specifying a `nodeSelector` in the | ||
| policy-level `appliedTo` without other selectors, an ACNP is applied to the selected Kubernetes Nodes. | ||
|
|
||
| An example Node NetworkPolicy that blocks ingress traffic from Pods with label `app=client` to Nodes with label | ||
| `kubernetes.io/hostname: k8s-node-control-plane`: | ||
|
|
||
| ```yaml | ||
| apiVersion: crd.antrea.io/v1beta1 | ||
| kind: ClusterNetworkPolicy | ||
| metadata: | ||
| name: ingress-drop-pod-to-node | ||
| spec: | ||
| priority: 5 | ||
| tier: application | ||
| appliedTo: | ||
| - nodeSelector: | ||
| matchLabels: | ||
| kubernetes.io/hostname: k8s-node-control-plane | ||
| ingress: | ||
| - name: drop-80 | ||
| action: Drop | ||
| from: | ||
| - podSelector: | ||
| matchLabels: | ||
| app: client | ||
| ports: | ||
| - protocol: TCP | ||
| port: 80 | ||
| ``` | ||
|
|
||
| An example Node NetworkPolicy that blocks egress traffic from Nodes with the label | ||
| `kubernetes.io/hostname: k8s-node-control-plane` to Nodes with the label `kubernetes.io/hostname: k8s-node-worker-1` | ||
| and some IP blocks: | ||
|
|
||
| ```yaml | ||
| apiVersion: crd.antrea.io/v1beta1 | ||
| kind: ClusterNetworkPolicy | ||
| metadata: | ||
| name: egress-drop-node-to-node | ||
| spec: | ||
| priority: 5 | ||
| tier: application | ||
| appliedTo: | ||
| - nodeSelector: | ||
| matchLabels: | ||
| kubernetes.io/hostname: k8s-node-control-plane | ||
| egress: | ||
| - name: drop-22 | ||
| action: Drop | ||
| to: | ||
| - nodeSelector: | ||
| matchLabels: | ||
| kubernetes.io/hostname: k8s-node-worker-1 | ||
| - ipBlock: | ||
| cidr: 192.168.77.0/24 | ||
| - ipBlock: | ||
| cidr: 10.10.0.0/24 | ||
| ports: | ||
| - protocol: TCP | ||
| port: 22 | ||
| ``` | ||
|
|
||
| ## Limitations | ||
|
|
||
| - This feature is currently only supported for Linux Nodes. | ||
| - Be cautious when you configure policies to Nodes, in particular, when configuring a default-deny policy applied to | ||
| Nodes. You should ensure Kubernetes and Antrea control-plane communication is exempt from the deny rules, otherwise | ||
| the cluster may go out-of-service and you may lose connectivity to the Nodes. | ||
| - Only ACNPs can be applied to Nodes. ANPs cannot be applied to Nodes. | ||
| - `nodeSelector` can only be specified in the policy-level `appliedTo` field, not in the rule-level `appliedTo`, and not | ||
| in a `Group` or `ClusterGroup`. | ||
| - ACNPs applied to Nodes cannot be applied to Pods at the same time. | ||
| - FQDN is not supported for ACNPs applied to Nodes. | ||
| - Layer 7 NetworkPolicy is not supported yet. | ||
| - For UDP or SCTP, when the `Reject` action is specified in an egress rule, it behaves identical to the `Drop` action. | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
General question: should we add more clarifications in the limitations section in terms of what it means to have a
ipBlockspecified in the ingress block of a node networkpolicy? This could cause a great deal of confusion for users. For example, consider a multi-cluster setup with non-overlapping CIDRs, pod-to-pod connectivity enabled with gateways. In case the node where the policy is applied to is not the gateway node, wouldn't the ingress ipBlock feature not work properly due to the SNAT at the gateway (if the user specify CIDRs from the other clusters)? In addition, out of cluster traffic tend to have similar issues due SNATs at cloud load balancers before they enter Nodes? @tnqnThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not specific to Node networkpolicy. Even for Pod networkpolicy, SNAT can happen in many cases (lke NodePort traffic), I feel it's not too hard for users to get that if your traffic gets SNATed before reaching the endpoint, the policy will be enforced based on the translated IP. But adding a note to explain the SNAT and IPBlock stuff in https://github.com/antrea-io/antrea/blob/main/docs/antrea-network-policy.md#notes-and-constraints sounds good to me.