[Security] Too many RBAC permissions for the antrea-agent ServiceAccount

[antoninbas]

This issue discloses a possible security issue in older Antrea releases, for which the antrea-agent ServiceAccount was granted too many permissions through K8s RBAC. Some of these permissions could have been used to do significant damage to other workloads and to the clusters, should a Node have become compromised (e.g. through a container escape) in a way that an attacker could gain access to the antrea-agent K8s API token.

At the time of writing this issue, Antrea has already been patched to address these security concerns. The following versions include all the necessary patches:

• Antrea v1.6.1 (as well as all future patch releases for the 1.6 release train)
• All minor releases >= v1.7 (at the time of writing this issue, Antrea v1.7.0 has not been released yet)

Credit goes to security researchers @yuvalavra and @sbenhai for reporting this issue to the Antrea maintainers.

The rest of this issue will list all the individual RBAC permissions that were identified as potential security risks. For each one, we will describe the steps taken to mitigate the risk, and point to the relevant patch when appropriate.

Please note that each possible attack and privilege escalation assumes that an attacker has already been successful in escaping from a container and gaining access to the Node's filesystem and the antrea-agent token. Make sure you protect your production clusters to avoid such container escapes in the first place, by following K8s security best practices.

antrea-agent can patch Pods in the cluster (patch verb for pods resource)

• Rationale for the permission: the Agent needs to patch Pod annotations to implement the NodePortLocal feature.
• Risk: A successful attacker can modify the image for all Pods in the cluster and gain execution into all Pods, including a powerful Pod such as the Antrea Controller, and take over the entire cluster.
• Resolution: this permission is actually not required to patch Pod annotations; instead, granting permission to patch the pods/status subresource is sufficient, since annotations are part of metadata and not of the spec.
• Patch: Reduce permission of antrea service accounts #3393
• Patched versions: v1.5.1+, v1.6.x

antrea-agent can patch Nodes in the cluster (patch verb for nodes resource)

• Rationale for the permission: the Agent needs to add the Node’s MAC address as a Node annotation for better networking performance on Windows (noEncap mode)
• Risk: A successful attacker can force powerful Pods (e.g. the Antrea Controller) to be scheduled on the compromised Node, and access their tokens to gain admin-level permissions to the cluster.
• Resolution: this permission is actually not required to patch Node annotations; instead, granting permission to patch the nodes/status subresource is sufficient, since annotations are part of metadata and not of the spec.
• Patch: Reduce permission of antrea service accounts #3393
• Patched versions: v1.5.1+, v1.6.x

antrea-agent can patch and update the services/status subresource

• Rationale for the permission: the Agent used that permission to improve the performance of the ServiceExternalIP feature for Services with ExternalTrafficPolicy set to Local.
• Risk: A successful attacker can abuse the token of the antrea-agent pod to launch man-in-the-middle attacks against the cluster by exploiting the unfixed Kubernetes vulnerability, CVE-2020-8554, through LoadBalancerIPs.
• Resolution: the permission was removed altogether since it was used for a niche performance optimization; we will be able to achieve the performance optimization differently in the future if needed.
• Patch: Reduce permission of antrea-agent service account #3691
• Patched versions: v1.6.1, v1.7.x

antrea-agent can patch the pods/status subresource

• Rationale for the permission: this is a consequence of restricting a previously existing permissions (patch pods became patch pods/status) as described above. The Agent needs to be able to patch Pod annotations.
• Risk: a successful attacker could abuse this permission to re-label Pods to facilitate confused deputy attacks against built-in controllers. For example, making a Pod match a Service selector in order to man-in-the-middle (MITM) the Service traffic, or making a Pod match a ReplicaSet selector so that the ReplicaSet controller deletes legitimate replicas.
• Resolution: after review, we decided that it was not possible to remove this permission. Other solutions that run as a DaemonSet (e.g., other CNIs) often grant the same permission to their "agent". While K8s does not provide a built-in way to limit patch operations to a specific annotation, it can be achieved using OPA / Gatekeeper. We have therefore decided to 1) keep the permission as it is, 2) document the risk in the Antrea documentation, 3) document how Gatekeeper can be used to prevent possible attackers from leveraging this permission to do more harm.
• Documentation: https://github.com/antrea-io/antrea/blob/v1.6.1/docs/security.md

antrea-agent can patch the nodes/status subresource

• Rationale for the permission: this is a consequence of restricting a previously existing permissions (patch nodes became patch nodes/status) as described above. The Agent needs to be able to patch Node annotations.
• Risk: a successful attacker could abuse this permission to affect scheduling by modifying Node fields like labels, capacity, and conditions.
• Resolution: after review, we decided that it was not possible to remove this permission. Other solutions that run as a DaemonSet (e.g., other CNIs) often grant the same permission to their "agent". While K8s does not provide a built-in way to limit patch operations to a specific annotation, it can be achieved using OPA / Gatekeeper. We have therefore decided to 1) keep the permission as it is, 2) document the risk in the Antrea documentation, 3) document how Gatekeeper can be used to prevent possible attackers from leveraging this permission to do more harm.
• Documentation: https://github.com/antrea-io/antrea/blob/v1.6.1/docs/security.md

[antoninbas]

Closing this issue as it is a security disclosure and we have already patched the latest Antrea minor release (v1.6) appropriately.