From 314aa372e8c68baf0d6363f1820e103ffe3fe8de Mon Sep 17 00:00:00 2001 From: "abby.huang" <78209557+abby-cyber@users.noreply.github.com> Date: Fri, 27 Oct 2023 18:08:32 +0800 Subject: [PATCH] add new doc for admission control (#34) * add new doc for admission control * add eng content * comment fix * format fix * add-new-doc-for-admission-control * minor changes --- .../3.1create-cluster-with-kubectl.md | 6 +- .../8.6.admission-control.md | 102 +++++++++++++++++ .../3.1create-cluster-with-kubectl.md | 6 +- .../8.6.admission-control.md | 103 ++++++++++++++++++ mkdocs.yml | 4 +- 5 files changed, 214 insertions(+), 7 deletions(-) create mode 100644 docs-2.0-en/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md create mode 100644 docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md diff --git a/docs-2.0-en/nebula-operator/3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md b/docs-2.0-en/nebula-operator/3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md index 31efe7c0aa9..34da86f4b6f 100644 --- a/docs-2.0-en/nebula-operator/3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md +++ b/docs-2.0-en/nebula-operator/3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md @@ -159,9 +159,9 @@ The following example shows how to create a NebulaGraph cluster by creating a cl The following parameters must be customized: - - `spec.metad.licenseManagerURL` - - `spec..image` - - `spec.imagePullSecrets` + - `spec.metad.licenseManagerURL` + - `spec..image` + - `spec.imagePullSecrets` Other configurable parameters are as follows: diff --git a/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md b/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md new file mode 100644 index 00000000000..653a7bab384 --- /dev/null +++ b/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md @@ -0,0 +1,102 @@ +# Enable admission control + +Kubernetes [Admission Control](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/) is a security mechanism running as a webhook at runtime. It intercepts and modifies requests to ensure the cluster's security. Admission webhooks involve two main operations: validation and mutation. NebulaGraph Operator supports only validation operations and provides some default admission control rules. This topic describes NebulaGraph Operator's default admission control rules and how to enable admission control. + +## Prerequisites + +You have already created a cluster using Kubernetes. For detailed steps, see [Creating a NebulaGraph Cluster with Kubectl](../3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md). + +## Admission control rules + +Kubernetes admission control allows you to insert custom logic or policies before Kubernetes API Server processes requests. This mechanism can be used to implement various security policies, such as restricting a Pod's resource consumption or limiting its access permissions. NebulaGraph Operator supports validation operations, which means it validates and intercepts requests without making changes. NebulaGraph Operator's default admission validation control rules include: + +- Ensuring the minimum number of replicas in high availability mode: + + - For Graph service: At least 2 replicas are required. + - For Meta service: At least 3 replicas are required. + - For Storage service: At least 3 replicas are required. + + !!! note + + High availability mode refers to the high availability of NebulaGraph cluster services. Storage and Meta services are stateful, and the number of replicas should be an odd number due to [Raft](../../1.introduction/3.nebula-graph-architecture/4.storage-service.md#raft) protocol requirements for data consistency. In high availability mode, at least 3 Storage services and 3 Meta services are required. Graph services are stateless, so their number of replicas can be even but should be at least 2. + +- Preventing additional PVs from being added to Storage service via `dataVolumeClaims`. + +- Disallowing shrinking the capacity of all service's PVCs, but allowing expansion. + +- Forbidding any secondary operation during Storage service scale-in/scale-out. + +## TLS certificates for admission webhooks + +To ensure secure communication and data integrity between the K8s API server and the admission webhook, this communication is done over HTTPS by default. This means that TLS certificates are required for the admission webhook. [cert-manager](https://cert-manager.io/docs/) is a Kubernetes certificate management controller that automates the issuance and renewal of certificates. NebulaGraph Operator uses cert-manager to manage certificates. + +Once cert-manager is installed and admission control is enabled, NebulaGraph Operator will automatically create an [Issuer](https://cert-manager.io/docs/concepts/issuer/) for issuing the necessary certificate for the admission webhook, and a [Certificate](https://cert-manager.io/docs/concepts/certificate/) for storing the issued certificate. The issued certificate is stored in the `nebula-operator-webhook-secret` Secret. + +## Steps of enabling admission control + +1. Install cert-manager. + + ```bash + kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.1/cert-manager.yaml + ``` + + It is suggested to deploy the latest version of cert-manager. For details, see the [official cert-manager documentation](https://cert-manager.io/docs/installation/). + +2. Modify the NebulaGraph Operator configuration file to enable admission control. Admission control is disabled by default and needs to be enabled manually. + + ```bash + # Check the current configuration + helm show values nebula-operator/nebula-operator + ``` + + ```bash + # Modify the configuration by setting `enableAdmissionWebhook` to `true`. + helm upgrade nebula-operator nebula-operator/nebula-operator --set enableAdmissionWebhook=true + ``` + + !!! note + + `nebula-operator` is the name of the chart repository, and `nebula-operator/nebula-operator` is the chart name. If the chart's namespace is not specified, it defaults to `default`. + +3. View the certificate Secret for the admission webhook. + + ```bash + kubectl get secret nebula-operator-webhook-secret -o yaml + ``` + + If the output includes certificate contents, it means that the admission webhook's certificate has been successfully created. + +4. Verify the control rules. + + - Verify the minimum number of replicas in high availability mode. + + ```bash + # Annotate the cluster to enable high availability mode. + $ kubectl annotate nc nebula nebula-graph.io/ha-mode=true + # Verify the minimum number of the Graph service's replicas. + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"graphd": {"replicas":1}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: spec.graphd.replicas: Invalid value: 1: should be at least 2 in HA mode + ``` + + - Verify preventing additional PVs from being added to Storage service. + + ```bash + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"dataVolumeClaims":[{"resources": {"requests": {"storage": "2Gi"}}, "storageClassName": "local-path"},{"resources": {"requests": {"storage": "3Gi"}}, "storageClassName": "fask-disks"}]}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" deniedthe request: spec.storaged.dataVolumeClaims: Forbidden: storaged dataVolumeClaims is immutable + ``` + + - Verify disallowing shrinking Storage service's PVC capacity. + + ```bash + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"dataVolumeClaims":[{"resources": {"requests": {"storage": "1Gi"}}, "storageClassName": "fast-disks"}]}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: spec.storaged.dataVolumeClaims: Invalid value: resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}: data volume size can only be increased + ``` + + - Verify disallowing any secondary operation during Storage service scale-in. + + ```bash + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"replicas": 5}}}' + nebulacluster.apps.nebula-graph.io/nebula patched + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"replicas": 3}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: [spec.storaged: Forbidden: field is immutable while in ScaleOut phase, spec.storaged.replicas: Invalid value: 3: field is immutable while not in Running phase] + ``` diff --git a/docs-2.0-zh/nebula-operator/3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md b/docs-2.0-zh/nebula-operator/3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md index 9ef2a7f933b..beb5e75d3e0 100644 --- a/docs-2.0-zh/nebula-operator/3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md +++ b/docs-2.0-zh/nebula-operator/3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md @@ -161,9 +161,9 @@ 必须自定义修改以下参数，其他参数可按需修改。 - - `spec.metad.licenseManagerURL` - - `spec..image` - - `spec.imagePullSecrets` + - `spec.metad.licenseManagerURL` + - `spec..image` + - `spec.imagePullSecrets` 可配置的参数描述如下： diff --git a/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md b/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md new file mode 100644 index 00000000000..2b5d916831b --- /dev/null +++ b/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md @@ -0,0 +1,103 @@ +# 开启准入控制 + +K8s 的[准入控制（Admission Control）](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/)是一种安全机制，并在运行时作为 Webhook 运行。通过准入 Webhook 对请求进行拦截和修改，从而保证集群的安全性。准入 Webhook 操作包括验证（Validating）和变更（Mutating）两类。NebulaGraph Operator 仅支持验证操作，并提供一些默认的准入控制规则。本文介绍 NebulaGraph Operator 的默认准入控制规则及如何开启准入控制。 + +## 前提条件 + +已使用 K8s 创建一个集群。具体步骤，参见[使用 Kubectl 创建{{nebula.name}}集群](../3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md)。 + +## 准入控制规则 + +K8s 的准入控制允许用户在 Kubernetes API Server 处理请求之前，插入自定义的逻辑或策略。这种机制可以用于实现一些安全策略，比如限制 Pod 的资源使用量，或者限制 Pod 的访问权限等。NebulaGraph Operator 仅支持验证操作，即对请求进行验证和拦截，不支持对请求进行变更操作。NebulaGraph Operator 默认的准入验证控制规则包括： + +- 满足高可用模式下的最小副本数： + + - Graph 服务：至少需要 2 个副本。 + - Meta 服务：至少需要 3 个副本。 + - Storage 服务：至少需要 3 个副本。 + + !!! note + + 高可用模式是指{{nebula.name}}集群服务的高可用。Storage 服务和 Meta 服务是有状态的服务，其副本数据通过 [Raft](../../1.introduction/3.nebula-graph-architecture/4.storage-service.md#raft) 协议保持一致性且副本数量不能为偶数。因此，高可用模式下，至少需要 3 个 Storage 服务和 3 个 Meta 服务。Graph 服务为无状态的服务，因此其副本数量可以为偶数，但至少需要 2 个副本。 + +- 禁止通过`dataVolumeClaims`为 Storage 服务追加额外的 PV。 + +- 禁止缩小所有服务的 PVC 的容量，但是可以扩容。 + +- 禁止在 Storage 服务扩缩容期间，进行任何二次操作。 + +## 为准入 Webhook 创建证书 + +为了确保通信的安全性和数据的完整性，K8s 的 API server 和准入 Webhook 之间的通信默认通过 HTTPS 协议进行，因此使用准入控制还需要为准入 Webhook 提供 TLS 证书。[cert-manager](https://cert-manager.io/docs/) 是一个 K8s 的证书管理控制器，可以自动化证书的签发和更新。NebulaGraph Operator 使用 cert-manager 来管理证书。 + +当 cert-manager 安装完成并且开启准入控制时，NebulaGraph Operator 会自动创建一个 [Issuer](https://cert-manager.io/docs/concepts/issuer/)，用于签发准入 Webhook 所需的证书，同时会创建一个 [Certificate](https://cert-manager.io/docs/concepts/certificate/)，用于存储签发的证书。签发的证书被存储在 +`nebula-operator-webhook-secret`的 Secret 中。 + +## 开启准入控制 + +1. 安装部署 cert-manager。 + + ```bash + kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.1/cert-manager.yaml + ``` + + 建议部署最新版本 cert-manager。详情参见 [cert-manager 官方文档](https://cert-manager.io/docs/installation/)。 + +2. 修改 NebulaGraph Operator 的配置文件，开启准入控制。默认准入控制是关闭的，需要手动开启。 + + ```bash + # 查看当前的配置 + helm show values nebula-operator/nebula-operator + ``` + + ```bash + # 修改配置，将`enableAdmissionWebhook`设置为`true`。 + helm upgrade nebula-operator nebula-operator/nebula-operator --set enableAdmissionWebhook=true + ``` + + !!! note + + `nebula-operator`为 chart 所在仓库的名称，`nebula-operator/nebula-operator`为 chart 的名称。如果没有指定 chart 的命名空间，默认为`default`。 + +3. 查看准入 Webhook 的证书 Secret。 + + ```bash + kubectl get secret nebula-operator-webhook-secret -o yaml + ``` + + 如果输出的结果中包含证书内容，则表示准入 Webhook 的证书已经创建成功。 + +4. 验证控制规则。 + + - 验证高可用模式下的最小副本数。 + + ```bash + # 标注集群为高可用模式 + $ kubectl annotate nc nebula nebula-graph.io/ha-mode=true + # 验证 Graph 服务的最小副本数 + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"graphd": {"replicas":1}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: spec.graphd.replicas: Invalid value: 1: should be at least 2 in HA mode + ``` + + - 验证禁止通过`dataVolumeClaims`为 Storage 服务追加额外的 PV。 + + ```bash + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"dataVolumeClaims":[{"resources": {"requests": {"storage": "2Gi"}}, "storageClassName": "local-path"},{"resources": {"requests": {"storage": "3Gi"}}, "storageClassName": "fask-disks"}]}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: spec.storaged.dataVolumeClaims: Forbidden: storaged dataVolumeClaims is immutable + ``` + + - 验证禁止缩小 Storage 服务的 PVC 的容量。 + + ```bash + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"dataVolumeClaims":[{"resources": {"requests": {"storage": "1Gi"}}, "storageClassName": "fast-disks"}]}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: spec.storaged.dataVolumeClaims: Invalid value: resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}: data volume size can only be increased + ``` + + - 验证禁止在 Storage 服务缩容期间，进行任何二次操作。 + + ```bash + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"replicas": 5}}}' + nebulacluster.apps.nebula-graph.io/nebula patched + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"replicas": 3}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: [spec.storaged: Forbidden: field is immutable while in ScaleOut phase, spec.storaged.replicas: Invalid value: 3: field is immutable while not in Running phase] + ``` \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index c25e23a2cab..047f0f1a88c 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -723,8 +723,9 @@ nav: - 回收 PV: nebula-operator/8.custom-cluster-configurations/storage/8.2.pv-reclaim.md - 管理集群日志: nebula-operator/8.custom-cluster-configurations/8.4.manage-running-logs.md - 启用 TLS 加密: nebula-operator/8.custom-cluster-configurations/8.5.enable-ssl.md - - 配置滚动更新策略: nebula-operator/11.rolling-update-strategy.md + - 开启准入控制: nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md - 升级悦数图数据库: nebula-operator/9.upgrade-nebula-cluster.md + - 配置滚动更新策略: nebula-operator/11.rolling-update-strategy.md - 备份与恢复: nebula-operator/10.backup-restore-using-operator.md - 故障自愈: nebula-operator/5.operator-failover.md - 常见问题: nebula-operator/7.operator-faq.md @@ -1161,6 +1162,7 @@ nav: - Reclaim PVs: nebula-operator/8.custom-cluster-configurations/storage/8.2.pv-reclaim.md - Manage cluster logs: nebula-operator/8.custom-cluster-configurations/8.4.manage-running-logs.md - Enable mTLS: nebula-operator/8.custom-cluster-configurations/8.5.enable-ssl.md + - Enable admission control: nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md - Upgrade NebulaGraph clusters: nebula-operator/9.upgrade-nebula-cluster.md - Specify a rolling update strategy: nebula-operator/11.rolling-update-strategy.md - Backup and restore: nebula-operator/10.backup-restore-using-operator.md