diff --git a/docs-2.0-en/nebula-operator/1.introduction-to-nebula-operator.md b/docs-2.0-en/nebula-operator/1.introduction-to-nebula-operator.md index 2038c952474..aa737a098d8 100644 --- a/docs-2.0-en/nebula-operator/1.introduction-to-nebula-operator.md +++ b/docs-2.0-en/nebula-operator/1.introduction-to-nebula-operator.md @@ -32,7 +32,7 @@ NebulaGraph Operator does not support the v1.x version of NebulaGraph. NebulaGra | NebulaGraph | NebulaGraph Operator | | ------------- | -------------------- | -| 3.5.x ~ 3.6.0 | 1.5.0, 1.6.x | +| 3.5.x ~ 3.6.0 | 1.5.0 ~ 1.7.x | | 3.0.0 ~ 3.4.1 | 1.3.0, 1.4.0 ~ 1.4.2 | | 3.0.0 ~ 3.3.x | 1.0.0, 1.1.0, 1.2.0 | | 2.5.x ~ 2.6.x | 0.9.0 | diff --git a/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/8.3.balance-data-when-scaling-storage.md b/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/8.3.balance-data-when-scaling-storage.md deleted file mode 100644 index 2b41e8ed539..00000000000 --- a/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/8.3.balance-data-when-scaling-storage.md +++ /dev/null @@ -1,114 +0,0 @@ -# Balance storage data after scaling out - -!!! enterpriseonly - - This feature is for NebulaGraph Enterprise Edition only. - -After the Storage service is scaled out, you can decide whether to balance the data in the Storage service. - -The scaling out of the NebulaGraph's Storage service is divided into two stages. In the first stage, the status of all pods is changed to `Ready`. In the second stage, the commands of `BALANCE DATA` and `BALANCE LEADER` are executed to balance data. These two stages decouple the scaling out process of the controller replica from the balancing data process, so that you can choose to perform the data balancing operation during low traffic period. The decoupling of the scaling out process from the balancing process can effectively reduce the impact on online services during data migration. - -You can define whether to balance data automatically or not with the parameter `enableAutoBalance` in the configuration file of the CR instance of the cluster you created. - -## Prerequisites - -You have created a NebulaGraph cluster. For how to create a cluster with Kubectl, see [Create a cluster with Kubectl](../3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md). - -## Steps - -The following example uses a cluster named `nebula` and the cluster's configuration file named `nebula_cluster.yaml` to show how to set `enableAutoBalance`. - -1. Run the following command to access the edit page of the `nebula` cluster. - - ```bash - kubectl edit nebulaclusters.apps.nebula-graph.io nebula - ``` - -2. Add `enableAutoBalance` and set its value to `true` under `spec.storaged`. - - ```yaml - apiVersion: apps.nebula-graph.io/v1alpha1 - kind: NebulaCluster - metadata: - name: nebula - spec: - graphd: - image: vesoft/nebula-graphd - logVolumeClaim: - resources: - requests: - storage: 2Gi - storageClassName: fast-disks - replicas: 1 - resources: - limits: - cpu: "1" - memory: 1Gi - requests: - cpu: 500m - memory: 500Mi - version: {{nebula.tag}} - imagePullPolicy: IfNotPresent - metad: - dataVolumeClaim: - resources: - requests: - storage: 2Gi - storageClassName: fast-disks - image: vesoft/nebula-metad - logVolumeClaim: - resources: - requests: - storage: 2Gi - storageClassName: fast-disks - replicas: 1 - resources: - limits: - cpu: "1" - memory: 1Gi - requests: - cpu: 500m - memory: 500Mi - version: {{nebula.tag}} - nodeSelector: - nebula: cloud - reference: - name: statefulsets.apps - version: v1 - schedulerName: default-scheduler - storaged: - enableAutoBalance: true //Set its value to true which means storage data will be balanced after the Storage service is scaled out. - dataVolumeClaims: - - resources: - requests: - storage: 2Gi - storageClassName: fast-disks - - resources: - requests: - storage: 2Gi - storageClassName: fast-disks - image: vesoft/nebula-storaged - logVolumeClaim: - resources: - requests: - storage: 2Gi - storageClassName: fast-disks - replicas: 3 - resources: - limits: - cpu: "1" - memory: 1Gi - requests: - cpu: 500m - memory: 500Mi - version: {{nebula.tag}} - ... - ``` - - - When the value of `enableAutoBalance` is set to `true`, the Storage data will be automatically balanced after the Storage service is scaled out. - - - When the value of `enableAutoBalance` is set to `false`, the Storage data will not be automatically balanced after the Storage service is scaled out. - - - When the `enableAutoBalance` parameter is not set, the system will not automatically balance Storage data by default after the Storage service is scaled out. - -3. Run `kubectl apply -f nebula_cluster.yaml` to push your configuration changes to the cluster. \ No newline at end of file diff --git a/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md b/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md new file mode 100644 index 00000000000..653a7bab384 --- /dev/null +++ b/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md @@ -0,0 +1,102 @@ +# Enable admission control + +Kubernetes [Admission Control](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/) is a security mechanism running as a webhook at runtime. It intercepts and modifies requests to ensure the cluster's security. Admission webhooks involve two main operations: validation and mutation. NebulaGraph Operator supports only validation operations and provides some default admission control rules. This topic describes NebulaGraph Operator's default admission control rules and how to enable admission control. + +## Prerequisites + +You have already created a cluster using Kubernetes. For detailed steps, see [Creating a NebulaGraph Cluster with Kubectl](../3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md). + +## Admission control rules + +Kubernetes admission control allows you to insert custom logic or policies before Kubernetes API Server processes requests. This mechanism can be used to implement various security policies, such as restricting a Pod's resource consumption or limiting its access permissions. NebulaGraph Operator supports validation operations, which means it validates and intercepts requests without making changes. NebulaGraph Operator's default admission validation control rules include: + +- Ensuring the minimum number of replicas in high availability mode: + + - For Graph service: At least 2 replicas are required. + - For Meta service: At least 3 replicas are required. + - For Storage service: At least 3 replicas are required. + + !!! note + + High availability mode refers to the high availability of NebulaGraph cluster services. Storage and Meta services are stateful, and the number of replicas should be an odd number due to [Raft](../../1.introduction/3.nebula-graph-architecture/4.storage-service.md#raft) protocol requirements for data consistency. In high availability mode, at least 3 Storage services and 3 Meta services are required. Graph services are stateless, so their number of replicas can be even but should be at least 2. + +- Preventing additional PVs from being added to Storage service via `dataVolumeClaims`. + +- Disallowing shrinking the capacity of all service's PVCs, but allowing expansion. + +- Forbidding any secondary operation during Storage service scale-in/scale-out. + +## TLS certificates for admission webhooks + +To ensure secure communication and data integrity between the K8s API server and the admission webhook, this communication is done over HTTPS by default. This means that TLS certificates are required for the admission webhook. [cert-manager](https://cert-manager.io/docs/) is a Kubernetes certificate management controller that automates the issuance and renewal of certificates. NebulaGraph Operator uses cert-manager to manage certificates. + +Once cert-manager is installed and admission control is enabled, NebulaGraph Operator will automatically create an [Issuer](https://cert-manager.io/docs/concepts/issuer/) for issuing the necessary certificate for the admission webhook, and a [Certificate](https://cert-manager.io/docs/concepts/certificate/) for storing the issued certificate. The issued certificate is stored in the `nebula-operator-webhook-secret` Secret. + +## Steps of enabling admission control + +1. Install cert-manager. + + ```bash + kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.1/cert-manager.yaml + ``` + + It is suggested to deploy the latest version of cert-manager. For details, see the [official cert-manager documentation](https://cert-manager.io/docs/installation/). + +2. Modify the NebulaGraph Operator configuration file to enable admission control. Admission control is disabled by default and needs to be enabled manually. + + ```bash + # Check the current configuration + helm show values nebula-operator/nebula-operator + ``` + + ```bash + # Modify the configuration by setting `enableAdmissionWebhook` to `true`. + helm upgrade nebula-operator nebula-operator/nebula-operator --set enableAdmissionWebhook=true + ``` + + !!! note + + `nebula-operator` is the name of the chart repository, and `nebula-operator/nebula-operator` is the chart name. If the chart's namespace is not specified, it defaults to `default`. + +3. View the certificate Secret for the admission webhook. + + ```bash + kubectl get secret nebula-operator-webhook-secret -o yaml + ``` + + If the output includes certificate contents, it means that the admission webhook's certificate has been successfully created. + +4. Verify the control rules. + + - Verify the minimum number of replicas in high availability mode. + + ```bash + # Annotate the cluster to enable high availability mode. + $ kubectl annotate nc nebula nebula-graph.io/ha-mode=true + # Verify the minimum number of the Graph service's replicas. + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"graphd": {"replicas":1}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: spec.graphd.replicas: Invalid value: 1: should be at least 2 in HA mode + ``` + + - Verify preventing additional PVs from being added to Storage service. + + ```bash + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"dataVolumeClaims":[{"resources": {"requests": {"storage": "2Gi"}}, "storageClassName": "local-path"},{"resources": {"requests": {"storage": "3Gi"}}, "storageClassName": "fask-disks"}]}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" deniedthe request: spec.storaged.dataVolumeClaims: Forbidden: storaged dataVolumeClaims is immutable + ``` + + - Verify disallowing shrinking Storage service's PVC capacity. + + ```bash + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"dataVolumeClaims":[{"resources": {"requests": {"storage": "1Gi"}}, "storageClassName": "fast-disks"}]}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: spec.storaged.dataVolumeClaims: Invalid value: resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}: data volume size can only be increased + ``` + + - Verify disallowing any secondary operation during Storage service scale-in. + + ```bash + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"replicas": 5}}}' + nebulacluster.apps.nebula-graph.io/nebula patched + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"replicas": 3}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: [spec.storaged: Forbidden: field is immutable while in ScaleOut phase, spec.storaged.replicas: Invalid value: 3: field is immutable while not in Running phase] + ``` diff --git a/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/8.2.pv-reclaim.md b/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/storage/8.2.pv-reclaim.md similarity index 92% rename from docs-2.0-en/nebula-operator/8.custom-cluster-configurations/8.2.pv-reclaim.md rename to docs-2.0-en/nebula-operator/8.custom-cluster-configurations/storage/8.2.pv-reclaim.md index b8cda0a2aad..17e96ecea25 100644 --- a/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/8.2.pv-reclaim.md +++ b/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/storage/8.2.pv-reclaim.md @@ -6,10 +6,8 @@ You can also define the automatic deletion of PVCs to release data by setting th ## Prerequisites -You have created a cluster. For how to create a cluster with Kubectl, see [Create a cluster with Kubectl](../3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md). -## Notes +You have created a cluster. For how to create a cluster with Kubectl, see [Create a cluster with Kubectl](../../3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md). -NebulaGraph Operator does not support dynamically adding or mounting storage volumes to a running storaged instance. ## Steps diff --git a/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/storage/8.6.dynamic-expand-pv.md b/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/storage/8.6.dynamic-expand-pv.md new file mode 100644 index 00000000000..c2be5cc5e57 --- /dev/null +++ b/docs-2.0-en/nebula-operator/8.custom-cluster-configurations/storage/8.6.dynamic-expand-pv.md @@ -0,0 +1,99 @@ +# Dynamically expand persistent volumes + +In a Kubernetes environment, NebulaGraph's data is stored on Persistent Volumes (PVs). Dynamic volume expansion refers to increasing the capacity of a volume without stopping the service, enabling NebulaGraph to accommodate growing data. This topic explains how to dynamically expand the PV for NebulaGraph services in a Kubernetes environment. + +!!! note + + - After the cluster is created, you cannot dynamically increase the number of PVs while the cluster is running. + - The method described in this topic is only for online volume expansion and does not support volume reduction. + +## Background + +In Kubernetes, a StorageClass is a resource that defines a particular storage type. It describes a class of storage, including its provisioner, parameters, and other details. When creating a PersistentVolumeClaim (PVC) and specifying a StorageClass, Kubernetes automatically creates a corresponding PV. The principle of dynamic volume expansion is to edit the PVC and increase the volume's capacity. Kubernetes will then automatically expand the capacity of the PV associated with this PVC based on the specified `storageClassName` in the PVC. During this process, new PVs are not created; the size of the existing PV is changed. Only dynamic storage volumes, typically those associated with a `storageClassName`, support dynamic volume expansion. Additionally, the `allowVolumeExpansion` field in the StorageClass must be set to `true`. For more details, see the [Kubernetes documentation on expanding Persistent Volume Claims](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims). + +In NebulaGraph Operator, you cannot directly edit PVC because Operator automatically creates PVC based on the configuration in the `spec..dataVolumeClaim` of the Nebula Graph cluster. Therefore, you need to modify the cluster's configuration to update the PVC and trigger dynamic online volume expansion for the PV. + +## Prerequisites + +- Kubernetes version is equal to or greater than 1.18. +- A StorageClass has been created in the Kubernetes environment. For details, see [Expanding Persistent Volumes Claims](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims). + - Ensure the `allowVolumeExpansion` field in the StorageClass is set to `true`. + - Make sure that the `provisioner` configured in the StorageClass supports dynamic expansion. +- A NebulaGraph cluster has been created in Kubernetes. For specific steps, see [Create a Nebula Graph Cluster with Kubectl](../../3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md). +- NebulaGraph cluster Pods are in running status. + +## Online volume expansion example + +In the following example, we assume that the StorageClass is named `ebs-sc` and the NebulaGraph cluster is named `nebula`. We will demonstrate how to dynamically expand the PV for the Storage service. + +1. Check the status of the Storage service Pod: + + ```bash + kubectl get pod + ``` + + Example output: + + ```bash + nebula-storaged-0 1/1 Running 0 43h + ``` + +2. Check the PVC and PV information for the Storage service: + + ```bash + # View PVC + kubectl get pvc + ``` + + Example output: + + ```bash + storaged-data-nebula-storaged-0 Bound pvc-36ca3871-9265-460f-b812-7e73a718xxxx 5Gi RWO ebs-sc 43h + ``` + + ```bash + # View PV and confirm that the capacity of the PV is 5Gi + kubectl get pv + ``` + + Example output: + + ```bash + pvc-36ca3871-9265-460f-b812-xxx 5Gi RWO Delete Bound default/storaged-data-nebula-storaged-0 ebs-sc 43h + ``` + +3. Assuming all the above-mentioned prerequisites are met, use the following command to request an expansion of the PV for the Storage service to 10Gi: + + ```bash + kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"dataVolumeClaims":[{"resources": {"requests": {"storage": "10Gi"}}, "storageClassName": "ebs-sc"}]}}}' + ``` + + Example output: + + ```bash + nebulacluster.apps.nebula-graph.io/nebula patched + ``` + +4. After waiting for about a minute, check the expanded PVC and PV information: + + ```bash + kubectl get pvc + ``` + + Example output: + + ```bash + storaged-data-nebula-storaged-0 Bound pvc-36ca3871-9265-460f-b812-7e73a718xxxx 10Gi RWO ebs-sc 43h + ``` + + ```bash + kubectl get pv + ``` + + Example output: + + ```bash + pvc-36ca3871-9265-460f-b812-xxx 10Gi RWO Delete Bound default/storaged-data-nebula-storaged-0 ebs-sc 43h + ``` + + As you can see, both the PVC and PV capacity have been expanded to 10Gi. \ No newline at end of file diff --git a/docs-2.0-zh/nebula-operator/1.introduction-to-nebula-operator.md b/docs-2.0-zh/nebula-operator/1.introduction-to-nebula-operator.md index 674d59d11f4..bf958948ae8 100644 --- a/docs-2.0-zh/nebula-operator/1.introduction-to-nebula-operator.md +++ b/docs-2.0-zh/nebula-operator/1.introduction-to-nebula-operator.md @@ -36,7 +36,7 @@ NebulaGraph Operator 不支持 v1.x 版本的 NebulaGraph，其与{{nebula.name} | {{nebula.name}}版本 | NebulaGraph Operator 版本 | | ------------------- | ------------------------- | -| 3.5.x ~ 3.6.0 | 1.5.0 、1.6.x | +| 3.5.x ~ 3.6.0 | 1.5.0 ~ 1.7.x | | 3.0.0 ~ 3.4.1 | 1.3.0、1.4.0 ~ 1.4.2 | | 3.0.0 ~ 3.3.x | 1.0.0、1.1.0、1.2.0 | | 2.5.x ~ 2.6.x | 0.9.0 | diff --git a/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/8.3.balance-data-when-scaling-storage.md b/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/8.3.balance-data-when-scaling-storage.md deleted file mode 100644 index b32b30c9d2f..00000000000 --- a/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/8.3.balance-data-when-scaling-storage.md +++ /dev/null @@ -1,114 +0,0 @@ -# 均衡扩容后的 Storage 数据 - -!!! enterpriseonly - - 仅企业版{{nebula.name}}集群支持均衡扩容后的 Storage 数据。 - -用户在扩容 Storage 服务后，可以自主决定是否均衡 Storage 服务中的数据。本文介绍如何设置均衡 Storage 数据。 - -{{nebula.name}}的 Storage 服务的扩容分为两个阶段，第一个阶段等待所有扩容的 Pods 为`Ready`的状态；第二个阶段执行`BALANCE DATA`和`BALANCE LEADER`命令均衡数据。这两个阶段将控制器副本的扩容过程与均衡数据过程解耦，使用户可以选择在低流量时进行均衡数据操作，有效降低数据迁移对在线服务的影响。 - -用户可以在集群的 CR 实例的配置文件中通过参数`enableAutoBalance`来控制是否自动均衡数据。 - -## 前提条件 - -已使用 K8s 创建一个集群。具体步骤，参见[使用 Kubectl 创建{{nebula.name}}集群](../3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md)。 - -## 操作步骤 - -以下示例使用名为`nebula`的集群、名为`nebula_cluster.yaml`的 YAML 配置文件，说明如何设置`enableAutoBalance`： - -1. 执行以下命令进入`nebula`集群的编辑页面。 - - ```bash - kubectl edit nebulaclusters.apps.nebula-graph.io nebula - ``` - -2. 在 YAML 文件的`spec.storaged`配置项中，添加`enableAutoBalance`并设置其值为`true`。 - - ```yaml - apiVersion: apps.nebula-graph.io/v1alpha1 - kind: NebulaCluster - metadata: - name: nebula - spec: - graphd: - image: vesoft/nebula-graphd - logVolumeClaim: - resources: - requests: - storage: 2Gi - storageClassName: fast-disks - replicas: 1 - resources: - limits: - cpu: "1" - memory: 1Gi - requests: - cpu: 500m - memory: 500Mi - version: {{nebula.tag}} - imagePullPolicy: IfNotPresent - metad: - dataVolumeClaim: - resources: - requests: - storage: 2Gi - storageClassName: fast-disks - image: vesoft/nebula-metad - logVolumeClaim: - resources: - requests: - storage: 2Gi - storageClassName: fast-disks - replicas: 1 - resources: - limits: - cpu: "1" - memory: 1Gi - requests: - cpu: 500m - memory: 500Mi - version: {{nebula.tag}} - nodeSelector: - nebula: cloud - reference: - name: statefulsets.apps - version: v1 - schedulerName: default-scheduler - storaged: - enableAutoBalance: true //将其值设置为 true 时表示扩容后自动均衡 Storage 数据。 - dataVolumeClaims: - - resources: - requests: - storage: 2Gi - storageClassName: fast-disks - - resources: - requests: - storage: 2Gi - storageClassName: fast-disks - image: vesoft/nebula-storaged - logVolumeClaim: - resources: - requests: - storage: 2Gi - storageClassName: fast-disks - replicas: 3 - resources: - limits: - cpu: "1" - memory: 1Gi - requests: - cpu: 500m - memory: 500Mi - version: {{nebula.tag}} - ... - ``` - - - 当设置`enableAutoBalance`的值为`true`时，表示扩容后自动均衡 Storage 数据。 - - - 当设置`enableAutoBalance`的值为`false`时，表示扩容后不会均衡 Storage 数据。 - - - 当不设置`enableAutoBalance`参数时，默认扩容 Storage 后系统不会自动均衡 Storage 数据。 - -3. 执行`kubectl apply -f nebula_cluster.yaml`使上述更新生效。 \ No newline at end of file diff --git a/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md b/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md new file mode 100644 index 00000000000..2b5d916831b --- /dev/null +++ b/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md @@ -0,0 +1,103 @@ +# 开启准入控制 + +K8s 的[准入控制（Admission Control）](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/)是一种安全机制，并在运行时作为 Webhook 运行。通过准入 Webhook 对请求进行拦截和修改，从而保证集群的安全性。准入 Webhook 操作包括验证（Validating）和变更（Mutating）两类。NebulaGraph Operator 仅支持验证操作，并提供一些默认的准入控制规则。本文介绍 NebulaGraph Operator 的默认准入控制规则及如何开启准入控制。 + +## 前提条件 + +已使用 K8s 创建一个集群。具体步骤，参见[使用 Kubectl 创建{{nebula.name}}集群](../3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md)。 + +## 准入控制规则 + +K8s 的准入控制允许用户在 Kubernetes API Server 处理请求之前，插入自定义的逻辑或策略。这种机制可以用于实现一些安全策略，比如限制 Pod 的资源使用量，或者限制 Pod 的访问权限等。NebulaGraph Operator 仅支持验证操作，即对请求进行验证和拦截，不支持对请求进行变更操作。NebulaGraph Operator 默认的准入验证控制规则包括： + +- 满足高可用模式下的最小副本数： + + - Graph 服务：至少需要 2 个副本。 + - Meta 服务：至少需要 3 个副本。 + - Storage 服务：至少需要 3 个副本。 + + !!! note + + 高可用模式是指{{nebula.name}}集群服务的高可用。Storage 服务和 Meta 服务是有状态的服务，其副本数据通过 [Raft](../../1.introduction/3.nebula-graph-architecture/4.storage-service.md#raft) 协议保持一致性且副本数量不能为偶数。因此，高可用模式下，至少需要 3 个 Storage 服务和 3 个 Meta 服务。Graph 服务为无状态的服务，因此其副本数量可以为偶数，但至少需要 2 个副本。 + +- 禁止通过`dataVolumeClaims`为 Storage 服务追加额外的 PV。 + +- 禁止缩小所有服务的 PVC 的容量，但是可以扩容。 + +- 禁止在 Storage 服务扩缩容期间，进行任何二次操作。 + +## 为准入 Webhook 创建证书 + +为了确保通信的安全性和数据的完整性，K8s 的 API server 和准入 Webhook 之间的通信默认通过 HTTPS 协议进行，因此使用准入控制还需要为准入 Webhook 提供 TLS 证书。[cert-manager](https://cert-manager.io/docs/) 是一个 K8s 的证书管理控制器，可以自动化证书的签发和更新。NebulaGraph Operator 使用 cert-manager 来管理证书。 + +当 cert-manager 安装完成并且开启准入控制时，NebulaGraph Operator 会自动创建一个 [Issuer](https://cert-manager.io/docs/concepts/issuer/)，用于签发准入 Webhook 所需的证书，同时会创建一个 [Certificate](https://cert-manager.io/docs/concepts/certificate/)，用于存储签发的证书。签发的证书被存储在 +`nebula-operator-webhook-secret`的 Secret 中。 + +## 开启准入控制 + +1. 安装部署 cert-manager。 + + ```bash + kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.1/cert-manager.yaml + ``` + + 建议部署最新版本 cert-manager。详情参见 [cert-manager 官方文档](https://cert-manager.io/docs/installation/)。 + +2. 修改 NebulaGraph Operator 的配置文件，开启准入控制。默认准入控制是关闭的，需要手动开启。 + + ```bash + # 查看当前的配置 + helm show values nebula-operator/nebula-operator + ``` + + ```bash + # 修改配置，将`enableAdmissionWebhook`设置为`true`。 + helm upgrade nebula-operator nebula-operator/nebula-operator --set enableAdmissionWebhook=true + ``` + + !!! note + + `nebula-operator`为 chart 所在仓库的名称，`nebula-operator/nebula-operator`为 chart 的名称。如果没有指定 chart 的命名空间，默认为`default`。 + +3. 查看准入 Webhook 的证书 Secret。 + + ```bash + kubectl get secret nebula-operator-webhook-secret -o yaml + ``` + + 如果输出的结果中包含证书内容，则表示准入 Webhook 的证书已经创建成功。 + +4. 验证控制规则。 + + - 验证高可用模式下的最小副本数。 + + ```bash + # 标注集群为高可用模式 + $ kubectl annotate nc nebula nebula-graph.io/ha-mode=true + # 验证 Graph 服务的最小副本数 + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"graphd": {"replicas":1}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: spec.graphd.replicas: Invalid value: 1: should be at least 2 in HA mode + ``` + + - 验证禁止通过`dataVolumeClaims`为 Storage 服务追加额外的 PV。 + + ```bash + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"dataVolumeClaims":[{"resources": {"requests": {"storage": "2Gi"}}, "storageClassName": "local-path"},{"resources": {"requests": {"storage": "3Gi"}}, "storageClassName": "fask-disks"}]}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: spec.storaged.dataVolumeClaims: Forbidden: storaged dataVolumeClaims is immutable + ``` + + - 验证禁止缩小 Storage 服务的 PVC 的容量。 + + ```bash + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"dataVolumeClaims":[{"resources": {"requests": {"storage": "1Gi"}}, "storageClassName": "fast-disks"}]}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: spec.storaged.dataVolumeClaims: Invalid value: resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}: data volume size can only be increased + ``` + + - 验证禁止在 Storage 服务缩容期间，进行任何二次操作。 + + ```bash + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"replicas": 5}}}' + nebulacluster.apps.nebula-graph.io/nebula patched + $ kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"replicas": 3}}}' + Error from server: admission webhook "nebulaclustervalidating.nebula-graph.io" denied the request: [spec.storaged: Forbidden: field is immutable while in ScaleOut phase, spec.storaged.replicas: Invalid value: 3: field is immutable while not in Running phase] + ``` \ No newline at end of file diff --git a/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/8.2.pv-reclaim.md b/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/storage/8.2.pv-reclaim.md similarity index 94% rename from docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/8.2.pv-reclaim.md rename to docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/storage/8.2.pv-reclaim.md index fec5bd24c81..52a2201a0d4 100644 --- a/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/8.2.pv-reclaim.md +++ b/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/storage/8.2.pv-reclaim.md @@ -4,14 +4,9 @@ NebulaGraph Operator 使用持久化卷 PV（Persistent Volume）和持久化卷 用户也可以在集群实例的配置文件中通过设置参数`spec.enablePVReclaim`为`true`来定义在删除集群后自动删除 PVC 以释放数据。至于在删除 PVC 后是否删除 PV，用户需要自定义 PV 的回收策略。参见 [StorageClass 中设置 reclaimPolicy](https://kubernetes.io/docs/concepts/storage/storage-classes/#reclaim-policy) 和 [PV Reclaiming](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#reclaiming) 了解 PV 回收策略。 - ## 前提条件 -已使用 K8s 创建一个集群。具体步骤，参见[使用 Kubectl 创建{{nebula.name}}集群](../3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md)。 - -## 注意事项 - -不支持在集群运行时动态增加或挂载 PV。 +已使用 K8s 创建一个集群。具体步骤，参见[使用 Kubectl 创建{{nebula.name}}集群](../../3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md)。 ## 操作步骤 diff --git a/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/storage/8.6.dynamic-expand-pv.md b/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/storage/8.6.dynamic-expand-pv.md new file mode 100644 index 00000000000..4e8e1403b54 --- /dev/null +++ b/docs-2.0-zh/nebula-operator/8.custom-cluster-configurations/storage/8.6.dynamic-expand-pv.md @@ -0,0 +1,95 @@ +# 动态在线扩容存储卷 + +在 K8s 环境中，{{nebula.name}}的数据存储在持久化存储卷（PV）上。动态在线扩容存储卷指的是在不停机的情况下增加存储卷的容量，以满足{{nebula.name}}数据增长的需求。本文介绍如何在 K8s 环境中为{{nebula.name}}的服务动态在线扩容存储卷。 + +!!! note + + - 集群创建后，不支持在集群运行时动态增加 PV 的数量。 + - 本文介绍的方法仅使用在线扩容存储卷，不支持在线缩容存储卷。 + +## 背景信息 + +K8s 中，StorageClass 是定义了一种存储类型的资源，它描述了一种存储的类，包括存储的提供者（provisioner）、参数和其他细节。当创建一个 PersistentVolumeClaim（PVC）并指定一个 StorageClass 时，K8s 会自动创建一个对应的 PV。动态扩容存储卷的原理是编辑 PVC 并增加存储卷的容量，然后 K8s 会根据 PVC 中指定的`storageClassName`自动扩容该 PVC 对应的 PV 的容量。在这个过程中，不会创建新的 PV，而是改变现有的 PV 的容量大小。只有动态存储卷才支持存储卷的动态扩容，即配置了`storageClassName`的 PVC。同时 StorageClass 的`allowVolumeExpansion`字段必须为`true`。详情参见 [Expanding Persistent Volumes Claims](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims)。 + +在 Operator 中，不能直接编辑 PVC，因为 Operator 会根据{{nebula.name}}集群服务的配置`spec..dataVolumeClaim`自动创建 PVC。因此，需要通过修改集群的配置来实现 PVC 的配置更新，然后自动触发 PV 的动态在线扩容。 + +## 前提条件 + +- K8s 的版本等于或高于 1.18。 +- 已在 K8s 环境中创建 StorageClass。详情参见 [Expanding Persistent Volumes Claims](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims)。 + - 确保 StorageClass 配置了`allowVolumeExpansion`字段并且值为`true`。 + - 确保 StorageClass 配置的`provisioner`支持动态扩容。 +- 在 K8s 中创建一个{{nebula.name}}集群。具体步骤，参见[使用 Kubectl 创建{{nebula.name}}集群](../../3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md)。 +- {{nebula.name}}集群 Pod 处于运行状态。 + +## 在线扩容存储卷示例 + +以下示例假设 StorageClass 的名称为`ebs-sc`，{{nebula.name}}集群的名称为`nebula`，演示如何在线扩容 Storage 服务的存储卷。 + +1. 查看 Storage 服务 Pod 的状态。 + + ```bash + kubectl get pod + ``` + + 示例输出： + ```bash + nebula-storaged-0 1/1 Running 0 43h + ``` + + +2. 查看 Storage 服务的 PVC 和 PV 信息。 + + ```bash + # 查看 PVC 信息 + kubectl get pvc + ``` + + 示例输出： + ```bash + storaged-data-nebula-storaged-0 Bound pvc-36ca3871-9265-460f-b812-7e73a718xxxx 5Gi RWO ebs-sc 43h + ``` + + ```bash + # 查看 PV 信息，确认 PV 的容量为 5Gi + kubectl get pv + ``` + + 示例输出： + + ```bash + pvc-36ca3871-9265-460f-b812-xxx 5Gi RWO Delete Bound default/storaged-data-nebula-storaged-0 ebs-sc 43h + ``` + +3. 在符合前提条件的情况下，执行以下命令请求扩容 Storage 服务的存储卷至 10Gi。 + + ```bash + kubectl patch nc nebula --type='merge' --patch '{"spec": {"storaged": {"dataVolumeClaims":[{"resources": {"requests": {"storage": "10Gi"}}, "storageClassName": "ebs-sc"}]}}}' + ``` + + 示例输出： + ```bash + nebulacluster.apps.nebula-graph.io/nebula patched + ``` + +4. 等待一分钟左右查看扩容后的 PVC 和 PV 信息。 + + ```bash + kubectl get pvc + ``` + + 示例输出： + ```bash + storaged-data-nebula-storaged-0 Bound pvc-36ca3871-9265-460f-b812-7e73a718xxxx 10Gi RWO ebs-sc 43h + ``` + + ```bash + kubectl get pv + ``` + + 示例输出： + ```bash + pvc-36ca3871-9265-460f-b812-xxx 10Gi RWO Delete Bound default/storaged-data-nebula-storaged-0 ebs-sc 43h + ``` + + 由上可见，PVC 和 PV 的容量都已扩容至 10Gi。 \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 403654c51db..aa836529344 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -244,9 +244,9 @@ extra: branch: release-1.2 tag: v1.2.0 operator: - release: 1.5.0 - tag: v1.5.0 - branch: release-1.5 + release: 1.7.1 + tag: v1.7.1 + branch: release-1.7 upgrade_from: 3.5.0 upgrade_to: 3.6.0 exporter: @@ -575,19 +575,19 @@ nav: - What is NebulaGraph Operator: nebula-operator/1.introduction-to-nebula-operator.md - Overview of using NebulaGraph Operator: nebula-operator/6.get-started-with-operator.md - Deploy NebulaGraph Operator: nebula-operator/2.deploy-nebula-operator.md - - Deploy clusters: -#ent - Deploy LM: nebula-operator/3.deploy-nebula-graph-cluster/3.0.deploy-lm.md + - Deploy clusters: - Deploy clusters with Kubectl: nebula-operator/3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md - Deploy clusters with Helm: nebula-operator/3.deploy-nebula-graph-cluster/3.2create-cluster-with-helm.md - Connect to NebulaGraph databases: nebula-operator/4.connect-to-nebula-graph-service.md - Configure clusters: - Custom configuration parameters for a NebulaGraph cluster: nebula-operator/8.custom-cluster-configurations/8.1.custom-conf-parameter.md - - Reclaim PVs: nebula-operator/8.custom-cluster-configurations/8.2.pv-reclaim.md -#ent - Balance storage data after scaling out: nebula-operator/8.custom-cluster-configurations/8.3.balance-data-when-scaling-storage.md + - Storage: + - Dynamically expand persistent volumes: nebula-operator/8.custom-cluster-configurations/storage/8.6.dynamic-expand-pv.md + - Reclaim PVs: nebula-operator/8.custom-cluster-configurations/storage/8.2.pv-reclaim.md - Manage cluster logs: nebula-operator/8.custom-cluster-configurations/8.4.manage-running-logs.md + - Enable admission control: nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md - Upgrade NebulaGraph clusters: nebula-operator/9.upgrade-nebula-cluster.md - Specify a rolling update strategy: nebula-operator/11.rolling-update-strategy.md -#ent - Backup and restore: nebula-operator/10.backup-restore-using-operator.md - Self-healing: nebula-operator/5.operator-failover.md - FAQ: nebula-operator/7.operator-faq.md @@ -927,10 +927,13 @@ nav: - 连接 NebulaGraph: nebula-operator/4.connect-to-nebula-graph-service.md - 配置 NebulaGraph: - 自定义 NebulaGraph 集群的配置参数: nebula-operator/8.custom-cluster-configurations/8.1.custom-conf-parameter.md - - 回收 PV: nebula-operator/8.custom-cluster-configurations/8.2.pv-reclaim.md + - 存储: + - 动态在线扩容存储卷: nebula-operator/8.custom-cluster-configurations/storage/8.6.dynamic-expand-pv.md + - 回收 PV: nebula-operator/8.custom-cluster-configurations/storage/8.2.pv-reclaim.md - 管理集群日志: nebula-operator/8.custom-cluster-configurations/8.4.manage-running-logs.md - - 配置滚动更新策略: nebula-operator/11.rolling-update-strategy.md + - 开启准入控制: nebula-operator/8.custom-cluster-configurations/8.6.admission-control.md - 升级 NebulaGraph: nebula-operator/9.upgrade-nebula-cluster.md + - 配置滚动更新策略: nebula-operator/11.rolling-update-strategy.md - 故障自愈: nebula-operator/5.operator-failover.md - 常见问题: nebula-operator/7.operator-faq.md