Skip to content

Commit

Permalink
[docs]update resource_tags note (#1367)
Browse files Browse the repository at this point in the history
# Versions 

- [x] dev
- [x] 3.0
- [x] 2.1
- [x] 2.0

# Languages

- [x] Chinese
- [x] English
  • Loading branch information
wangbo authored Nov 16, 2024
1 parent 948498e commit 5d896fb
Show file tree
Hide file tree
Showing 9 changed files with 36 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,10 @@ FE 不参与用户数据的处理计算等工作,因此是一个资源消耗

> 注:默认情况下,用户的 `resource_tags.location` 属性为空,在 2.0.2(含)之前的版本中,默认情况下,用户不受 tag 的限制,可以使用任意资源组。在 2.0.3 版本之后,默认情况下,普通用户只能使用 `default` 资源组。root 和 admin 用户可以使用任意资源组。
:::tip
需要注意的是,属性 `resource_tags.location` 每次修改完成之后,用户需要重新建立连接才能使变更生效。
:::

这样,我们通过对节点的划分,以及对用户的资源使用限制,实现了不同用户查询上的物理资源隔离。更进一步,我们可以给不同的业务部门创建不同的用户,并限制每个用户使用不同的资源组。以避免不同业务部分之间使用资源干扰。比如集群内有一张业务表需要共享给所有 9 个业务部门使用,但是希望能够尽量避免不同部门之间的资源抢占。则我们可以为这张表创建 3 个副本,分别存储在 3 个资源组中。接下来,我们为 9 个业务部门创建 9 个用户,每 3 个用户限制使用一个资源组。这样,资源的竞争程度就由 9 降低到了 3。

另一方面,针对在线和离线任务的隔离。我们可以利用资源组的方式实现。比如我们可以将节点划分为 Online 和 Offline 两个资源组。表数据依然以 3 副本的方式存储,其中 2 个副本存放在 Online 资源组,1 个副本存放在 Offline 资源组。Online 资源组主要用于高并发低延迟的在线数据服务,而一些大查询或离线 ETL 操作,则可以使用 Offline 资源组中的节点执行。从而实现在统一集群内同时提供在线和离线服务的能力。
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,10 @@ FE 不参与用户数据的处理计算等工作,因此是一个资源消耗
set property for 'user3' 'resource_tags.location' = 'group_a, group_b, group_c';
```

:::tip
需要注意的是,属性 `resource_tags.location` 每次修改完成之后,用户需要重新建立连接才能使变更生效。
:::

设置完成后,user1 在发起对 UserTable 表的查询时,只会访问 `group_a` 资源组内节点上的数据副本,并且查询仅会使用 `group_a` 资源组内的节点计算资源。而 user3 的查询可以使用任意资源组内的副本和计算资源。

这样,我们通过对节点的划分,以及对用户的资源使用限制,实现了不同用户查询上的物理资源隔离。更进一步,我们可以给不同的业务部门创建不同的用户,并限制每个用户使用不同的资源组。以避免不同业务部分之间使用资源干扰。比如集群内有一张业务表需要共享给所有 9 个业务部门使用,但是希望能够尽量避免不同部门之间的资源抢占。则我们可以为这张表创建 3 个副本,分别存储在 3 个资源组中。接下来,我们为 9 个业务部门创建 9 个用户,每 3 个用户限制使用一个资源组。这样,资源的竞争程度就由 9 降低到了 3。
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,10 @@ set property for 'user3' 'resource_tags.location' = 'group_a, group_b, group_c';
注:默认情况下,用户的 `resource_tags.location` 属性为空,在 2.0.2(含)之前的版本中,默认情况下,用户不受 tag 的限制,可以使用任意资源组。在 2.0.3 版本之后,默认情况下,普通用户只能使用 `default` 资源组。root 和 admin 用户可以使用任意资源组。
:::

:::tip
需要注意的是,属性 `resource_tags.location` 每次修改完成之后,用户需要重新建立连接才能使变更生效。
:::

这样,我们通过对节点的划分,以及对用户的资源使用限制,实现了不同用户查询上的物理资源隔离。更进一步,我们可以给不同的业务部门创建不同的用户,并限制每个用户使用不同的资源组。以避免不同业务部分之间使用资源干扰。比如集群内有一张业务表需要共享给所有 9 个业务部门使用,但是希望能够尽量避免不同部门之间的资源抢占。则我们可以为这张表创建 3 个副本,分别存储在 3 个资源组中。接下来,我们为 9 个业务部门创建 9 个用户,每 3 个用户限制使用一个资源组。这样,资源的竞争程度就由 9 降低到了 3。

另一方面,针对在线和离线任务的隔离。我们可以利用资源组的方式实现。比如我们可以将节点划分为 Online 和 Offline 两个资源组。表数据依然以 3 副本的方式存储,其中 2 个副本存放在 Online 资源组,1 个副本存放在 Offline 资源组。Online 资源组主要用于高并发低延迟的在线数据服务,而一些大查询或离线 ETL 操作,则可以使用 Offline 资源组中的节点执行。从而实现在统一集群内同时提供在线和离线服务的能力。
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,10 @@ FE 不参与用户数据的处理计算等工作,因此是一个资源消耗

> 注:默认情况下,用户的 `resource_tags.location` 属性为空,在 2.0.2(含)之前的版本中,默认情况下,用户不受 tag 的限制,可以使用任意资源组。在 2.0.3 版本之后,默认情况下,普通用户只能使用 `default` 资源组。root 和 admin 用户可以使用任意资源组。
:::tip
需要注意的是,属性 `resource_tags.location` 每次修改完成之后,用户需要重新建立连接才能使变更生效。
:::

这样,我们通过对节点的划分,以及对用户的资源使用限制,实现了不同用户查询上的物理资源隔离。更进一步,我们可以给不同的业务部门创建不同的用户,并限制每个用户使用不同的资源组。以避免不同业务部分之间使用资源干扰。比如集群内有一张业务表需要共享给所有 9 个业务部门使用,但是希望能够尽量避免不同部门之间的资源抢占。则我们可以为这张表创建 3 个副本,分别存储在 3 个资源组中。接下来,我们为 9 个业务部门创建 9 个用户,每 3 个用户限制使用一个资源组。这样,资源的竞争程度就由 9 降低到了 3。

另一方面,针对在线和离线任务的隔离。我们可以利用资源组的方式实现。比如我们可以将节点划分为 Online 和 Offline 两个资源组。表数据依然以 3 副本的方式存储,其中 2 个副本存放在 Online 资源组,1 个副本存放在 Offline 资源组。Online 资源组主要用于高并发低延迟的在线数据服务,而一些大查询或离线 ETL 操作,则可以使用 Offline 资源组中的节点执行。从而实现在统一集群内同时提供在线和离线服务的能力。
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,10 @@ FE 不参与用户数据的处理计算等工作,因此是一个资源消耗

> 注:默认情况下,用户的 `resource_tags.location` 属性为空,在 2.0.2(含)之前的版本中,默认情况下,用户不受 tag 的限制,可以使用任意资源组。在 2.0.3 版本之后,默认情况下,普通用户只能使用 `default` 资源组。root 和 admin 用户可以使用任意资源组。
:::tip
需要注意的是,属性 `resource_tags.location` 每次修改完成之后,用户需要重新建立连接才能使变更生效。
:::

这样,我们通过对节点的划分,以及对用户的资源使用限制,实现了不同用户查询上的物理资源隔离。更进一步,我们可以给不同的业务部门创建不同的用户,并限制每个用户使用不同的资源组。以避免不同业务部分之间使用资源干扰。比如集群内有一张业务表需要共享给所有 9 个业务部门使用,但是希望能够尽量避免不同部门之间的资源抢占。则我们可以为这张表创建 3 个副本,分别存储在 3 个资源组中。接下来,我们为 9 个业务部门创建 9 个用户,每 3 个用户限制使用一个资源组。这样,资源的竞争程度就由 9 降低到了 3。

另一方面,针对在线和离线任务的隔离。我们可以利用资源组的方式实现。比如我们可以将节点划分为 Online 和 Offline 两个资源组。表数据依然以 3 副本的方式存储,其中 2 个副本存放在 Online 资源组,1 个副本存放在 Offline 资源组。Online 资源组主要用于高并发低延迟的在线数据服务,而一些大查询或离线 ETL 操作,则可以使用 Offline 资源组中的节点执行。从而实现在统一集群内同时提供在线和离线服务的能力。
Expand Down
4 changes: 4 additions & 0 deletions versioned_docs/version-1.2/admin-manual/multi-tenant.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,10 @@ Node resource division refers to setting tags for BE nodes in a Doris cluster, a
set property for'user2''resource_tags.location' = 'group_b';
set property for'user3''resource_tags.location' = 'group_a, group_b, group_c';
```

:::tip
It should be noted that after each modification of the ```resource_tags.location``` property, the user needs to re-establish the connection for the changes to take effect.
:::

After the setting is complete, when user1 initiates a query on the UserTable table, it will only access the data copy on the nodes in the `group_a` resource group, and the query will only use the node computing resources in the `group_a` resource group. The query of user3 can use copies and computing resources in any resource group.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,10 @@ Node resource division refers to setting tags for BE nodes in a Doris cluster, a
By default, the user's `resource_tags.location` attribute is empty. In versions prior to 2.0.2 (inclusive), by default, users are not restricted by tags and can use any resource group. After version 2.0.3, normal users can only use the `default` resource group by default. Root and Admin user can use any resource group.
:::
:::tip
It should be noted that after each modification of the ```resource_tags.location``` property, the user needs to re-establish the connection for the changes to take effect.
:::
In this way, we have achieved physical resource isolation for different user queries by dividing nodes and restricting user resource usage. Furthermore, we can create different users for different business departments and restrict each user from using different resource groups. In order to avoid the use of resource interference between different business parts. For example, there is a business table in the cluster that needs to be shared by all 9 business departments, but it is hoped that resource preemption between different departments can be avoided as much as possible. Then we can create 3 copies of this table and store them in 3 resource groups. Next, we create 9 users for 9 business departments, and limit the use of one resource group for every 3 users. In this way, the degree of competition for resources is reduced from 9 to 3.
On the other hand, for the isolation of online and offline tasks. We can use resource groups to achieve this. For example, we can divide nodes into two resource groups, Online and Offline. The table data is still stored in 3 copies, of which 2 copies are stored in the Online resource group, and 1 copy is stored in the Offline resource group. The Online resource group is mainly used for online data services with high concurrency and low latency. Some large queries or offline ETL operations can be executed using nodes in the Offline resource group. So as to realize the ability to provide online and offline services simultaneously in a unified cluster.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,10 @@ Node resource division refers to setting tags for BE nodes in a Doris cluster, a

> Note: By default, the user's `resource_tags.location` attribute is empty. In versions prior to 2.0.2 (inclusive), by default, users are not restricted by tags and can use any resource group. After version 2.0.3, normal users can only use the `default` resource group by default. Root and Admin user can use any resource group.
:::tip
It should be noted that after each modification of the ```resource_tags.location``` property, the user needs to re-establish the connection for the changes to take effect.
:::
In this way, we have achieved physical resource isolation for different user queries by dividing nodes and restricting user resource usage. Furthermore, we can create different users for different business departments and restrict each user from using different resource groups. In order to avoid the use of resource interference between different business parts. For example, there is a business table in the cluster that needs to be shared by all 9 business departments, but it is hoped that resource preemption between different departments can be avoided as much as possible. Then we can create 3 copies of this table and store them in 3 resource groups. Next, we create 9 users for 9 business departments, and limit the use of one resource group for every 3 users. In this way, the degree of competition for resources is reduced from 9 to 3.
On the other hand, for the isolation of online and offline tasks. We can use resource groups to achieve this. For example, we can divide nodes into two resource groups, Online and Offline. The table data is still stored in 3 copies, of which 2 copies are stored in the Online resource group, and 1 copy is stored in the Offline resource group. The Online resource group is mainly used for online data services with high concurrency and low latency. Some large queries or offline ETL operations can be executed using nodes in the Offline resource group. So as to realize the ability to provide online and offline services simultaneously in a unified cluster.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,10 @@ Node resource division refers to setting tags for BE nodes in a Doris cluster, a

> Note: By default, the user's `resource_tags.location` attribute is empty. In versions prior to 2.0.2 (inclusive), by default, users are not restricted by tags and can use any resource group. After version 2.0.3, normal users can only use the `default` resource group by default. Root and Admin user can use any resource group.
:::tip
It should be noted that after each modification of the ```resource_tags.location``` property, the user needs to re-establish the connection for the changes to take effect.
:::
In this way, we have achieved physical resource isolation for different user queries by dividing nodes and restricting user resource usage. Furthermore, we can create different users for different business departments and restrict each user from using different resource groups. In order to avoid the use of resource interference between different business parts. For example, there is a business table in the cluster that needs to be shared by all 9 business departments, but it is hoped that resource preemption between different departments can be avoided as much as possible. Then we can create 3 copies of this table and store them in 3 resource groups. Next, we create 9 users for 9 business departments, and limit the use of one resource group for every 3 users. In this way, the degree of competition for resources is reduced from 9 to 3.
On the other hand, for the isolation of online and offline tasks. We can use resource groups to achieve this. For example, we can divide nodes into two resource groups, Online and Offline. The table data is still stored in 3 copies, of which 2 copies are stored in the Online resource group, and 1 copy is stored in the Offline resource group. The Online resource group is mainly used for online data services with high concurrency and low latency. Some large queries or offline ETL operations can be executed using nodes in the Offline resource group. So as to realize the ability to provide online and offline services simultaneously in a unified cluster.
Expand Down

0 comments on commit 5d896fb

Please sign in to comment.