Skip to content

Support using B/K/KB/M/MB/G/GB/T/TB/P/PB as unit in session variable exec_mem_limit #4063

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
morningman merged 3 commits into from
Jul 13, 2020
Merged

Support using B/K/KB/M/MB/G/GB/T/TB/P/PB as unit in session variable exec_mem_limit #4063

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
70254f7
xx
yangzhg Jul 10, 2020
7a01548
add docs
yangzhg Jul 10, 2020
e344bb3
add docs
yangzhg Jul 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/en/administrator-guide/variables.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ SET forward_to_master = concat('tr', 'u', 'e');

* `exec_mem_limit`

Used to set the memory limit for a single query. The default is 2GB, in bytes.
Used to set the memory limit for a single query. The default is 2GB, you can set it in B/K/KB/M/MB/G/GB/T/TB/P/PB, the default is B.

This parameter is used to limit the memory that can be used by an instance of a single query fragment in a query plan. A query plan may have multiple instances, and a BE node may execute one or more instances. Therefore, this parameter does not accurately limit the memory usage of a query across the cluster, nor does it accurately limit the memory usage of a query on a single BE node. The specific needs need to be judged according to the generated query plan.

Expand Down
2 changes: 1 addition & 1 deletion docs/zh-CN/administrator-guide/variables.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ SET forward_to_master = concat('tr', 'u', 'e');

* `exec_mem_limit`

用于设置单个查询的内存限制。默认为 2GB，单位为字节。
用于设置单个查询的内存限制。默认为 2GB，单位为B/K/KB/M/MB/G/GB/T/TB/P/PB, 默认为B。

该参数用于限制一个查询计划中，单个查询计划的实例所能使用的内存。一个查询计划可能有多个实例，一个 BE 节点可能执行一个或多个实例。所以该参数并不能准确限制一个查询在整个集群的内存使用，也不能准确限制一个查询在单一 BE 节点上的内存使用。具体需要根据生成的查询计划判断。

Expand Down
10 changes: 10 additions & 0 deletions fe/src/main/java/org/apache/doris/qe/VariableMgr.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import org.apache.doris.common.ErrorCode;
import org.apache.doris.common.ErrorReport;
import org.apache.doris.common.PatternMatcher;
import org.apache.doris.common.util.ParseUtil;
import org.apache.doris.common.util.TimeUtils;
import org.apache.doris.persist.EditLog;

Expand Down Expand Up @@ -220,6 +221,15 @@ public static void setVar(SessionVariable sessionVariable, SetVar setVar) throws
setVar.getType(), setVar.getVariable(),
new StringLiteral(TimeUtils.checkTimeZoneValidAndStandardize(setVar.getValue().getStringValue())));
}
if (setVar.getVariable().toLowerCase().equals("exec_mem_limit")) {
try {
setVar = new SetVar(
setVar.getType(), setVar.getVariable(),
new StringLiteral(Long.toString(ParseUtil.analyzeDataVolumn(setVar.getValue().getStringValue()))));
} catch (AnalysisException e) {
throw new DdlException(e.getMessage());
}
}

// To modify to default value.
VarAttr attr = ctx.getField().getAnnotation(VarAttr.class);
Expand Down