Skip to content

Commit

Permalink
[config](checksum) Disable consistency checker by default (#9699)
Browse files Browse the repository at this point in the history
Disable by default because current checksum logic has some bugs.
And it will also bring some overhead.
  • Loading branch information
morningman authored May 22, 2022
1 parent ad4da4a commit d270f4f
Show file tree
Hide file tree
Showing 4 changed files with 22 additions and 9 deletions.
10 changes: 7 additions & 3 deletions docs/en/admin-manual/config/fe-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -1017,17 +1017,21 @@ IsMutable:true

MasterOnly:true

Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*. Default is from 23:00 to 04:00
Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*.

If the two times are the same, no consistency check will be triggered.

### consistency_check_end_time

Default:04
Default:23

IsMutable:true

MasterOnly:true

Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*. Default is from 23:00 to 04:00
Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*.

If the two times are the same, no consistency check will be triggered.

### export_tablet_num_per_task

Expand Down
10 changes: 7 additions & 3 deletions docs/zh-CN/admin-manual/config/fe-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -1017,19 +1017,23 @@ colocote join PlanFragment instance 的 memory_limit = exec_mem_limit / min (que

一致性检查开始时间

一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`。 默认为 23:00 至 04:00
一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`

如果两个时间相同,则不会触发一致性检查。

### `consistency_check_end_time`

默认值:04
默认值:23

是否可以动态配置:true

是否为 Master FE 节点独有的配置项:true

一致性检查结束时间

一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`。 默认为 23:00 至 04:00
一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`

如果两个时间相同,则不会触发一致性检查。

### `export_tablet_num_per_task`

Expand Down
7 changes: 5 additions & 2 deletions fe/fe-core/src/main/java/org/apache/doris/common/Config.java
Original file line number Diff line number Diff line change
Expand Up @@ -856,12 +856,15 @@ public class Config extends ConfigBase {
// Configurations for consistency check
/**
* Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*.
* Default is from 23:00 to 04:00
* If start time == end time, the checker will stop scheduling.
* And default is disabled.
* TODO(cmy): Disable by default because current checksum logic has some bugs.
* And it will also bring some overhead.
*/
@ConfField(mutable = true, masterOnly = true)
public static String consistency_check_start_time = "23";
@ConfField(mutable = true, masterOnly = true)
public static String consistency_check_end_time = "4";
public static String consistency_check_end_time = "23";
/**
* Default timeout of a single consistency check task. Set long enough to fit your tablet size.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@
import com.google.common.collect.Maps;
import com.google.common.collect.Multiset;
import com.google.common.collect.Sets;

import org.apache.commons.collections.map.HashedMap;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
Expand Down Expand Up @@ -640,7 +641,8 @@ private void sendFragment() throws TException, RpcException, UserException {
switch (code) {
case TIMEOUT:
throw new RpcException(pair.first.backend.getHost(), "send fragment timeout. backend id: "
+ pair.first.backend.getId());
+ pair.first.backend.getId() + " fragment: " +
DebugUtil.printId(pair.first.rpcParams.params.fragment_instance_id));
case THRIFT_RPC_ERROR:
SimpleScheduler.addToBlacklist(pair.first.backend.getId(), errMsg);
throw new RpcException(pair.first.backend.getHost(), "rpc failed");
Expand Down

0 comments on commit d270f4f

Please sign in to comment.