[Bug] [stream-load] Doris 3.0.5 stream load cpu usage highly than Doris 2.1.6

[zicat]

Search before asking

• I had searched in the issues and found no similar issues.

Version

Doris 3.0.5

What's Wrong?

In Doris 3.0.5 , create a table as follow:
CREATE TABLE test_table(...) ENGINE=OLAP DUPLICATE KEY(...) PARTITION BY RANGE(...)() DISTRIBUTED BY RANDOM BUCKETS AUTO PROPERTIES ( "group_commit_interval_ms" = "20000", "group_commit_data_bytes" = "655360000" );

Using stream load api to put data to the table, the cpu is high than Doris2. In Doris 2, the avg cpu usage is 0.7% but in Doris 3 the avg cpu usage is 4%.

I try to search and the reason may be caused by the code in be/src/pipeline/exec/group_commit_scan_operator.cpp

What You Expected?

I expected the be service cpu usage is same than doris2.

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[zicat]

@Gabriel39

[felixwluo]

You can try to set the parameters group_commit_insert_threads=2 and group_commit_wait_replay_wal_finish=true in be and restart all be. It is expected that the CPU usage of be will be reduced to half of the previous one.

[dataroaring]

#49822 fixed in 3.0.6