[Fix](rowset) When a rowset is cooled down, it is directly deleted. This can result in data query misses in the second phase of a two-phase query.

[eldenmoon]

Proposed changes

related pr #20732

There are two reasons for moving the logic of delayed deletion from the Tablet to the StorageEngine. The first reason is to consolidate the logic and unify the delayed operations. The second reason is that delayed garbage collection during queries can cause rowsets to remain in the "stale rowsets" state, preventing the timely deletion of rowset metadata, It may cause rowset metadata too large.

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

[eldenmoon]

run buildall

[eldenmoon]

run buildall

[github-actions]

clang-tidy review says "All clean, LGTM! 👍"

[github-actions]

clang-tidy review says "All clean, LGTM! 👍"

[eldenmoon]

run buildall

[github-actions]

clang-tidy review says "All clean, LGTM! 👍"

[hello-stephen]

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 55.23 seconds
stream load tsv: 503 seconds loaded 74807831229 Bytes, about 141 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162201973 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712113957_clickbench_pr_176842.html

[hello-stephen]

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 56.99 seconds
stream load tsv: 501 seconds loaded 74807831229 Bytes, about 142 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select: 29.7 seconds inserted 10000000 Rows, about 336K ops/s
storage size: 17167713010 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712114009_clickbench_pr_176840.html

[hello-stephen]

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 49.89 seconds
stream load tsv: 450 seconds loaded 74807831229 Bytes, about 158 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 56 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 28 seconds loaded 861443392 Bytes, about 29 MB/s
insert into select: 26.1 seconds inserted 10000000 Rows, about 383K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712035601_clickbench_pr_176839.html

[hello-stephen]

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.63 seconds
stream load tsv: 452 seconds loaded 74807831229 Bytes, about 157 MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc: 56 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 28 seconds loaded 861443392 Bytes, about 29 MB/s
insert into select: 26.3 seconds inserted 10000000 Rows, about 380K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712040022_clickbench_pr_176835.html

[eldenmoon]

run buildall

[github-actions]

clang-tidy review says "All clean, LGTM! 👍"

[github-actions]

clang-tidy review says "All clean, LGTM! 👍"

[eldenmoon]

run buildall

[github-actions]

clang-tidy review says "All clean, LGTM! 👍"

[hello-stephen]

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 54.1 seconds
stream load tsv: 503 seconds loaded 74807831229 Bytes, about 141 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17167434455 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712170729_clickbench_pr_177058.html

[hello-stephen]

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 53.46 seconds
stream load tsv: 504 seconds loaded 74807831229 Bytes, about 141 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17168016719 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712170930_clickbench_pr_177059.html

[hello-stephen]

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 50.38 seconds
stream load tsv: 454 seconds loaded 74807831229 Bytes, about 157 MB/s
stream load json: 23 seconds loaded 2358488459 Bytes, about 97 MB/s
stream load orc: 56 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 29 seconds loaded 861443392 Bytes, about 28 MB/s
insert into select: 26.2 seconds inserted 10000000 Rows, about 381K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712091244_clickbench_pr_177057.html

[eldenmoon]

run buildall

[github-actions]

clang-tidy review says "All clean, LGTM! 👍"

[github-actions]

PR approved by anyone and no changes requested.

[hello-stephen]

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 54.07 seconds
stream load tsv: 506 seconds loaded 74807831229 Bytes, about 140 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.4 seconds inserted 10000000 Rows, about 340K ops/s
storage size: 17168907269 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712204407_clickbench_pr_177299.html

[eldenmoon]

run buildall

[github-actions]

clang-tidy review says "All clean, LGTM! 👍"

[hello-stephen]

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 54.45 seconds
stream load tsv: 502 seconds loaded 74807831229 Bytes, about 142 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17165354409 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712230006_clickbench_pr_177431.html

[dataroaring]

LGTM

[TsukiokaKogane]

this pr could cause unused rowset not gc, fixed in #30904