fix: add a threshold for compact block

[zhyass]

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

Add a threshold 2 for compact blocks, if the selected blocks are less than 2*max_threads, compact is skipped.

Add a limit to batch compaction.

optimize table [database.]table_name compact [segment] [limit segment_count];

Fixes #8316
Fixes #8241

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Updated
databend	⬜️ Ignored (Inspect)		Oct 22, 2022 at 4:58PM (UTC)

[BohuTANG]

Unit test failed: test_fuse_snapshot_optimize_compact

[BohuTANG]

There are many tests failed and need to address.

[BohuTANG]

Some sqllogic tests related to compact still need to modify.

[BohuTANG]

@soyeric128 for the new option:
optimize table <table> compact [segment] [limit <segment count>];