tiflash can't gc completely after set tiflash replica 0

[lilinghai]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiFlash version? (Required)

master

[breezewish]

Since we use a lot of logical split now, the estimated rows and size (which is based on pack statistics) is no longer accurate. The inaccurate rows and size misleads the GC and prevented DTFiles being recycled.

We also need to continuously observe whether there are other problems when the estimated rows and size is very inaccurate.

There are two possible solution directions:

A. Make estimated rows and size more accurate.
B. Always use the real rows and size to drive segment GC.

[JaySon-Huang]

An example is that a segment with range [27192,30593), while the DMFile in stable contains two packs. The first pack contains rows [16994, 23791) ∪ [33991, 35386) and the second pack contains rows [35386, 37381).

No data is valid under the segment, which is generated by logical split, but now we can not execute gc on segment like this. Now the gc routines use the min-max index for decision-making

tiflash/dbms/src/Storages/DeltaMerge/DeltaMergeStore_InternalBg.cpp

Lines 391 to 400 in 6a305d3

	bool shouldCompactStableWithTooMuchDataOutOfSegmentRange(const DMContext & context, //
	const SegmentPtr & seg,
	const SegmentSnapshotPtr & snap,
	const SegmentPtr & prev_seg,
	const SegmentPtr & next_seg,
	double invalid_data_ratio_threshold,
	const LoggerPtr & log)
	{
	auto [first_pack_included, last_pack_included] = snap->stable->isFirstAndLastPackIncludedInRange(context, seg->getRowKeyRange());
	// Do a quick check about whether the DTFile is completely included in the segment range

tiflash/dbms/src/Storages/DeltaMerge/StableValueSpace.cpp

Lines 400 to 434 in 6a305d3

	std::pair<bool, bool> StableValueSpace::Snapshot::isFirstAndLastPackIncludedInRange(const DMContext & context, const RowKeyRange & range) const
	{
	// Usually, this method will be called for some "cold" key ranges.
	// Loading the index into cache may pollute the cache and make the hot index cache invalid.
	// So don't refill the cache if the index does not exist.
	bool first_pack_included = false;
	bool last_pack_included = false;
	for (size_t i = 0; i < stable->files.size(); i++)
	{
	const auto & file = stable->files[i];
	auto filter = DMFilePackFilter::loadFrom(
	file,
	context.db_context.getGlobalContext().getMinMaxIndexCache(),
	/*set_cache_if_miss*/ false,
	{range},
	RSOperatorPtr{},
	IdSetPtr{},
	context.db_context.getFileProvider(),
	context.getReadLimiter(),
	context.tracing_id);
	const auto & use_packs = filter.getUsePacks();
	if (i == 0)
	{
	// TODO: this check may not be correct when support multiple files in a stable, let's just keep it now for simplicity
	first_pack_included = use_packs.empty() || use_packs[0];
	}
	if (i == stable->files.size() - 1)
	{
	// TODO: this check may not be correct when support multiple files in a stable, let's just keep it now for simplicity
	last_pack_included = use_packs.empty() || use_packs.back();
	}
	}
	return std::make_pair(first_pack_included, last_pack_included);
	}