compact: Update filtered blocks list before second downsample pass

[tghartland]

Fixes #7488

If the second downsampling pass is given the same filteredMetas list as the first pass, it will create duplicates of blocks created in the first pass.

It will also not be able to do further downsampling e.g 5m->1h using blocks created in the first pass, as it will not be aware of them.

The metadata was already being synced before the second pass, but not updated into the filteredMetas list.

• I added CHANGELOG entry for this change.
• Change is not relevant to the end user.

Changes

Re-uses the same block filtering logic from the first pass, to regenerate the list of filtered blocks from freshly synced metadata before the second pass.

I haven't made any changes regarding the metrics logic that takes place after the first filtering:

thanos/cmd/thanos/compact.go

Lines 465 to 469 in 57b42d1

	for _, meta := range filteredMetas {
	resolutionLabel := meta.Thanos.ResolutionString()
	downsampleMetrics.downsamples.WithLabelValues(resolutionLabel)
	downsampleMetrics.downsampleFailures.WithLabelValues(resolutionLabel)
	}

as I'm not sure what exactly this is doing. So there may be some downsamples that happen in the second pass that aren't counted in the metrics, if that's what it's doing. Let's keep that separate to this fix.

Verification

Tested against a bucket that was creating duplicate downsampled blocks on v0.35.1.
Verified in compactor logs that no duplicate block is created on the second pass: #7488 (comment)

[yeya24]

Thanks. I think the fix makes sense to me.

[tghartland]

For this failure on the compactor tests:
https://github.com/thanos-io/thanos/actions/runs/9699030328/job/26793642104?pr=7492#step:5:2050
It is reaching a value of 18 blocks cleaned, and the test is looking for 19.

It was changed in the test from 18 to 19 here 2ed48f7 one day after the change that introduced the filter code which ends up duplicating a block 68bef3f.

The reasoning at the time was that the hardcoded test value might have been taken from a run of the CI that didn't report the full number in the metric, but looking at it now this +1 has to be from a block being cleaned up that was a duplicate from the downsampling.

[tghartland]

@yeya24 I made some further changes to fix the e2e compaction tests, please could you take another look when you have the time?

[yeya24]

Thanks! Sorry for the lag. Let's merge it