Skip to content

Commit

Permalink
cst/ducktape: Accept errors due to gap in manifest
Browse files Browse the repository at this point in the history
The scrubber test may remove the first segment in the manifest. This
causes a difference between the manifest start offset and the first
segment start offset. The mismatch causes a validation to fail when
spillover is applied (which is correct behavior).

To avoid the test failing due to this expected set of events, the error
log is accepted in the test.
  • Loading branch information
abhijat committed May 16, 2024
1 parent c1ec803 commit 7a9e604
Showing 1 changed file with 20 additions and 1 deletion.
21 changes: 20 additions & 1 deletion tests/rptest/tests/cloud_storage_scrubber_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -487,6 +487,9 @@ def _assert_segment_metadata_anomalies(
# Remove the metadata for the penultimate segment, thus creating an offset gap
# for the scrubber to detect
found = False
seg_to_remove_name = None
seg_to_remove_meta = None
detect_at_seg_meta = None
for crnt_seg, next_seg in itertools.pairwise(sorted_segments):
name, meta = crnt_seg

Expand All @@ -507,11 +510,27 @@ def _assert_segment_metadata_anomalies(
found = True
break

assert found, f"Could not find a segment to remove from the STM manifest"
assert found, "Could not find a segment to remove from the STM manifest"

self.logger.info(f"Removing segment with meta {seg_to_remove_meta}")
self.logger.info(f"Anomaly should be detected at {detect_at_seg_meta}")
manifest['segments'].pop(seg_to_remove_name)
"""
The operation above may remove the first segment in the manifest. This
causes a difference between the manifest start offset and the first
segment start offset. The mismatch causes a validation to fail when
spillover is applied (which is correct behavior) by archival STM.
To avoid the test failing due to this expected set of events, the error
log is added to accept list here.
"""
sorted_segments = sorted(manifest['segments'].items(),
key=lambda entry: entry[1]['base_offset'])
expect_error_at_offset = sorted_segments[0][1]['base_offset']
self.expected_error_logs.append(
re.compile(
fr"Can't apply spillover_cmd.*base_offset: {expect_error_at_offset}"
))

# Forcefully reset the manifest. Remote writes are disabled first
# to make this operation safe.
Expand Down

0 comments on commit 7a9e604

Please sign in to comment.