Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI Failure (unknown exception thrown: std::runtime_error (Attempted to perform operation: 'offset_data_stream()' on a closed segment) in CloudStorageTimingStressTest.test_cloud_storage_with_partition_moves #16585

Closed
vbotbuildovich opened this issue Feb 13, 2024 · 4 comments
Labels
area/cloud-storage Shadow indexing subsystem auto-triaged used to know which issues have been opened from a CI job ci-failure sev/medium Bugs that do not meet criteria for high or critical, but are more severe than low.

Comments

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Feb 13, 2024

https://buildkite.com/redpanda/redpanda/builds/44935

Module: rptest.tests.cloud_storage_timing_stress_test
Class: CloudStorageTimingStressTest
Method: test_cloud_storage_with_partition_moves
Arguments: {
    "cleanup_policy": "compact,delete"
}
test_id:    CloudStorageTimingStressTest.test_cloud_storage_with_partition_moves
status:     FAIL
run time:   171.950 seconds

<BadLogLines nodes=docker-rp-32(1) example="ERROR 2024-02-09 07:30:53,913 [shard 0:fetc] kafka - fetch.cc:1171 - unknown exception thrown: std::runtime_error (Attempted to perform operation: 'offset_data_stream()' on a closed segment: {offset_tracker:{term:1, base_offset:1243, committed_offset:1373, dirty_offset:1373}, compacted_segment=1, finished_self_compaction=1, finished_windowed_compaction=1, generation={2}, reader={/var/lib/redpanda/data/kafka/test-topic/0_28/1243-1-v1.log, (33189 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=1, tombstone=1, index={file:/var/lib/redpanda/data/kafka/test-topic/0_28/1243-1-v1.base_index, offsets:{0}, index:{header_bitflags:0, base_offset:{0}, max_offset:{1373}, base_timestamp:{timestamp: 1707463848454}, max_timestamp:{timestamp: 1707463848454}, batch_timestamps_are_monotonic:1, with_offset:true, non_data_timestamps:0, broker_timestamp:{{timestamp: 1707463848455}}, num_compactible_records_appended:{1}, index(2,2,2)}, step:32768, needs_persistence:0}})">
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 269, in run_test
    return self.test_context.function(self.test)
  File "/usr/local/lib/python3.10/dist-packages/ducktape/mark/_mark.py", line 481, in wrapper
    return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  File "/root/tests/rptest/services/cluster.py", line 173, in wrapped
    redpanda.raise_on_bad_logs(
  File "/root/tests/rptest/services/redpanda.py", line 1412, in raise_on_bad_logs
    raise BadLogLines(bad_lines)
rptest.services.utils.BadLogLines: <BadLogLines nodes=docker-rp-32(1) example="ERROR 2024-02-09 07:30:53,913 [shard 0:fetc] kafka - fetch.cc:1171 - unknown exception thrown: std::runtime_error (Attempted to perform operation: 'offset_data_stream()' on a closed segment: {offset_tracker:{term:1, base_offset:1243, committed_offset:1373, dirty_offset:1373}, compacted_segment=1, finished_self_compaction=1, finished_windowed_compaction=1, generation={2}, reader={/var/lib/redpanda/data/kafka/test-topic/0_28/1243-1-v1.log, (33189 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=1, tombstone=1, index={file:/var/lib/redpanda/data/kafka/test-topic/0_28/1243-1-v1.base_index, offsets:{0}, index:{header_bitflags:0, base_offset:{0}, max_offset:{1373}, base_timestamp:{timestamp: 1707463848454}, max_timestamp:{timestamp: 1707463848454}, batch_timestamps_are_monotonic:1, with_offset:true, non_data_timestamps:0, broker_timestamp:{{timestamp: 1707463848455}}, num_compactible_records_appended:{1}, index(2,2,2)}, step:32768, needs_persistence:0}})">

JIRA Link: CORE-1775

@vbotbuildovich vbotbuildovich added auto-triaged used to know which issues have been opened from a CI job ci-failure labels Feb 13, 2024
@michael-redpanda michael-redpanda added the area/cloud-storage Shadow indexing subsystem label Feb 15, 2024
@michael-redpanda michael-redpanda changed the title CI Failure (key symptom) in CloudStorageTimingStressTest.test_cloud_storage_with_partition_moves CI Failure (unknown exception thrown: std::runtime_error (Attempted to perform operation: 'offset_data_stream()' on a closed segment) in CloudStorageTimingStressTest.test_cloud_storage_with_partition_moves Feb 15, 2024
@vbotbuildovich
Copy link
Collaborator Author

@vbotbuildovich
Copy link
Collaborator Author

@piyushredpanda
Copy link
Contributor

Not seen in at 6 weeks, closing

@michael-redpanda
Copy link
Contributor

Automatically closing issue to match current state of CORE-1775

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/cloud-storage Shadow indexing subsystem auto-triaged used to know which issues have been opened from a CI job ci-failure sev/medium Bugs that do not meet criteria for high or critical, but are more severe than low.
Projects
None yet
Development

No branches or pull requests

5 participants