Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[v24.2.x] archival: consistent log size probes across replicas (pull) #24365

Merged
merged 1 commit into from
Dec 2, 2024
Merged

[v24.2.x] archival: consistent log size probes across replicas (pull) #24365

merged 1 commit into from
Dec 2, 2024

Conversation

nvartolomei
Copy link
Contributor

Backport of PR #24342

@nvartolomei nvartolomei added this to the v24.2.x-next milestone Nov 28, 2024
@nvartolomei nvartolomei added the kind/backport PRs targeting a stable branch label Nov 28, 2024
@nvartolomei nvartolomei marked this pull request as ready for review November 28, 2024 16:31
@nvartolomei
archival: consistent log size probes across replicas
11ea3f6 
We called update probe only from leaders and after exiting the upload
loop which led to inconsistent and stale metrics.

Fix this by introducing a subscription mechanism to the STM which
is the source of truth for the manifest state and must be consistent
across all replicas.

The first attempt was in
redpanda-data#24257 but the feedback
suggested that the approach in this commit is better.

(cherry picked from commit ab1dd53)
@nvartolomei nvartolomei force-pushed the nv/manual-backport-24342-v24.2.x-442 branch from 97063bb to 11ea3f6 Compare November 28, 2024 17:27
@vbotbuildovich
Copy link
Collaborator

non flaky failures in https://buildkite.com/redpanda/redpanda/builds/58982#01937419-5b05-4bf6-83fa-76869472ff8a:

"rptest.tests.e2e_shadow_indexing_test.EndToEndShadowIndexingTestWithDisruptions.test_write_with_node_failures.cloud_storage_type=CloudStorageType.ABS"

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Nov 28, 2024 •
edited
Loading

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/58982#01937419-5b05-4bf6-83fa-76869472ff8a
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/58982#01937415-210f-4fb7-95e2-a258a8f29007

@vbotbuildovich
Copy link
Collaborator

Retry command for Build#58982

please wait until all jobs are finished before running the slash command

/ci-repeat 1
tests/rptest/tests/e2e_shadow_indexing_test.py::EndToEndShadowIndexingTestWithDisruptions.test_write_with_node_failures@{"cloud_storage_type":2}

@nvartolomei
Copy link
Contributor Author

/ci-repeat 1
skip-redpanda-build
skip-rebase
debug
tests/rptest/tests/e2e_shadow_indexing_test.py::EndToEndShadowIndexingTestWithDisruptions.test_write_with_node_failures@{"cloud_storage_type":2}

@nvartolomei nvartolomei disabled auto-merge November 29, 2024 11:51
@nvartolomei nvartolomei requested a review from Lazin November 29, 2024 11:51
@nvartolomei nvartolomei merged commit 2528c1f into redpanda-data:v24.2.x Dec 2, 2024
17 checks passed
@nvartolomei nvartolomei deleted the nv/manual-backport-24342-v24.2.x-442 branch December 2, 2024 12:48
@piyushredpanda piyushredpanda modified the milestones: v24.2.x-next, v24.2.13 Dec 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Lazin Lazin Lazin approved these changes

Assignees
No one assigned
Labels
area/redpanda kind/backport PRs targeting a stable branch
Projects
None yet
Milestone
v24.2.13
Development

Successfully merging this pull request may close these issues.
4 participants
@nvartolomei @vbotbuildovich @Lazin @piyushredpanda