Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI Failure (AssertionError: Expected consumer to be throttled, metric value: 0) in EndToEndThrottlingTest.test_throttling #14139

Closed
oleiman opened this issue Oct 12, 2023 · 58 comments · Fixed by #17455, #17474, #18451 or #21379
Closed

CI Failure (AssertionError: Expected consumer to be throttled, metric value: 0) in EndToEndThrottlingTest.test_throttling #14139

oleiman opened this issue Oct 12, 2023 · 58 comments · Fixed by #17455, #17474, #18451 or #21379
Assignees
@nvartolomei
Labels
area/cloud-storage Shadow indexing subsystem ci-failure ci-rca/test CI Root Cause Analysis - Test Issue kind/bug Something isn't working sev/medium Bugs that do not meet criteria for high or critical, but are more severe than low.

Comments

@oleiman
Copy link
Member

oleiman commented Oct 12, 2023 •
edited by github-actions bot
Loading

https://buildkite.com/redpanda/redpanda/builds/38843#018b258e-3c04-46de-a3d9-1b6073086727

Module: rptest.tests.e2e_shadow_indexing_test
Class:  EndToEndThrottlingTest
Method: test_throttling
Arguments:
{
  "cloud_storage_type": 1
}

test_id:    rptest.tests.e2e_shadow_indexing_test.EndToEndThrottlingTest.test_throttling.cloud_storage_type=CloudStorageType.S3
status:     FAIL
run time:   4 minutes 13.106 seconds


    AssertionError('Expected consumer to be throttled, metric value: 0')
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 269, in run_test
    return self.test_context.function(self.test)
  File "/usr/local/lib/python3.10/dist-packages/ducktape/mark/_mark.py", line 481, in wrapper
    return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  File "/root/tests/rptest/services/cluster.py", line 82, in wrapped
    r = f(self, *args, **kwargs)
  File "/root/tests/rptest/tests/e2e_shadow_indexing_test.py", line 1294, in test_throttling
    assert times_throttled > 0, f"Expected consumer to be throttled, metric value: {times_throttled}"
AssertionError: Expected consumer to be throttled, metric value: 0

JIRA Link: CORE-2828
@oleiman oleiman added kind/bug Something isn't working ci-failure labels Oct 12, 2023
@oleiman oleiman mentioned this issue Oct 12, 2023
Merged
7 tasks
@andijcr andijcr added area/cloud-storage Shadow indexing subsystem sev/low Bugs which are non-functional paper cuts, e.g. typos, issues in log messages labels Oct 13, 2023
@StephanDollberg
Copy link
Member

https://buildkite.com/redpanda/redpanda/builds/39015

@ztlpn ztlpn mentioned this issue Oct 25, 2023
Closed
@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/39814
*https://buildkite.com/redpanda/redpanda/builds/40017

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/40517
*https://buildkite.com/redpanda/redpanda/builds/40972
*https://buildkite.com/redpanda/redpanda/builds/41483
*https://buildkite.com/redpanda/redpanda/builds/41734
*https://buildkite.com/redpanda/vtools/builds/10812
*https://buildkite.com/redpanda/redpanda/builds/41748
*https://buildkite.com/redpanda/redpanda/builds/41822
*https://buildkite.com/redpanda/redpanda/builds/42019
*https://buildkite.com/redpanda/redpanda/builds/42062
*https://buildkite.com/redpanda/redpanda/builds/42201
*https://buildkite.com/redpanda/redpanda/builds/42256
*https://buildkite.com/redpanda/redpanda/builds/42398
*https://buildkite.com/redpanda/redpanda/builds/42514
*https://buildkite.com/redpanda/redpanda/builds/42631
*https://buildkite.com/redpanda/redpanda/builds/42652

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/vtools/builds/11146

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/42890

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/43108

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/43255

@mmaslankaprv mmaslankaprv mentioned this issue Jan 4, 2024
Merged
7 tasks
@michael-redpanda michael-redpanda mentioned this issue Jan 5, 2024
Merged
7 tasks
@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/43425
*https://buildkite.com/redpanda/vtools/builds/11290
*https://buildkite.com/redpanda/redpanda/builds/43521
*https://buildkite.com/redpanda/redpanda/builds/43534

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/43625

@piyushredpanda piyushredpanda mentioned this issue Jan 11, 2024
Merged
@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/44032

@andijcr andijcr self-assigned this Jan 26, 2024
@andijcr andijcr added sev/medium Bugs that do not meet criteria for high or critical, but are more severe than low. and removed sev/low Bugs which are non-functional paper cuts, e.g. typos, issues in log messages labels Jan 26, 2024
@rockwotj rockwotj mentioned this issue Jan 31, 2024
Merged
@r-vasquez r-vasquez mentioned this issue Jan 31, 2024
Merged
@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/44552

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/44740

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/44850
*https://buildkite.com/redpanda/vtools/builds/11761
*https://buildkite.com/redpanda/vtools/builds/11770
*https://buildkite.com/redpanda/redpanda/builds/44935
*https://buildkite.com/redpanda/redpanda/builds/44994
*https://buildkite.com/redpanda/redpanda/builds/45000
*https://buildkite.com/redpanda/redpanda/builds/45001

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/45088

@piyushredpanda piyushredpanda mentioned this issue Feb 20, 2024
Merged
@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/45148

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/48915
*https://buildkite.com/redpanda/vtools/builds/13626

@nvartolomei nvartolomei mentioned this issue May 11, 2024
Closed
@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/48955
*https://buildkite.com/redpanda/redpanda/builds/48965

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/vtools/builds/13656
*https://buildkite.com/redpanda/redpanda/builds/48983

@Lazin Lazin mentioned this issue May 13, 2024
Merged
7 tasks
@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/vtools/builds/13666
*https://buildkite.com/redpanda/redpanda/builds/48990
*https://buildkite.com/redpanda/vtools/builds/13673

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/49046
*https://buildkite.com/redpanda/vtools/builds/13709

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/47049
*https://buildkite.com/redpanda/redpanda/builds/47052
*https://buildkite.com/redpanda/redpanda/builds/47062
*https://buildkite.com/redpanda/vtools/builds/12578
*https://buildkite.com/redpanda/redpanda/builds/48760
*https://buildkite.com/redpanda/redpanda/builds/48760
*https://buildkite.com/redpanda/redpanda/builds/48769
*https://buildkite.com/redpanda/redpanda/builds/48769
*https://buildkite.com/redpanda/redpanda/builds/48752
*https://buildkite.com/redpanda/redpanda/builds/48752
*https://buildkite.com/redpanda/redpanda/builds/48752
*https://buildkite.com/redpanda/redpanda/builds/48769
*https://buildkite.com/redpanda/redpanda/builds/48769
*https://buildkite.com/redpanda/redpanda/builds/48760
*https://buildkite.com/redpanda/redpanda/builds/48760
*https://buildkite.com/redpanda/redpanda/builds/48772
*https://buildkite.com/redpanda/redpanda/builds/48772
*https://buildkite.com/redpanda/redpanda/builds/48770
*https://buildkite.com/redpanda/redpanda/builds/48770
*https://buildkite.com/redpanda/redpanda/builds/48773
*https://buildkite.com/redpanda/redpanda/builds/48773
*https://buildkite.com/redpanda/redpanda/builds/48773
*https://buildkite.com/redpanda/redpanda/builds/48773
*https://buildkite.com/redpanda/redpanda/builds/48774
*https://buildkite.com/redpanda/redpanda/builds/48774
*https://buildkite.com/redpanda/redpanda/builds/48774
*https://buildkite.com/redpanda/redpanda/builds/48772
*https://buildkite.com/redpanda/redpanda/builds/48772
*https://buildkite.com/redpanda/redpanda/builds/48773
*https://buildkite.com/redpanda/redpanda/builds/48773
*https://buildkite.com/redpanda/redpanda/builds/48770
*https://buildkite.com/redpanda/redpanda/builds/48770
*https://buildkite.com/redpanda/redpanda/builds/48770
*https://buildkite.com/redpanda/redpanda/builds/48770
*https://buildkite.com/redpanda/redpanda/builds/48772
*https://buildkite.com/redpanda/redpanda/builds/48772
*https://buildkite.com/redpanda/redpanda/builds/48777
*https://buildkite.com/redpanda/redpanda/builds/48777
*https://buildkite.com/redpanda/redpanda/builds/48774
*https://buildkite.com/redpanda/redpanda/builds/48774
*https://buildkite.com/redpanda/redpanda/builds/48777
*https://buildkite.com/redpanda/redpanda/builds/48777
*https://buildkite.com/redpanda/redpanda/builds/48775
*https://buildkite.com/redpanda/redpanda/builds/48775
*https://buildkite.com/redpanda/redpanda/builds/48779
*https://buildkite.com/redpanda/redpanda/builds/48775
*https://buildkite.com/redpanda/redpanda/builds/48775
*https://buildkite.com/redpanda/redpanda/builds/48779
*https://buildkite.com/redpanda/redpanda/builds/48779
*https://buildkite.com/redpanda/redpanda/builds/48775
*https://buildkite.com/redpanda/redpanda/builds/48775
*https://buildkite.com/redpanda/redpanda/builds/48798
*https://buildkite.com/redpanda/redpanda/builds/48798
*https://buildkite.com/redpanda/redpanda/builds/48811
*https://buildkite.com/redpanda/redpanda/builds/48811
*https://buildkite.com/redpanda/redpanda/builds/48811
*https://buildkite.com/redpanda/redpanda/builds/48811
*https://buildkite.com/redpanda/redpanda/builds/48798
*https://buildkite.com/redpanda/redpanda/builds/48798
*https://buildkite.com/redpanda/redpanda/builds/48812
*https://buildkite.com/redpanda/redpanda/builds/48812
*https://buildkite.com/redpanda/redpanda/builds/48812
*https://buildkite.com/redpanda/redpanda/builds/48812
*https://buildkite.com/redpanda/vtools/builds/13546
*https://buildkite.com/redpanda/redpanda/builds/48813
*https://buildkite.com/redpanda/redpanda/builds/48813
*https://buildkite.com/redpanda/redpanda/builds/48813
*https://buildkite.com/redpanda/redpanda/builds/48815
*https://buildkite.com/redpanda/redpanda/builds/48815
*https://buildkite.com/redpanda/redpanda/builds/48814
*https://buildkite.com/redpanda/redpanda/builds/48814
*https://buildkite.com/redpanda/redpanda/builds/48812
*https://buildkite.com/redpanda/redpanda/builds/48812
*https://buildkite.com/redpanda/redpanda/builds/48814
*https://buildkite.com/redpanda/redpanda/builds/48814
*https://buildkite.com/redpanda/redpanda/builds/48815
*https://buildkite.com/redpanda/redpanda/builds/48815
*https://buildkite.com/redpanda/redpanda/builds/48814
*https://buildkite.com/redpanda/redpanda/builds/48814
*https://buildkite.com/redpanda/redpanda/builds/48813
*https://buildkite.com/redpanda/redpanda/builds/48813
*https://buildkite.com/redpanda/redpanda/builds/48811
*https://buildkite.com/redpanda/redpanda/builds/48815
*https://buildkite.com/redpanda/redpanda/builds/48815
*https://buildkite.com/redpanda/vtools/builds/13566
*https://buildkite.com/redpanda/vtools/builds/13567
*https://buildkite.com/redpanda/redpanda/builds/48879
*https://buildkite.com/redpanda/vtools/builds/13590
*https://buildkite.com/redpanda/vtools/builds/13606
*https://buildkite.com/redpanda/redpanda/builds/48882
*https://buildkite.com/redpanda/vtools/builds/13607
*https://buildkite.com/redpanda/redpanda/builds/48915
*https://buildkite.com/redpanda/vtools/builds/13626
*https://buildkite.com/redpanda/redpanda/builds/48955
*https://buildkite.com/redpanda/redpanda/builds/48965
*https://buildkite.com/redpanda/vtools/builds/13656
*https://buildkite.com/redpanda/redpanda/builds/48983
*https://buildkite.com/redpanda/redpanda/builds/48990
*https://buildkite.com/redpanda/vtools/builds/13666
*https://buildkite.com/redpanda/vtools/builds/13673
*https://buildkite.com/redpanda/redpanda/builds/49046
*https://buildkite.com/redpanda/vtools/builds/13709
*https://buildkite.com/redpanda/vtools/builds/13727
*https://buildkite.com/redpanda/redpanda/builds/49192
*https://buildkite.com/redpanda/vtools/builds/13739
*https://buildkite.com/redpanda/vtools/builds/13754
*https://buildkite.com/redpanda/redpanda/builds/49263
*https://buildkite.com/redpanda/vtools/builds/13865
*https://buildkite.com/redpanda/vtools/builds/13874
*https://buildkite.com/redpanda/vtools/builds/13892
*https://buildkite.com/redpanda/vtools/builds/13923
*https://buildkite.com/redpanda/vtools/builds/13944
*https://buildkite.com/redpanda/vtools/builds/13970
*https://buildkite.com/redpanda/redpanda/builds/49403
*https://buildkite.com/redpanda/vtools/builds/13989
*https://buildkite.com/redpanda/redpanda/builds/49536
*https://buildkite.com/redpanda/vtools/builds/14049
*https://buildkite.com/redpanda/vtools/builds/14048
*https://buildkite.com/redpanda/vtools/builds/14079
*https://buildkite.com/redpanda/vtools/builds/14078
*https://buildkite.com/redpanda/vtools/builds/14089
*https://buildkite.com/redpanda/vtools/builds/14092
*https://buildkite.com/redpanda/vtools/builds/14098
*https://buildkite.com/redpanda/vtools/builds/14105
*https://buildkite.com/redpanda/vtools/builds/14099
*https://buildkite.com/redpanda/redpanda/builds/49584
*https://buildkite.com/redpanda/redpanda/builds/49594
*https://buildkite.com/redpanda/redpanda/builds/49631
*https://buildkite.com/redpanda/vtools/builds/14181
*https://buildkite.com/redpanda/vtools/builds/14183
*https://buildkite.com/redpanda/vtools/builds/14199
*https://buildkite.com/redpanda/vtools/builds/14252
*https://buildkite.com/redpanda/vtools/builds/14244
*https://buildkite.com/redpanda/vtools/builds/14267
*https://buildkite.com/redpanda/vtools/builds/14265
*https://buildkite.com/redpanda/vtools/builds/14279
*https://buildkite.com/redpanda/vtools/builds/14280
*https://buildkite.com/redpanda/vtools/builds/14442
*https://buildkite.com/redpanda/redpanda/builds/49986
*https://buildkite.com/redpanda/vtools/builds/14509
*https://buildkite.com/redpanda/redpanda/builds/49994
*https://buildkite.com/redpanda/redpanda/builds/50033
*https://buildkite.com/redpanda/redpanda/builds/50037
*https://buildkite.com/redpanda/vtools/builds/14567
*https://buildkite.com/redpanda/redpanda/builds/50060
*https://buildkite.com/redpanda/redpanda/builds/50067
*https://buildkite.com/redpanda/redpanda/builds/50053
*https://buildkite.com/redpanda/redpanda/builds/50078
*https://buildkite.com/redpanda/redpanda/builds/50093
*https://buildkite.com/redpanda/redpanda/builds/50091
*https://buildkite.com/redpanda/vtools/builds/14589
*https://buildkite.com/redpanda/vtools/builds/14598
*https://buildkite.com/redpanda/redpanda/builds/50116
*https://buildkite.com/redpanda/redpanda/builds/50119

@mmaslankaprv mmaslankaprv mentioned this issue Jun 12, 2024
Merged
7 tasks
@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/50145
*https://buildkite.com/redpanda/redpanda/builds/50144
*https://buildkite.com/redpanda/redpanda/builds/50152
*https://buildkite.com/redpanda/redpanda/builds/50196
*https://buildkite.com/redpanda/redpanda/builds/50230
*https://buildkite.com/redpanda/redpanda/builds/50234
*https://buildkite.com/redpanda/vtools/builds/14710
*https://buildkite.com/redpanda/redpanda/builds/50325
*https://buildkite.com/redpanda/vtools/builds/14768
*https://buildkite.com/redpanda/redpanda/builds/50381
*https://buildkite.com/redpanda/redpanda/builds/50411
*https://buildkite.com/redpanda/vtools/builds/14865

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/50508
*https://buildkite.com/redpanda/redpanda/builds/50507
*https://buildkite.com/redpanda/vtools/builds/14929
*https://buildkite.com/redpanda/vtools/builds/14982
*https://buildkite.com/redpanda/vtools/builds/14997
*https://buildkite.com/redpanda/vtools/builds/15021
*https://buildkite.com/redpanda/redpanda/builds/50685
*https://buildkite.com/redpanda/redpanda/builds/50723
*https://buildkite.com/redpanda/redpanda/builds/50731
*https://buildkite.com/redpanda/redpanda/builds/50784

@bashtanov bashtanov mentioned this issue Jun 27, 2024
Closed
7 tasks
@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/50828

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/50871
*https://buildkite.com/redpanda/redpanda/builds/50937
*https://buildkite.com/redpanda/redpanda/builds/50985
*https://buildkite.com/redpanda/redpanda/builds/51038
*https://buildkite.com/redpanda/redpanda/builds/51081
*https://buildkite.com/redpanda/redpanda/builds/51092
*https://buildkite.com/redpanda/redpanda/builds/51100
*https://buildkite.com/redpanda/vtools/builds/15297

@andijcr andijcr mentioned this issue Jul 5, 2024
Merged
7 tasks
@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/51129
*https://buildkite.com/redpanda/redpanda/builds/51132
*https://buildkite.com/redpanda/redpanda/builds/51132

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/vtools/builds/15352

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/51193

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/51210

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/vtools/builds/15488
*https://buildkite.com/redpanda/redpanda/builds/51299

@nvartolomei nvartolomei assigned nvartolomei and unassigned Lazin Jul 11, 2024
This was referenced Jul 12, 2024
Merged
Merged
@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/51358
*https://buildkite.com/redpanda/redpanda/builds/51395
*https://buildkite.com/redpanda/redpanda/builds/51436
*https://buildkite.com/redpanda/redpanda/builds/51433
*https://buildkite.com/redpanda/redpanda/builds/51445
*https://buildkite.com/redpanda/vtools/builds/15612

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/vtools/builds/15627
*https://buildkite.com/redpanda/redpanda/builds/51531

@vbotbuildovich
Copy link
Collaborator

*https://buildkite.com/redpanda/redpanda/builds/51554
*https://buildkite.com/redpanda/redpanda/builds/51564
*https://buildkite.com/redpanda/redpanda/builds/51569

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nvartolomei nvartolomei

Labels
area/cloud-storage Shadow indexing subsystem ci-failure ci-rca/test CI Root Cause Analysis - Test Issue kind/bug Something isn't working sev/medium Bugs that do not meet criteria for high or critical, but are more severe than low.
Projects
None yet
Milestone
No milestone
6 participants
@Lazin @andijcr @nvartolomei @oleiman @StephanDollberg @vbotbuildovich