Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI not running all subsystem tests #3884

Closed
sandreim opened this issue Mar 28, 2024 · 4 comments · Fixed by #3885
Closed

CI not running all subsystem tests #3884

sandreim opened this issue Mar 28, 2024 · 4 comments · Fixed by #3885
Labels
I2-bug The node fails to follow expected behavior. T10-tests This PR/Issue is related to tests.

Comments

@sandreim
Copy link
Contributor

sandreim commented Mar 28, 2024

Uncovered this while working on #3879 that touches collation-generation. As it can be seen on that PR all CI tests are passing, but when I run locally with cargo test, they fail.

I believe some recent changes in this area might be the culprit: https://github.com/paritytech/polkadot-sdk/blob/master/.gitlab/pipeline/test.yml#L28

I have fixed the failing tests in #3883

Ouput of running collation-generation tests on master via cargo test --profile=release -p polkadot-node-collation-generation

failures:
    tests::distribute_collation_for_free_cores_with_async_backing_enabled_and_elastic_scaling::case_1
    tests::distribute_collation_for_free_cores_with_async_backing_enabled_and_elastic_scaling::case_2
    tests::distribute_collation_for_free_cores_with_async_backing_enabled_and_elastic_scaling::case_3
    tests::distribute_collation_for_occupied_core_with_async_backing_enabled::case_1
    tests::distribute_collation_for_occupied_core_with_async_backing_enabled::case_2
    tests::distribute_collation_for_occupied_cores_with_async_backing_enabled_and_elastic_scaling::case_1
    tests::distribute_collation_for_occupied_cores_with_async_backing_enabled_and_elastic_scaling::case_2
    tests::distribute_collation_for_occupied_cores_with_async_backing_enabled_and_elastic_scaling::case_3
    tests::no_collation_is_distributed_for_occupied_core_with_async_backing_disabled::case_1
    tests::no_collation_is_distributed_for_occupied_core_with_async_backing_disabled::case_2

test result: FAILED. 10 passed; 10 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s

🙈

@sandreim sandreim added I2-bug The node fails to follow expected behavior. T10-tests This PR/Issue is related to tests. labels Mar 28, 2024
@sandreim
Copy link
Contributor Author

CC @alvicsam @AndreiEres

@ordian
Copy link
Member

ordian commented Mar 29, 2024

do we want to run tests with and w/o elastic-scaling-experimental feature?

@sandreim
Copy link
Contributor Author

I think it makes sense to do so.

@ggwpez
Copy link
Member

ggwpez commented Apr 1, 2024

There was a similar report here, not sure if related: #3845

github-merge-queue bot pushed a commit that referenced this issue Apr 2, 2024
Fixes
#3884 (comment)

After moving regression tests to benchmarks
(#3741) we don't need to
filter tests anymore.

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: Alin Dima <alin@parity.io>
Co-authored-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: Dastan <88332432+dastansam@users.noreply.github.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Clara van Staden <claravanstaden64@gmail.com>
Co-authored-by: Ron <yrong1997@gmail.com>
Co-authored-by: Vincent Geddes <vincent@snowfork.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Ank4n pushed a commit that referenced this issue Apr 9, 2024
Fixes
#3884 (comment)

After moving regression tests to benchmarks
(#3741) we don't need to
filter tests anymore.

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: Alin Dima <alin@parity.io>
Co-authored-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: Dastan <88332432+dastansam@users.noreply.github.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Clara van Staden <claravanstaden64@gmail.com>
Co-authored-by: Ron <yrong1997@gmail.com>
Co-authored-by: Vincent Geddes <vincent@snowfork.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
dharjeezy pushed a commit to dharjeezy/polkadot-sdk that referenced this issue Apr 9, 2024
Fixes
paritytech#3884 (comment)

After moving regression tests to benchmarks
(paritytech#3741) we don't need to
filter tests anymore.

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: Alin Dima <alin@parity.io>
Co-authored-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: Dastan <88332432+dastansam@users.noreply.github.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Clara van Staden <claravanstaden64@gmail.com>
Co-authored-by: Ron <yrong1997@gmail.com>
Co-authored-by: Vincent Geddes <vincent@snowfork.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
I2-bug The node fails to follow expected behavior. T10-tests This PR/Issue is related to tests.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants