Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nextest --partition integration #281

Open
TriplEight opened this issue May 15, 2023 · 0 comments
Open

nextest --partition integration #281

TriplEight opened this issue May 15, 2023 · 0 comments
Labels
A-nextest Area: nextest integration https://github.com/nextest-rs/nextest C-enhancement Category: A new feature or an improvement for an existing one

Comments

@TriplEight
Copy link

--partition https://nexte.st/book/partitioning.html.
The idea is that the tests are divided (randomly with count and deterministically with hash) to run on the separate machines. This can be used for different platform-specific tests or just to scale horizontally.
The issue is that this command succeeds on each of the partitions, but certainly gives only the partial coverage.

There needs to be a way to collect only the llvm-cov-related instrumentation artifacts to merge them together in the second job and generate the coverage report. nextest's --archive-file does not entirely fit for the job (in #265 author receives the artifact from the previous job and runs tests and coverage from it.

  1. It doesn't assume that llvm-cov artifacts are getting into this archive because nextest generates it possibly without running the tests
  2. --archive-file is designed to run the tests from it, so it should exist before the tests are executed, and nextest does not support a lot of the test-running options for this flag too.

This is both a bug and a feature request, it would be nice to have an archive flag and the ability to generate a report from it for llvm-cov too.

Originally posted by @TriplEight in #271 (comment)


I've written some pipelines to test this behavior, there are much more details #274

@taiki-e taiki-e added C-enhancement Category: A new feature or an improvement for an existing one A-nextest Area: nextest integration https://github.com/nextest-rs/nextest labels May 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-nextest Area: nextest integration https://github.com/nextest-rs/nextest C-enhancement Category: A new feature or an improvement for an existing one
Projects
None yet
Development

No branches or pull requests

2 participants