Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a complete report of test plan #320

Closed
giladmaya opened this issue Dec 29, 2022 · 3 comments
Closed

Add a complete report of test plan #320

giladmaya opened this issue Dec 29, 2022 · 3 comments
Assignees

Comments

@giladmaya
Copy link

Is your feature request related to a problem? Please describe.
Running the tests is a bit frustrating. When running the tests via 'make test' it seems that the test stops after the first failure; hence we can't tell if additional tests are broken or even how many tests remained in the test plan.
As a result, contributors will be able to better understand the impact of the change they've made and the overhead of re-running the tests and discovering only one issue at a time will be reduced significantly.

Describe the solution you'd like
I would add an option to run make test with stop-on-first-failure approach. The default behavior would be to complete the entire test plan.
On both options, the failures would be collected during the run and exported to a text file.

@Yuval-Ariel
Copy link
Contributor

running the unit tests in parallel already provides such functionality wither with gnu parallel or ctest if you're using cmake. please refer to https://docs.speedb.dev/documentation/getting-started/how-to-contribute/contribute-code#c++-unit-tests and see if that works out for you.
also, @AmnonHanuhov , can you elaborate more about the dependencies regarding this issue? #122 and c528ec8

@Yuval-Ariel
Copy link
Contributor

also note this issue - #288

@AmnonHanuhov
Copy link
Contributor

running the unit tests in parallel already provides such functionality wither with gnu parallel or ctest if you're using cmake. please refer to https://docs.speedb.dev/documentation/getting-started/how-to-contribute/contribute-code#c++-unit-tests and see if that works out for you.
also, @AmnonHanuhov , can you elaborate more about the dependencies regarding this issue? #122 and c528ec8

Regarding #122 there are no special dependencies when using gtest-parallel for running the tests in parallel, that is, there is no need to install additional python packages and the use of ‘make check’ is exactly the same.
Currently on the main branch, to run all the tests in parallel and to overcome the issue mentioned by @giladmaya , we execute ‘make check’, which uses GNU Parallel (has to be pre installed on the users machine) to run the tests in parallel.
After #122 will be merged, to run the tests in parallel you will still only need to execute ‘make check’, which will use gtest-parallel, but the good thing is gtest-parallel comes bundled in our repo.

@Yuval-Ariel Yuval-Ariel closed this as not planned Won't fix, can't repro, duplicate, stale Jan 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants