Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Taal: sort and restructure tests #301

Closed
sloede opened this issue Nov 6, 2020 · 3 comments · Fixed by #327
Closed

Taal: sort and restructure tests #301

sloede opened this issue Nov 6, 2020 · 3 comments · Fixed by #327
Assignees

Comments

@sloede
Copy link
Member

sloede commented Nov 6, 2020

Following up on the conversation started in #299 (comment), we should - once all elixirs and tests are ported to Taal and no comparisons to Taam are needed anymore - sort all tests by dimension and equation name.

At this point, we should also consider if

  1. it would make sense to further separate the tests by dimension and equation in different files for easier/faster testing (Mix more 3D elixirs and port correspondings tests #299 (comment))
  2. we really need tests on 3 OSs for everything (Mix more 3D elixirs and port correspondings tests #299 (comment))

Re 1:
If we do this, we should keep an eye on the GitHub Actions usage limits, such as to not make testing slower again. OTOH, if we do separate by dimension and equation, it might be nice to provide "aggregate" test files that allow easy execution of similar tests, e.g., if one wants to run all 2D tests, or all Euler tests etc.

@ranocha
Copy link
Member

ranocha commented Nov 9, 2020

  1. it would make sense to further separate the tests by dimension and equation in different files for easier/faster testing

I don't think that's useful. Setting up Julia and building Trixi takes between one (Linux) and four minutes (Windows) per CI job. Moreover, most CI time is spent on compilation. Hence, we benefit from using fewer CI jobs in parallel. Our current setup is fine, I think. Nevertheless, I would like to sort the tests by equation in the test files.

2. we really need tests on 3 OSs for everything

Currently, we test 2D in serial and parallel on all three OS and everything else only on Linux. That's okay for me and scales well - we test all essential features possibly involving non-Julia-only dependencies on all three OS but restrict the more expensive tests to one OS.

@sloede
Copy link
Member Author

sloede commented Nov 9, 2020

OK, then let's leave the current setup of what is tested where (point 2) as it is.

Your arguments re 1) also make sense, thus for CI testing we should leave it as it is. However, for developing and intermediate testing I think it still would be nice to be able to run, e.g., just 2D/Euler tests. Right now, I always comment out half the document, which is annoying.

Thus, given that we keep the tests together by dimension for CI, would you still object to separating them into files sorted by equation for different reasons?

@ranocha
Copy link
Member

ranocha commented Nov 9, 2020

Thus, given that we keep the tests together by dimension for CI, would you still object to separating them into files sorted by equation for different reasons?

No, that's totally okay for me 👍

@ranocha ranocha mentioned this issue Nov 11, 2020
50 tasks
@ranocha ranocha self-assigned this Nov 12, 2020
@ranocha ranocha linked a pull request Nov 12, 2020 that will close this issue
@ranocha ranocha closed this as completed Nov 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants