Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more tests to ctsm_sci for submodule updates #1859

Open
billsacks opened this issue Sep 29, 2022 · 5 comments
Open

Add more tests to ctsm_sci for submodule updates #1859

billsacks opened this issue Sep 29, 2022 · 5 comments
Labels
testing additions or changes to tests

Comments

@billsacks
Copy link
Member

Motivated by the recent CDEPS bug that prevented NEON spinup cases from running past year 100, @wwieder wondered if it would make sense to have a special test suite with some extra tests that are run when updating externals. These would be tests whose main purpose is to cover changes in externals rather than the CTSM code itself. I like this idea, assuming this test suite would have more than just a few tests in it and/or it has some larger / longer tests (e.g., longer than an hour or with a high core-hour cost). (If it just has a few small-ish tests, then we might as well just keep them in the standard aux_clm test suite.)

For baseline comparisons, we could either generate baselines manually from the previous tag whenever we need to run this test suite or not worry about baseline comparisons for these tests.

An initial member of this test suite could be a > 100-year NEON spinup case.

Another possibility that I just thought of – to reduce the number of different test suites we have – is to add these tests to the ctsm_sci test suite. The purpose of that test suite is different, and the times it makes sense to run it are also different, but there may be some overlap, and maybe (significant) externals updates are about the right frequency at which we should be running the ctsm_sci test suite.

@billsacks billsacks added the testing additions or changes to tests label Sep 29, 2022
@wwieder
Copy link
Contributor

wwieder commented Sep 29, 2022

This may be a different topic, but I wondered if we wanted to think about creating a scientific test suite that could more automate tests that run simulations that @olyson does for answer changing tags, e.g. short I2000 CTSM-sp cases, or more involved (BGC simulations?). Maybe the question here is can we really automate Keith (or would we want/need to)?

@ekluzek
Copy link
Collaborator

ekluzek commented Apr 14, 2024

Recent issues with NEON again make me think we should do something here.

@samsrabin
Copy link
Collaborator

See also #2442.

@wwieder wwieder removed the next this should get some attention in the next week or two. Normally each Thursday SE meeting. label Apr 25, 2024
@samsrabin
Copy link
Collaborator

It would be good to include the RXCROPMATURITY system test as well.

@wwieder
Copy link
Contributor

wwieder commented Apr 25, 2024

@ekluzek suggested we use CTSM_sci test list (which evaluates scientifically supported compsets), but

@samsrabin thought this may be overkill, because they could be caught with shorter tests?

@wwieder wwieder added this to the CESM3 milestone Apr 25, 2024
@wwieder wwieder changed the title Consider adding a test suite with extra tests that are run when updating externals Add more tests to ctsm_sci tests for submodule updates Apr 25, 2024
@wwieder wwieder changed the title Add more tests to ctsm_sci tests for submodule updates Add more tests to ctsm_sci for submodule updates Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing additions or changes to tests
Development

No branches or pull requests

4 participants