Testing: Generalized testing rules #1502
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`make all` (the default rule) builds the executables specified by
`BUILDS`. This was hard-coded, but can be promoted to a user-defined
configuration for user-defined builds.
This should be seen as a simple alias to `make build/${b}/MOM6`.
The `repro` build was also incorrectly in the list, even though it was
also conditionally added. It has been removed from the default list.
A similar modification was made to CONFIGS, which select the "tc"
experiments. The default is still to do a `tc*` glob.
Documentation was also added.
Macros for generating individual rules for the tc's were added. This was generalized to support the contents of CONFIGS, which is now a user-defined parameter. The wildcard rules have now been replaced with more explicit rules, in preparation for the merge of TESTS and TEST_TYPES. A method for excluding tests has been added for tc3 support, and could be extended to future tests.
The Makefile rules were extended to support multiple iterations of dimension testing. Examples shown below: * test.dim Run all dimension tests * test.dim.l Run all L dimension tests * tc2.dim Run all tc2 dimension tests * tc2.dim.l Run the tc2 L dimension test Also, TESTS and TEST_TYPES were merged into a single variable, and the old plural test names (e.g. test.grids) were removed and are now handled as singular tests => test.grid. The GitHub actions testing was updated to reflect these new non-plural names. It will take some iteration to confirm that they are working.
Codecov Report
@@ Coverage Diff @@
## dev/gfdl #1502 +/- ##
=========================================
Coverage 29.06% 29.06%
=========================================
Files 237 237
Lines 71645 71645
=========================================
Hits 20822 20822
Misses 50823 50823 Continue to review full report at Codecov.
|
adcroft
approved these changes
Oct 1, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds new rules and features to the MOM6 testing Makefile:
BUILDSfor executablesCONFIGSfor configurations (tc1, tc2, etc)TESTSfor test types (grid,rotate, etc)tcand can be added toCONFIGS, although this is still the default behavior.test.dimcan now be used to run all dimensional teststc2will run all tc2 teststc2.gridwill run the symmetric/asymmetric grid tests on tc2tc2.dimwill run all dimension tests on tc2tc2.dim.qwill run the Q-dimensional tests on tc2NOTE:
tc2will run diagnostic tests, but the others will not; usetc2.grid.diagto run them, for example.test.rotationsis nowtest.rotate.%) rules have been redefined as explicitly over the tests.reprobuild was incorrectly being built, even whenDO_REPRO_TESTSwas unset. This has been fixed.make cleannow deletes bothbuildanddeps(including FMS), thereby reversing the behavior ofMakefile.make.buildrule is provided to only delete the MOM6 buildsThe GitHub Actions tests have also been updated to use the renamed