Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix import of moved method in MVKO test #6671

Merged

Conversation

mkstratos
Copy link
Contributor

@mkstratos mkstratos commented Oct 7, 2024

CIME.utils.append_testlog was moved in ESMCI/cime#4660
This updates the MVKO test to use that method (other CIME SystemTests were updated in that PR)

[BFB]

@mkstratos mkstratos requested a review from rljacob October 7, 2024 20:37
@rljacob rljacob self-assigned this Oct 7, 2024
@rljacob rljacob added the Testing Anything related to unit/system tests label Oct 7, 2024
rljacob added a commit that referenced this pull request Oct 8, 2024
CIME.utils.append_testlog was moved in ESMCI/cime#4660
This updates the MVKO test to use that method (other CIME SystemTests were updated in that PR)
[bfb]
@rljacob
Copy link
Member

rljacob commented Oct 8, 2024

The append_testlog error went away but now the MVKO test is reporting a fail:

 ---------------------------------------------------
2024-10-08 01:34:41: BASELINE FAIL for test 'JNextNbfb20241008_003411'.
    Test status: fail; Variables analyzed: 5; Rejecting: 5; Critical value: 0; Ensembles: statistically different
    EVV results can be viewed at:
        https://web.lcrc.anl.gov/public/e3sm/e3smtest/evv/MVKO_PS.T62_oQU240.GMPAS-NYF.chrysalis_intel.C.JNextNbfb20241008_003411/index.html

@rljacob
Copy link
Member

rljacob commented Oct 8, 2024

The baseline its comparing against is in /lcrc/group/e3sm2/baselines/chrys/intel/master/MVKO_PS.T62_oQU240.GMPAS-NYF.chrysalis_intel and its dated May 8. But there should be no changes in answers for a WCYCLE config of MPAS since v3.0.0. @jonbob do you know if anything might have changed in GMPAS-NYF since May 8?

@jonbob
Copy link
Contributor

jonbob commented Oct 8, 2024

@rljacob -- I'll look and see if we changed anything specifically for NYF or the oQU240 mesh, but nothing is coming to mind

@jonbob
Copy link
Contributor

jonbob commented Oct 8, 2024

@rljacob -- we did change the mapping files for T62 because there had been an incorrect T62 scrip file (PR #6459). It was non-BFB but T62 is not used for the coupled model, so that was deemed acceptable

@rljacob
Copy link
Member

rljacob commented Oct 8, 2024

Sure its non-BFB but would it be climate changing? That seems to be what this test is saying.

https://web.lcrc.anl.gov/public/e3sm/e3smtest/evv/MVKO_PS.T62_oQU240.GMPAS-NYF.chrysalis_intel.C.JNextNbfb20241008_003411/index.html

@jonbob
Copy link
Contributor

jonbob commented Oct 8, 2024

@rljacob - I guess it could be climate-changing? But the oQU240 is a grid we only use for testing and it's not considered scientifically validated. Maybe the only real way to find out is to run this test with and without the PR that changed the mapping and domain files and see. Please just yell if that's something you want me to take on

@mkstratos
Copy link
Contributor Author

The different map files seem to be the cause for the climate changing differences. This test: https://web.lcrc.anl.gov/public/e3sm/ac.mkelleher/evv/MVKO_PS.T62_oQU240.GMPAS-NYF.chrysalis_intel.C.20241008_121250_blo0rd run with current master and the old map files passes.

@rljacob
Copy link
Member

rljacob commented Oct 9, 2024

Ok I'll bless it. Its better if oQU240 is more sensitive then less sensitive.

@rljacob rljacob added the BFB PR leaves answers BFB label Oct 9, 2024
@rljacob rljacob merged commit 2089345 into E3SM-Project:master Oct 9, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BFB PR leaves answers BFB Testing Anything related to unit/system tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants