Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Debug zppy diffs for v3.0.0 #930

Open
tomvothecoder opened this issue Feb 3, 2025 · 0 comments · May be fixed by #931
Open

[Bug]: Debug zppy diffs for v3.0.0 #930

tomvothecoder opened this issue Feb 3, 2025 · 0 comments · May be fixed by #931
Assignees
Labels
bug Bug fix (will increment patch version)

Comments

@tomvothecoder
Copy link
Collaborator

tomvothecoder commented Feb 3, 2025

What happened?

Related to E3SM-Project/zppy#651 (comment)

It is a headache to review 2k+ mis mached images and capture the meaningful signal. Most of the problem was cause by:

1. location of RMSE and CORR printed differently

2. in new e3sm_diags, there is inconsistency of units in upper and middle figures, example **in old version**, `W m-2` is used (if variable is derived) instead of `W/m2` used in model data. [expected](https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.forsyth2/zppy_weekly_comprehensive_v2_www/test_pr651_both_commits_20250117/v2.LR.historical_0201/image_check_failures_comprehensive_v2/e3sm_diags/atm_monthly_180x360_aave/model_vs_obs_1982-1983/lat_lon/CERES-EBAF-TOA-v4.1/ceres_ebaf_toa_v4.1-FLUTC-JJA-global.png_expected.png)

3. ChatGPT helped a bit to filter out problems from 1 and 2. And I manually check rest of the diffs in lat-lon, images have actual difference listed as follows:
ERA5-TREFHT-ANN-land.png
ERA5_ext-QREFHT-ANN-global.png   
ERA5_ext-U10-ANN-global.png      
GPCP_v3.2-PRECT-ANN-global.png   
HadISST_CL-SST-ANN-global.png    
HadISST_PD-SST-ANN-global.png    
HadISST_PI-SST-ANN-global.png    
MACv2-AODVIS-ANN-global.png      
MERRA2-OMEGA-850-ANN-global.png      
MERRA2-PSL-ANN-global.png       
MERRA2-T-850-ANN-global.png     
MERRA2-TAUXY-ANN-ocean.png      
MERRA2-TREFHT-ANN-land.png      
MERRA2-TREFMNAV-ANN-global.png  
MERRA2-TREFMXAV-ANN-global.png  
_MISRCOSP-CLDLOW_TAU1.3_9.4_MISR-ANN-global.png  
MISRCOSP-CLDLOW_TAU1.3_MISR-ANN-global.png      
MISRCOSP-CLDLOW_TAU9.4_MISR-ANN-global.png_      
OMI-MLS-TCO-ANN-60S60N.png
ceres_ebaf_surface_v4.1-ALBEDO_SRF-ANN-global.png

I think they can be further grouped, the MISRCOSP diff can be explained by a recent fix. Other than this, some figures missing the last longitude grid, some missing last latitude, and some have more mysterious diffs... For reference here is the image checker results.

@tomvothecoder maybe you can recognize if there are some diffs that are expected?

What did you expect to happen? Are there are possible answers you came across?

Minimal Complete Verifiable Example (MVCE)

Relevant log output

Anything else we need to know?

No response

Environment

Latest main and v3.0.0rc1/rc2

@tomvothecoder tomvothecoder added the bug Bug fix (will increment patch version) label Feb 3, 2025
@tomvothecoder tomvothecoder linked a pull request Feb 3, 2025 that will close this issue
28 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug fix (will increment patch version)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants