NEON Data in Container

[TeaganKing]

NEON seems to be pulling in the wrong initial data as well as data from V1 instead of V2.

In the tutorial notebook, I'm seeing the following error (this should be looking for 2022 data):
Model ctsm missing file finidat = '/home/user/CLM-NEON/MOAB.transient/run/inputdata/lnd/ctsm/initdata/MOAB.2021-09-02.clm2.r.0268-01-01-00000.nc'

[ekluzek]

@TeaganKing I think this because the Tutorial pulls a specific version of the container escomp/ctsm-lab-2.3-preview. I think this should work if you use a newer version of the container.

This is actually an issue that I wanted to bring up, which is that we try to have things work with previous versions of CTSM, but changes in NEON aren't setup in a way to allow that to happen. So you kind of need to be using the latest versions for NEON to work (at least for some of the changes that have happened).

[ekluzek]

The issue that I saw was that when I ran a previous version of CTSM the NEON test failed (that passed previously) because of these type of changes. We need to think about how to avoid having this happen and also how to let people know about these type of backward compatibility issues.

@TeaganKing and @wwieder I wonder about having a meeting to discuss this?

[TeaganKing]

Hi @ekluzek , Thanks for taking a look at this! I should have specified that I was pulling escomp/cesm-lab-neon instead of ctsm-lab-2.3-preview. I actually was chatting with @wwieder earlier today about this, so he may also have specific ideas of how to address this. Happy to meet sometime if need be.

[wwieder]

I think there are actually two issues going on here. The topic Teagan actually pasted into her initial posting is regarding initial conditions data

NEON/MOAB.transient/run/inputdata/lnd/ctsm/initdata/MOAB.2021-09-02.clm2.r.0268-01-01-00000.nc'

which is not the most up to date file, which should be dated 2022-10-15. I'll fix this when Cheyenne is back up with changes made to the neon_gcs_upload.py script.

The topic of the post, however, was about in datm data that's being read in. Teagan's run was pointing to v1 data, not the updated v2 datasets.
Model datm missing file file1 = '/home/user/CLM-NEON/MOAB.transient/run/inputdata/atm/cdeps/v1/MOAB/MOAB_atm_2018-01.nc'

I was unable to reproduce this error, but a safe method may be to force the code to pull from V2 data. @negin513 suggested:

run_neon --neon-sites $site --output-root ~/CLM-NEON --neon-version v2

I can make this modification to the NEON_Simulation_Tutorial.

[wwieder]

This can be closed with #1872, which addresses the location of initial conditions files.