Single point simulations

[wwieder]

We are discussing deprecating the PTCLM toolchain (as the toolchain is being remade). This will be replaced with the faster scripts that pull atmospheric forcing, surface dataset and domain file with python scripts from @swensosc (currently in tools/contrib) that can be written over / modified as needed.

Are there features / functions that we'll lose by doing this? Specifically, @olyson how do you set up single point urban simulations?
If / when we go down this route we'll need to:

• modify singlept.py script, Create NEON surface dataset #1352
  • update singlept.py script to python3 (currently python2)
  • develop testing to bring this out of /tools/contrib?
• Create new modify_singlept_NEON script that reads in NEON observations, Modify NEON surface data #1353
  • modify dominant PFTs & soil properties
  • eventually modify parameter file (foliar C:N, allocation, others?)
• Batch process multiple sites, Wrapper: Batch run single point sites, including NEON #1354
  • communicate with PLUMBER2 & NEON for distributing tower forcing data with CTSM
  • improve @olyson .csh script that can simulate multiple sites at once. This may need to be done more generically to run in the cloud.
• Provide jupyter notebooks for sample analyses, Visualization: Python scripts for NEON analyses #1355
• Update users guide to reflect this new workflow, Documentation for single point simulations, especially NEON #1494

This can be addressed by the SE supported by NCAR-NEON funding

[swensosc]

my understanding was that the reason ptclm takes as long as it does is due to its use of mapping files. Another way forward would be to write a script that would go to the raw data files and just grab the values at the closest point. This would be a little work, but it would preclude the need to point to an existing surface data file. It would also represent our 'best guess' for the site, rather than getting values that have been averaged over a much larger area.

…

On Thu, Oct 29, 2020 at 10:36 AM will wieder ***@***.***> wrote: We are discussing deprecating the PTCLM toolchain (as the toolchain is being remade). This will be replaced with the faster scripts that pull atmospheric forcing, surface dataset and domain file with python scripts from @swensosc <https://github.com/swensosc> (currently in tools/contrib) that can be written over / modified as needed. Are there features / functions that we'll lose by doing this? Specifically, @olyson <https://github.com/olyson> how do you set up single point urban simulations? If / when we go down this route we'll need to: - update to python3 (currently python2) - develop testing to bring this out of /tools/contrib - update users guide to reflect this new workflow - communicate with PLUMBER2 & NEON for distributing tower forcing data with CTSM - improve @olyson <https://github.com/olyson> .csh script that can simulate multiple sites at once. This may need to be done more generically to run in the cloud. This can be addressed by the SE supported by NCAR-NEON funding — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1198>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGRN57ERNPTJJPNCW5UQSR3SNGKZLANCNFSM4TD7HFVA> .

[olyson]

For urban sites, I generally just start with an existing surface dataset/domain file and edit using NCL.

[billsacks]

my understanding was that the reason ptclm takes as long as it does is due to its use of mapping files. Another way forward would be to write a script that would go to the raw data files and just grab the values at the closest point. This would be a little work, but it would preclude the need to point to an existing surface data file. It would also represent our 'best guess' for the site, rather than getting values that have been averaged over a much larger area.

That's a good point, @swensosc . @ekluzek @slevisconsulting @negin513 I wonder if it's worth considering an option to the new toolchain that just does a nearest neighbor mapping rather than a conservative mapping. If it's easy to do, we could make this the default for single-point runs, but I'm starting to wonder if it would be a useful option to have in general. My impression is that it should be easy to implement this, since I think it would just mean changing one argument to the esmf regridding routine. I guess, though, the main value in doing this would be to speed the process, so it's only worth doing if it actually does speed the process significantly. (I'm suggesting this instead of a custom script just to keep the process consistent and to make it easier for us to implement that option, under the assumption that ESMF's nearest neighbor mapping would be about as efficient as any custom script we wrote.)

I don't think this would work for the topography standard deviation which is computed from the 1 km dataset, though. There may be other cases that don't work as intended in mksurfdata_map if using nearest neighbor, but off-hand I can't think of others.

[ekluzek]

@billsacks yes I like this idea better. @swensosc creating a script to handle all dozen of the input files to mksurfdata sounds a bit problematic. As @billsacks points out it would need to be synchronized with the mksurfdata_map code as well. If we thought we wanted to replace the mksurfdata_map code with a script (which isn't a bad thing to consider) that would seem better. In the long run having a script replace mksurfdata_map would be a good goal to make it easier to work with -- but that would be a big project.

But, I really like the idea of building into the planned toolchain the ability to get nearest neighbor mapping. That should be tons faster. And yes the standard dev of the 1km datasets is the only thing I can think of that wouldn't work either. But, you really wouldn't get a good value for that for a single point anyway. Maybe that should just be assigned an arbitrary value for single point sites? @swensosc what would be a good way to set this value for a tower site?

I know soil color is handled in a strange way, but I think the nearest neighbor method should work for it.

[ekluzek]

@olyson we still have mksurfdata create datasets for MexicoCity, Vancouver-CAN, Camden-NJ and urbanc_alpha. And the model is setup to run these out of the box. Is there still a need to be able to do this? Or can these special cases be removed? We do create these with each update of the surface datasets. If we didn't need to do that, we could remove some complexity.

We also have the capacity to create a special urban test case called asphalt-jungle (with no permeable road). I don't think we run that test anymore, so we could likely remove that.

[olyson]

I think it could be useful to keep at least one of these as part of the test suite? I think you could remove all but Camden-NJ, as I've found that to be useful very recently.

[swensosc]

The standard deviation of the topography is used in the fractional snow cover parameterization. For site level, a uniform snow cover is more appropriate so I usually set the standard dev to 20. This results in a steep curve and a nearly uniform snow cover fraction. I also usually set fmax to zero, which essentially inactivates the saturated fraction parameterization, which is another subgrid parameterization that isn't appropriate for most site level simulations.

…

On Thu, Oct 29, 2020 at 3:04 PM Erik Kluzek ***@***.***> wrote: @billsacks <https://github.com/billsacks> yes I like this idea better. @swensosc <https://github.com/swensosc> creating a script to handle all dozen of the input files to mksurfdata sounds a bit problematic. As @billsacks <https://github.com/billsacks> points out it would need to be synchronized with the mksurfdata_map code as well. If we thought we wanted to replace the mksurfdata_map code with a script (which isn't a bad thing to consider) that would seem better. In the long run having a script replace mksurfdata_map would be a good goal to make it easier to work with -- but that would be a big project. But, I really like the idea of building into the planned toolchain the ability to get nearest neighbor mapping. That should be tons faster. And yes the standard dev of the 1km datasets is the only thing I can think of that wouldn't work either. But, you really wouldn't get a good value for that for a single point anyway. Maybe that should just be assigned an arbitrary value for single point sites? @swensosc <https://github.com/swensosc> what would be a good way to set this value for a tower site? I know soil color is handled in a strange way, but I think the nearest neighbor method should work for it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1198 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGRN57G7LR3J7ULTVRE6W23SNHKFFANCNFSM4TD7HFVA> .

[dlawrenncar]

This raises the question about whether or not these types of things should somehow get set by default for single point runs, which I guess might be possible if a user uses a compset to setup a run.

…

On Fri, Oct 30, 2020 at 7:24 AM swensosc ***@***.***> wrote: The standard deviation of the topography is used in the fractional snow cover parameterization. For site level, a uniform snow cover is more appropriate so I usually set the standard dev to 20. This results in a steep curve and a nearly uniform snow cover fraction. I also usually set fmax to zero, which essentially inactivates the saturated fraction parameterization, which is another subgrid parameterization that isn't appropriate for most site level simulations. On Thu, Oct 29, 2020 at 3:04 PM Erik Kluzek ***@***.***> wrote: > @billsacks <https://github.com/billsacks> yes I like this idea better. > @swensosc <https://github.com/swensosc> creating a script to handle all > dozen of the input files to mksurfdata sounds a bit problematic. As > @billsacks <https://github.com/billsacks> points out it would need to be > synchronized with the mksurfdata_map code as well. If we thought we wanted > to replace the mksurfdata_map code with a script (which isn't a bad thing > to consider) that would seem better. In the long run having a script > replace mksurfdata_map would be a good goal to make it easier to work with > -- but that would be a big project. > > But, I really like the idea of building into the planned toolchain the > ability to get nearest neighbor mapping. That should be tons faster. And > yes the standard dev of the 1km datasets is the only thing I can think of > that wouldn't work either. But, you really wouldn't get a good value for > that for a single point anyway. Maybe that should just be assigned an > arbitrary value for single point sites? @swensosc > <https://github.com/swensosc> what would be a good way to set this value > for a tower site? > > I know soil color is handled in a strange way, but I think the nearest > neighbor method should work for it. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#1198 (comment)>, or > unsubscribe > < https://github.com/notifications/unsubscribe-auth/AGRN57G7LR3J7ULTVRE6W23SNHKFFANCNFSM4TD7HFVA > > . > — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1198 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFABYVC2LZQDS3W3NNN3OCDSNK5BZANCNFSM4TD7HFVA> .

[swensosc]

sometimes I use single point runs when debugging a global run. in that case, it is still useful to have a script that just pulls a point from a global run, and keeps those parameters identical. On Fri, Oct 30, 2020 at 9:25 AM David Lawrence <notifications@github.com> wrote:

…

This raises the question about whether or not these types of things should somehow get set by default for single point runs, which I guess might be possible if a user uses a compset to setup a run. On Fri, Oct 30, 2020 at 7:24 AM swensosc ***@***.***> wrote: > The standard deviation of the topography is used in the fractional snow > cover parameterization. For site level, a uniform snow cover is more > appropriate so I usually set the standard dev to 20. This results in a > steep curve and a nearly uniform snow cover fraction. I also usually set > fmax to zero, which essentially inactivates the saturated fraction > parameterization, which is another subgrid parameterization that isn't > appropriate for most site level simulations. > > On Thu, Oct 29, 2020 at 3:04 PM Erik Kluzek ***@***.***> > wrote: > > > @billsacks <https://github.com/billsacks> yes I like this idea better. > > @swensosc <https://github.com/swensosc> creating a script to handle all > > dozen of the input files to mksurfdata sounds a bit problematic. As > > @billsacks <https://github.com/billsacks> points out it would need to be > > synchronized with the mksurfdata_map code as well. If we thought we > wanted > > to replace the mksurfdata_map code with a script (which isn't a bad thing > > to consider) that would seem better. In the long run having a script > > replace mksurfdata_map would be a good goal to make it easier to work > with > > -- but that would be a big project. > > > > But, I really like the idea of building into the planned toolchain the > > ability to get nearest neighbor mapping. That should be tons faster. And > > yes the standard dev of the 1km datasets is the only thing I can think of > > that wouldn't work either. But, you really wouldn't get a good value for > > that for a single point anyway. Maybe that should just be assigned an > > arbitrary value for single point sites? @swensosc > > <https://github.com/swensosc> what would be a good way to set this value > > for a tower site? > > > > I know soil color is handled in a strange way, but I think the nearest > > neighbor method should work for it. > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > <#1198 (comment)>, or > > unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AGRN57G7LR3J7ULTVRE6W23SNHKFFANCNFSM4TD7HFVA > > > > . > > > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#1198 (comment)>, or > unsubscribe > < https://github.com/notifications/unsubscribe-auth/AFABYVC2LZQDS3W3NNN3OCDSNK5BZANCNFSM4TD7HFVA > > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1198 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGRN57CJSDAVDDVZNZGJT3TSNLLHPANCNFSM4TD7HFVA> .

[ekluzek]

@danicalombardozzi @wwieder @negin513 @jedwards4b and myself met today and talked about this some in context of the work with NEON. I proposed that both NEON and supported, unsupported sites be setup to use user-mod-directories to get the settings right. The more highly supported sites (like NEON) would have fewer things in the user-mod directory and mostly encoded in XML. For an unsupported tower site the surface datasets would be put in the user_nl_clm file.

So for NEON you'd do this to run the specific Yellowstone NEON tower site...

./create_newcase --compset I1PtClm50Bgc --res CLM_USRDAT --user-mods-dir neon/YELL

For a supported PLUMBER2 site you'd have something similar with maybe plumber2/HARV for the user-mod directory for it. The difference to above is that there would likely be more settings in the user-mod directory itself rather than encapsulated in the XML files.

When a user runs the singlept script to create data for an unsupported site it would be something like this...

./create_newcase --compset I1PtClm50Bgc --res CLM_USRDAT --user-mods-dir mydirectoryIcreated

Where the output of the singlept script is to create a user-mod directory that the use can point to run their case.

By, using a user-mod-directory in each case it makes the workflow more similar between all of the options. The compset and resolution above already exist so there we are just taking advantage of something that's already there.

[slevis-lmwg]

In the context of the NEON work discussed above, @negin513 mentioned to me that the question was posed whether to generate surface datasets for such site simulations on the fly each time one of these simulations is performed. Assuming I understood the issue correctly, my recommendation would be to generate the surface datasets only once to avoid the risk of slightly different copies of the same surface dataset inadvertently creeping into the project. Generating the surface datasets once will contribute to consistency in the performed simulations.
(If I misunderstood the issue, pls disregard my comment.)

[danicalombardozzi]

@slevisconsulting: Thanks for your input! We settled on making the surface datasets once rather than on the fly.

[ekluzek]

@jedwards4b has come up with a great idea to simplify things to eliminate domain files by overloading PTS_MODE in cime to get the latitude and longitude from the xml variables PTS_LAT and PTS_LON. Domain files can be helpful for regional grids to give the size and area and mask of each gridcell. But, for single point sites none of those things are really meaningful.

The cime PR for this is here...

ESMCI/cime#3868

[ekluzek]

PR #1309 brings in the changes so that domain files are not needed when using the NUOPC coupler

[wwieder]

I'm also going to assign @glemieux and @rgknox to this issue, as bringing in FATES functionality for single point cases would also be nice with this effort (but also maybe warrant a specific issue to track).

[rgknox]

Working through some tests at one of our NGEE-Tropics sites for FATES, BCI panama.

In my typical workflow, when running a RES=CLM_USRDAT, I also use:

./xmlchange DATM_MODE=CLM1PT

This generates a file: run/datm.streams.txt.CLM1PT.CLM_USRDAT

My build process doesn't seem to complain about any of my xml settings, but it can't find the stream file. Is this deprecated, or is there a new way to import site level met drivers?

[ekluzek]

@rgknox there's a bug in the latest cime that we have in the latest CTSM that causes this to not work. So consider this non functional until I get it to work. The cime issue is here...

ESMCI/cime#3905

[ekluzek]

@rgknox a workaround I found that seems to work for me for an individual case is to set the ATM_GRID and LND_GRID in the case by hand to CLM_USRDAT with xmlchange. Try that and see if you can get your case to work. Be sure to let me know if it does.

[rgknox]

Thanks @ekluzek

The model is still trying to find an unset fatmlndfrc file:

Model ctsm missing file fatmlndfrc = '/raid1/lbleco/cesm/cesm_input_datasets//share/domains/UNSET'

[ekluzek]

@rgknox you had a comment above about fatmlndfrc, which points out something that looks screwy, but works. But, I've seen confusion with this now, and agree that we should address it. There's also some other cases where UNSET is used in cdeps and cmeps that we should change as well.

datm:

model_maskfile/model_meshfile is UNSET

nuopc.runcofig: single_column_lnd_domainfile = UNSET

env_run.xml:

*_DOMAIN_FILE/MESH are UNSET which might be OK
MASK_MESH is UNSET which might be OK
PROXY,MPI_RUN_COMMAND,DATM_CPLHIST_* are UNSET which I think is OK