Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify NEON surface data #1353

Closed
7 tasks done
wwieder opened this issue Apr 22, 2021 · 4 comments · Fixed by #1278
Closed
7 tasks done

Modify NEON surface data #1353

wwieder opened this issue Apr 22, 2021 · 4 comments · Fixed by #1278
Assignees
Labels
enhancement new capability or improved behavior of existing capability

Comments

@wwieder
Copy link
Contributor

wwieder commented Apr 22, 2021

  • Build off of existing script tools/contrib/modify_singlept_site with modify_singlept_NEON

  • Include these modified surface dataset in inputdata (or do users run this script)?

  • Updating PFTs

    • PCT_NAT_PFT should reflect vegetation at the site
    • This is now being done in subset_data.py frrom Singlept and regional subsetting script #1360 reading in data file used to generate surface datasets.
    • This will may require input from the NEON team on this file.
      • A copy of this file on Cheyenne is available here /glade/p/cgd/tss/people/wwieder/inputdata/single_point/NEON_SiteData/NEONtowerSiteMetadata.csv
    • How do we handle sites with mixed vegetation KONZ or crops KONA?
  • Soil Properties, soil texture

    • Modify surface data with NEON observations that are posted in NEON bucket (soil texture, PFTs, etc).
    • PCT_SAND, PCT_CLAY on surface data should come from sandTotal and sandTotal on NEON.
    • This will require mapping depths associated with nlevsoi on the surface dataset with biogeoTopDepth and biogeoBottomDepth in NEON data.
    • Here, I'd suggest that for CTSM soil levels that cover two NEON layers the data are NOT averaged, but CTSM use the largest depth measured
    • Sample data can be seen here
  • Soil Properties, ORGANIC

    • ORGANIC on surface dataset needs to be "kg/m3 (assumed carbon content 0.58 gC per gOM)".
      Notes below should be checked (including units of NEON data)
    • carbonTot in NEON data are given as gC/kg soil, bulk density is reported as g/cm3 and depth is in cm
    • Thus for this conversion:
      ORGANIC = carbonTot * bulkDensity / 0.58 **This has been modified based on this comment
  • check metadata provided with NEON observations

  • depth to bedrock, zbedrock

    • Megapit data should all be from 0-200 cm. Thus, rocky sites that don't get that deep should also modify zbedrock

    • Here's a quick look at sampling depth by sites
      image

    • This raises some interesting questions:

      • RMNP and HARV, I'd expect to see shallower soils
      • Elsewhere (HEAL and TOOL) it looks like sampling didn't drill down into permafrost? Is this accurate?
      • Elsewhere, I don't really know what to expect (e.g. BARR, PUUM, OAES)?
@wwieder wwieder added the enhancement new capability or improved behavior of existing capability label Apr 22, 2021
@wwieder wwieder changed the title Modify surface data for NEON Modify NEON surface data Apr 22, 2021
@wwieder
Copy link
Contributor Author

wwieder commented Apr 22, 2021

Initially I'd suggest we just use a single PFT for each site, even those with mixed vegetation, as it will make analyses much easier. It also is easier than trying to determine the relative cover of each pft over the tower footprint. @danicalombardozzi we can discuss further.

@wwieder
Copy link
Contributor Author

wwieder commented May 18, 2021

Eventually we may want to let users more easily define the PFT's present at a site (and their % cover). @negin513 is this simple to implement or should we wait to do it later?

@ekluzek
Copy link
Collaborator

ekluzek commented May 18, 2021

@wwieder my understanding is that the originating subset_data.py script handles the changes for PFT's. This is doing the changes on top of that script that are needed just for NEON sites. So I think you would handle any changes to PFT's in that original script.

@negin513 can you clarify here?

@ekluzek ekluzek mentioned this issue May 18, 2021
@wwieder wwieder reopened this Jul 13, 2021
olyson added a commit to fang-bowen/CTSM that referenced this issue Dec 13, 2021
Redo options list to remove positional arguments that were difficult to input correctly.
Transient runs now use run_type startup and get finidat from s3 server unless --run-from-postad option is used (or finidat is not
available). Use mpi instead of mpi-serial, this mod was recommended for container use. Add a new script neon_finidat_upload which
allows authorized users to upload finidat files to the s3 server.

This includes the following changes to the script for updating the surface dataset at neon sites using available neon data (i.e.
modify_singlept_site_neon.py) to address ctsm/issues ESCOMP#1353, ESCOMP#1429, and neon/issue ESCOMP#44:

Update Organic calculation to use the following equation based on discussions in
https://github.com/ESCOMP/CTSM/pull/1375/files#r669590971 :
ORGANIC = estimatedOC * bulkDensity / 0.58

Because estimatedOC is rounded to the nearest integer in neon data, it is sometimes bigger than carbonTot. Therefore, in cases where
estimatedOC > carbonTot, we use carbonTot instead of estimatedOC in the above equation.

Previously, we had missing data on neon files for some neon soil layers (see Modified NEON surface datasets have errors  ESCOMP#1429
(comment)). Therefore, it caused some missing values in the updated dataset. Here, we interpolate to fill in the missing data using
different interpolation techniques. Finally, we chose to use linear interpolation to fill in missing data.

This includes the scripts for modification of the surface dataset for neon sites to address ESCOMP#1429.
Specifically, the following has been addressed in this PR:

Update the calculation of ORGANIC to use the new field (CaCO3) from NEON data.
For this calculation if CaCO3 data is available, we first calculate inorganic carbon by:
inorganic carbon = (caco3 /100.0869)*12.0107
Next, we calculate organic carbon by subtracting inorganic carbon from the total carbon:
[organic carbon = carbon_tot - inorganic carbon]
If the CaCO3 is not available then the code uses carbonTot and estimatedOC by NEON.

Discussed here (Modified NEON surface datasets have errors  ESCOMP#1429 (comment))
For the Ag sites (KONA and STER), it changes the PCT_NATVEG, PCT_CROP, and PCT_NAT_PFT to avoid the error that we previously had in
spin-up: surfrd_veg_all ERROR: sum of wt_nat_patch not 1.00000000000000 at nl= 1 sum is: 0.000000000000000E+000

Discussed here (Modified NEON surface datasets have errors  ESCOMP#1429 (comment))
There was a typo previously in the NEON data for ABBY sites caused by mix of sample measurements. Please note that this was updated
by hand once data was downloaded from NEON site.

With recent versions of CIME, the LILAC build with a user-defined machine was broken for a couple of reasons. This fixes it.

Fix mksurfdata_map for 1x1_brazil. Get tools testing working again. Increase skip_steps by 1, which is needed for a change in CAM
where balance checks need to occur after the radiation update now rather than before. glob changed for bsd_glob in perl MkDepends
for mksurfdata_map.
@wwieder
Copy link
Contributor Author

wwieder commented Dec 17, 2021

closed with #1474

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement new capability or improved behavior of existing capability
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

5 participants