Update link_fixdirs.sh for new data directories

[GeorgeGayno-NOAA]

DESCRIPTION OF CHANGES:

Update link_fixdirs.sh for new data directories.

• Point to fix_NEW instead of fix.
• Add link to fix_fv3_fracoro.
• Remove link to old directory fix_fv3.

TESTS CONDUCTED:

See below.

DEPENDENCIES:

None.

DOCUMENTATION:

N/A

ISSUE:

Fixes #659.

[GeorgeGayno-NOAA]

The branch at 3d3bb0b was compiled on Hera, Orion, Jet, and WCOSS2. The ./fix/link_fixdirs.sh script was invoked to set the 'fix' directory links. All regression tests passed.

[GeorgeGayno-NOAA]

@KateFriedman-NOAA There are still folks who use the old directories. Will they remain?

[KateFriedman-NOAA]

@KateFriedman-NOAA There are still folks who use the old directories. Will they remain?

I was hoping to remove the current "fix" folder (older version) to save space. Do you happen to know who is using it? We could potentially keep "fix" with a new name and move folks to it as needed....but really it's a very old fix copy now and folks should be moving away from it. I don't want to impact existing experiments though...might need to set a deadline or something.

Was looking to promote "fix_NEW" to "fix" and implement a new versioning within very soon. That versioning would allow for backwards compatibility and avoid the current issue in the future.

[GeorgeGayno-NOAA]

@KateFriedman-NOAA There are still folks who use the old directories. Will they remain?

I was hoping to remove the current "fix" folder (older version) to save space. Do you happen to know who is using it? We could potentially keep "fix" with a new name and move folks to it as needed....but really it's a very old fix copy now and folks should be moving away from it. I don't want to impact existing experiments though...might need to set a deadline or something.

Was looking to promote "fix_NEW" to "fix" and implement a new versioning within very soon. That versioning would allow for backwards compatibility and avoid the current issue in the future.

@JeffBeck-NOAA had some concerns.

[JeffBeck-NOAA]

Thanks, @GeorgeGayno-NOAA! Hi @KateFriedman-NOAA! My concern was just with the paths. The SRW App uses the old paths to source fixed files, so if the new path would use "fix_NEW", we'd just have to update that in our workflow. If "fix_NEW" were to become "fix", then it wouldn't be an issue. Thanks!

[KateFriedman-NOAA]

@JeffBeck-NOAA I did not know that the SRW app was using fix, this is great to know, thank you! Ok...sounds like I can proceed with the planned promotion of fix_NEW to fix in the near future. Correct me if I misunderstood. I plan to make a symlink from fix_NEW to fix when that happens to catch folks who still point to fix_NEW. My main concern in this promotion would be if changing the underlying fix files would hurt the systems using it (science-wise). If it's just a path issue (e.g. path suddenly missing) then I'll cover that with symlinks. How does that sound? Any remaining concerns? Thanks!

@GeorgeGayno-NOAA This PR is changing the FIX_DIR point from fix to fix_NEW. If that's to get new files in fix_NEW that don't exist in the current fix set then proceed. If not, you may want to back those path updates out and know that the promotion of fix_NEW to fix will cover that. I leave that up to you.

[JeffBeck-NOAA]

@KateFriedman-NOAA, if there is a symlink from "fix" to "fix_NEW" that should resolve any concerns we have with the SRW App. Then, once you promote "fix_NEW" to "fix", the symlink can be removed and we're back to just a single "fix" directory. That would work! However, if file names changed within the fixed directory, we'll need to update the workflow with the new names. Do you know of any modifications to the fixed files that we might need to be aware of? If it changes things scientifically, say through using higher-resolution fixed data, that'd be great to know!

[KateFriedman-NOAA]

@JeffBeck-NOAA The fix_NEW set is the fix set plus updates since late last year. The fix set hasn't been updated since then. Never meant for fix_NEW to stay un-promoted this long, lol!

FYI, my plans (if they go as I hope) would be to delete fix, rename fix_NEW to fix, and then symlink fix_NEW to fix. It would look like this afterwards:

fix
fix_NEW -> fix

I just did a diff between fix and fix_NEW to show general diffs between the two. I have put the diff output onto Hera here:
/scratch1/NCEPDEV/global/glopara/diff_fix_fix_NEW.out

Please take a look at the diffs to see if files that the SRW app use are impacted. For the most part, the differences are adding entire new fix file subsets to support GFSv17 into fix_NEW but some existing files were also updated. There were also a few renames (e.g. fix_wave_gfs -> fix_wave).

[JeffBeck-NOAA]

@KateFriedman-NOAA, that all sounds good and shouldn't disrupt anything with the SRW App. Based on your difference file, most things impacted are in "fix_fv3_fracoro", which we don't use. I do see some files were moved to fix_orog and a few files in fix_sfc_climo and fix_am may have been added for vegetation type and rrtmg, respectively. It's possible this will impact things, so I'll just need to run a test case with the App once the change takes place and modify accordingly. Thanks again for the info!

[GeorgeGayno-NOAA]

@JeffBeck-NOAA I did not know that the SRW app was using fix, this is great to know, thank you! Ok...sounds like I can proceed with the planned promotion of fix_NEW to fix in the near future. Correct me if I misunderstood. I plan to make a symlink from fix_NEW to fix when that happens to catch folks who still point to fix_NEW. My main concern in this promotion would be if changing the underlying fix files would hurt the systems using it (science-wise). If it's just a path issue (e.g. path suddenly missing) then I'll cover that with symlinks. How does that sound? Any remaining concerns? Thanks!

@GeorgeGayno-NOAA This PR is changing the FIX_DIR point from fix to fix_NEW. If that's to get new files in fix_NEW that don't exist in the current fix set then proceed. If not, you may want to back those path updates out and know that the promotion of fix_NEW to fix will cover that. I leave that up to you.

@KateFriedman-NOAA I was asked to add a link to fix_fv3_fracoro. Under fix_NEW, there are files that do not exist under fix. And according to @yangfanglin, these fix_NEW files were created with the latest MODIS data. (NOAA-EMC/global-workflow#863 (comment))

[KateFriedman-NOAA]

It's possible this will impact things, so I'll just need to run a test case with the App once the change takes place and modify accordingly.

@JeffBeck-NOAA Okie dokie...since my intention is to delete the current fix, would it be possible for you to run a test with the SRW app and point it fix_NEW before I make the change? Likely not making the fix_NEW to fix promotion for another week or so.

Note, more info, I keep snapshots of each fix set on HPSS, so the current fix is also on HPSS here:
/5year/NCEPDEV/emc-global/emc.glopara/fv3fix/fix_20211007.tar

I'll likely make a final tarball of fix before I do anything with it, just to be safe.

[KateFriedman-NOAA]

I was asked to add a link to fix_fv3_fracoro. Under fix_NEW, there are files that do not exist under fix. And according to @yangfanglin, these fix_NEW files were created with the latest MODIS data. (NOAA-EMC/global-workflow#863 (comment))

@GeorgeGayno-NOAA Okie dokie...sounds like you should proceed with this change then. After I have completed the promotion of fix_NEW to fix you'll want to change the paths again to remove "_NEW", although a symlink from fix_NEW -> fix will keep things from breaking. Then we'll be sticking with fix moving forward. Walter and I are working on a new versioning construct...I'll send details on that to component managers soon, it should be seamless at the start.