[BUG] For GUNW workflow, HRRR Availability Checks Assume HRRR CONUS - needs AK too

[cmarshak]

#593 assumes HRRR CONUS.

What would need to be changed are the following pieces:

• spatial check of HRRR here:

  RAiDER/tools/RAiDER/cli/raider.py

  Lines 611 to 620 in 3dcdad6

  	if not iargs.file and iargs.bucket:
  	# only use GUNW ID for checking if HRRR available
  	iargs.file = aws.get_s3_file(iargs.bucket, iargs.bucket_prefix, '.nc')
  	if iargs.weather_model == 'HRRR' and (iargs.interpolate_time == 'azimuth_time_grid'):
  	file_name_str = str(iargs.file)
  	gunw_nc_name = file_name_str.split('/')[-1]
  	gunw_id = gunw_nc_name.replace('.nc', '')
  	if not RAiDER.aria.prepFromGUNW.check_hrrr_dataset_availablity_for_s1_azimuth_time_interpolation(gunw_id):
  	print('The required HRRR data for time-grid interpolation is not available; returning None and not modifying GUNW dataset')
  	return

  - i.e. whether it's CONUS or AK
• addition of model keyword argument (hrrrak is a Herbie model short name):

  RAiDER/tools/RAiDER/models/hrrr.py

  Lines 27 to 34 in 36a9fec

  	def check_hrrr_dataset_availability(dt: datetime) -> bool:
  	"""Note a file could still be missing within the models valid range"""
  	H = Herbie(dt,
  	model='hrrr',
  	product='nat',
  	fxx=0)
  	avail = (H.grib_source is not None)
  	return avail

• addition of model keyword argument here (hrrrak has a different model step time):

  RAiDER/tools/RAiDER/aria/prepFromGUNW.py

  Lines 50 to 71 in 36a9fec

  	def check_hrrr_dataset_availablity_for_s1_azimuth_time_interpolation(gunw_id: str) -> bool:
  	"""Determines if all the times for azimuth interpolation are available using Herbie; note that not all 1 hour times
  	are available within the said date range of HRRR.
  	Parameters
  	----------
  	gunw_id : str
  	Returns
  	-------
  	bool
  	"""
  	ref_acq_time = _get_acq_time_from_gunw_id(gunw_id, 'reference')
  	sec_acq_time = _get_acq_time_from_gunw_id(gunw_id, 'secondary')
  	model_step_hours = 1
  	ref_times_for_interp = get_times_for_azimuth_interpolation(ref_acq_time, model_step_hours)
  	sec_times_for_interp = get_times_for_azimuth_interpolation(sec_acq_time, model_step_hours)
  	ref_dataset_availability = list(map(check_hrrr_dataset_availability, ref_times_for_interp))
  	sec_dataset_availability = list(map(check_hrrr_dataset_availability, sec_times_for_interp))
  	return all(ref_dataset_availability) and all(sec_dataset_availability)

[cmarshak]

In the hyp3 API - this will be the error in the RAiDER step:

======Weather Model class object=====
Weather model time: 2022-02-22 15:00:00
Latitude resolution: 0.02702702702702703
Longitude resolution: 0.02702702702702703
Native projection: PROJCRS["unknown",BASEGEOGCRS["unknown",DATUM["unknown",ELLIPSOID["unknown",6371229,0,LENGTHUNIT["metre",1,ID["EPSG",9001]]]],PRIMEM["Greenwich",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8901]]],CONVERSION["unknown",METHOD["Polar Stereographic (variant B)",ID["EPSG",9829]],PARAMETER["Latitude of standard parallel",60,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8832]],PARAMETER["Longitude of origin",225,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8833]],PARAMETER["False easting",0,LENGTHUNIT["metre",1],ID["EPSG",8806]],PARAMETER["False northing",0,LENGTHUNIT["metre",1],ID["EPSG",8807]]],CS[Cartesian,2],AXIS["(E)",south,MERIDIAN[90,ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9122]]],ORDER[1],LENGTHUNIT["metre",1,ID["EPSG",9001]]],AXIS["(N)",south,MERIDIAN[180,ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9122]]],ORDER[2],LENGTHUNIT["metre",1,ID["EPSG",9001]]]]
ZMIN: -100.0
ZMAX: 26000.0
k1 = 0.776
k2 = 0.233
k3 = 3750.0
Humidity type = q
=====================================
Class name: hrrrak
Dataset: hrrrak
=====================================
A: []
B: []
Number of points in Lon/Lat = 117/141
Total number of grid points (3D): 940329
=====================================
Extent of the weather model is (xmin, ymin, xmax, ymax):-156.29, 57.66, -147.35, 61.73
Extent of the input is (xmin, ymin, xmax, ymax): -154.60, 58.75, -149.40, 60.70
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/conda/envs/RAiDER/lib/python3.12/site-packages/RAiDER/cli/__main__.py", line 44, in <module>
    main()
  File "/opt/conda/envs/RAiDER/lib/python3.12/site-packages/RAiDER/cli/__main__.py", line 40, in main
    process_entry_point.load()()
  File "/opt/conda/envs/RAiDER/lib/python3.12/site-packages/RAiDER/cli/raider.py", line 552, in calcDelaysGUNW
    cube_filenames = calcDelays([path_cfg])
                     ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/RAiDER/lib/python3.12/site-packages/RAiDER/cli/raider.py", line 305, in calcDelays
    weather_model_file = getWeatherFile(wfiles, times, t, model._Name, interp_method)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/RAiDER/lib/python3.12/site-packages/RAiDER/cli/raider.py", line 651, in getWeatherFile
    weather_model_file = combine_weather_files(
                         ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/RAiDER/lib/python3.12/site-packages/RAiDER/cli/raider.py", line 689, in combine_weather_files
    time_grid = get_time_grid_for_aztime_interp(datasets, t, model)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/RAiDER/lib/python3.12/site-packages/RAiDER/cli/raider.py", line 783, in get_time_grid_for_aztime_interp
    raise NotImplementedError('Azimuth Time is currently only implemented for HRRR')
NotImplementedError: Azimuth Time is currently only implemented for HRRR

[dbekaert]

@bbuzz31 could you have a look at this?

[cmarshak]

@bbuzz31 - I don't have time to review what you did carefully. I raised this issue specifically for GUNW via hyp3. I would be delighted if you completed this.

I am uncertain if what you did resolves the items I cited above (I just replaced the items with permalinks for easier navigation thought they do reference previous commits as things continue to get moved around). If the fix is related to something else, please leave the issue open.

One important test would be:

raider.py ++process calcDelaysGUNW -m HRRR -f <GUNW_ID>

https://search.asf.alaska.edu/#/?polygon=POINT(-162.189%2055.52)&resultsLoaded=true&granule=S1-GUNW-D-R-073-tops-20211021_20201002-170422-00164W_00055N-PP-90ba-v2_0_5-amplitude&dataset=SENTINEL-1%20INTERFEROGRAM%20(BETA)&zoom=3.000&center=-105.734,26.469

Though this would just be one item as some of the control flow is related to hyp3 specifically (via bucket/prefix arguments) when HRRR being requested at the present-day before HRRR becomes available.

Hope this clarifies.

[cmarshak]

@bbuzz31 @jlmaurer - I tried a sample Anchorage workflow last week with some standard testing and it looks OK. Here are some links:

https://hyp3-a19-jpl-contentbucket-1wfnatpznlg8b.s3.us-west-2.amazonaws.com/1cbb26e9-931e-4722-bd0d-4756eb36b14d/S1-GUNW-D-R-131-tops-20220210_20220129-163002-00154W_00060N-PP-3efd-v3_0_0.nc
https://hyp3-a19-jpl-contentbucket-1wfnatpznlg8b.s3.us-west-2.amazonaws.com/99cedbcb-ad10-4031-aac9-9883b21f7053/S1-GUNW-D-R-131-tops-20220222_20220210-163002-00155W_00059N-PP-5789-v3_0_0.nc

Check them - they look fine to my untrained eye.

However, this issue is not closed.

As we know with HRRR, there are data gaps. I spent a lot of time trying to make sure that when HRRR is not available, hyp3/Raider step still completes (as in the case when the time series continues to the present as mentioned). It's hard to evaluate if that is still happening since there are no tests.

[bbuzz31]

These look fine to me!