cloud cover from NOAA GOES-16 example

[lindsayplatt]

I wanted to test out what cloud cover may look like with the river conditions visualization (request from Robert Mason & Athena Clark). I thought it would be best to test outside of that repo first because it is super complex and saw an opportunity to create a working example that could be leveraged to pull NOAA GOES data in the future.

cloud_timelapse.mp4

To build this cloud cover animation, you need to be on VPN (even though the NOAA AWS bucket is public, aws.s3 R package requires credentials to work and our creds only work on VPN). Then run the following:

library(scipiper)
setwd("cloud_cover")
scmake()

[cnell-usgs]

interesting to see the cloud cover animation. Immediately I googled where the cloudiest places in the US are. Are the data provided on the daily scale?

[lindsayplatt]

Oh good question! I was using 00 in the "hour" field as a hard-coded value but I just tried subbing in "12" and I get data from around noon on that same day. So, it seems like we could get hourly data this way.

time_coverage_start: 2020-10-01T12:01:16.1Z
time_coverage_end: 2020-10-01T12:03:53.4Z

[lindsayplatt]

The main point of putting this quick remake example together was to test out how cloud cover would look on top of the river conditions visualization. Since this is just a prototype, I did not want to get bogged down in some of the challenges with building that pipeline (it's old and could use a makeover). As such, I just made some minor tweaks to this pipeline as a temporary way to get cloud cover + the gage conditions. I am not committing these changes because it would make this pipeline not able to function independently. So instead, I am recording my steps below. The key is that the dates you are using for the cloud data will need to exist in the time frame that the last river conditions build included (I used 2020-09-01 to 2020-09-30 for this example).

cloud_timelapse.mp4

First, I manually ported over data from gage-conditions-gif.

1. Copied over 2_process/out/dv_stat_styles.rds as in/gage_data_all.rds
2. Copied over 1_fetch/out/site_locations.rds as in/gage_loc_all.rds

Then, I ran the following code to prepare data to be used in this pipeline

library(scipiper)
library(sf)
library(dplyr)

dates <- scmake("dates")
proj <- scmake("proj_str")
conus_sf <- scmake("conus_sf")

# Keep only sites in CONUS
sites_sf <- readRDS("in/gage_loc_all.rds") %>% 
  st_as_sf(coords = c("lon", "lat"), crs = "+proj=longlat +datum=WGS84") %>% 
  st_transform(proj) %>% 
  st_crop(conus_sf)

# Keep only data for this time period & CONUS sites
site_data <- readRDS("in/gage_data_all.rds") %>% 
  mutate(Date = as.Date(dateTime)) %>% 
  filter(Date %in% dates) %>% 
  filter(site_no %in% sites_sf$site_no) %>% 
  mutate(cex = cex*0.5) %>% # have to shrink these so they don't look crowded
  mutate(lwd = lwd*0.5) %>% # have to shrink these so they don't look crowded
  select(site_no, Date, bg, pch, cex, col, lwd)

# Save to use in pipeline
saveRDS(sites_sf, "in/site_loc_ready.rds")
saveRDS(site_data, "in/site_data_ready.rds")

Next, I had to go manually edit the src/map_clouds.R code.

Insert the following into the map_clouds.R code as directed.

# Add after conus_sf is plot, but before clouds (~ line 6-7)
this_date_sites <- readRDS("in/site_loc_ready.rds") %>% 
  left_join(filter(readRDS("in/site_data_ready.rds"), 
                   Date == as.Date(date)), by = "site_no")
plot(st_geometry(this_date_sites), add = TRUE, bg = this_date_sites$bg, 
     pch = this_date_sites$pch, cex = this_date_sites$cex, 
     col = this_date_sites$col, lwd = this_date_sites$lwd)

Lastly, run scmake()

The final video created should now have the gage data underneath the clouds. Be sure not to commit the changes to src/map_clouds.R or the new files added to in/.

[lindsayplatt]

I also just tried a cloud moisture version (Cloud Moisture Imagery, product code = ABI-L2-CMIPC) for good measure. The overlay goes from light to dark grey for less or more moisture. Closer to precip than just cloud cover would be. You can kind of see how rain came over MN, WI, MI and then gages turned darker blue, but it is difficult. Hard to see anything else that is happening at the same time though. I definitely am not paying much attention to the drought circles.

cloud_moisture_timelapse.mp4