Ideas for upcoming developent

[betolink]

SciPy was a blast! a lot of interesting developments were presented, here are some ideas I thought about while I attended some of these talks:

Dataviz:

• Anywidget for CMR metadata representation.
  • Our current metadata representation is outdated and is heavy on the notebook side, I think Anywidget is the way to go to present our results, perhaps taking pystac-client does as baseline and go from there.
  • Results from collections are a bit different from results from granules, for example we can have an HTML widget for a collection that tells us their S3 credential endpoints, geographic and temporal boundaries perhaps links to DOIs etc. Granules will have a collapsible accordion for links and we could even filter them using an input box.
• Lonboard to visualize granule footprints
  • Similar to df.explore() in geopandas, we could implement this feature using Lonboard since it can handle thousands or potentially millions of geometries, the tricky part would be how to serialize the results from CMR into something like a geo-parquet dataframe and then pass it to Loneboard to for example visualize L2 satellite passes at the granule level!
  • With metadata rich results we could use this to visualize granule properties at scale without having to touch the data (e.g. cloud coverage for Landsat, HLS etc)

Access:

• VirtualiZarr dmrpp fast access
  • Ayush started to work on this integration in Opening virtual datasets with NASA dmrpp files #605, once completed it will bring a lot of benefits specially to those OPeNDAP-backed collections, we talked a bit about having an end to end example with some L2 datasets from SWOT and ICESat-2 to demonstrate performance gains.
• Format-aware caching:
  • This is mainly to override fsspec defaults, and let the user decide what cache and cache size we need to use to stream mission data, what I was calling "smart_open" I think this is critical for rapid access to mission data formats (.nc, .h5 etc)
• GraphQL "lean queries": we need to asses if we need all the information we get from querying UMM vs querying only what we need to a) represent the data and b) access it. @briannapagan started a conversation on this in python_cmr