Improvements in copernicus-marine-client related code

[veenstrajelmer]

Follow up todo's from #630:

• re-enable CMEMS download per period #705
• check time extents after cmems preprocessing #707
• catch errors from catalog issues (maintread) and add solution to print >> reset function
• no_metadata_cache or remove_metadata_cache needed? >> probably not
• check if start/stoptimes are in dataset, cmc.open_dataset(dataset_id="cmems_mod_glo_phy-cur_anfc_0.083deg_PT6H-i", start_datetime="1921-01-01", end_datetime="1921-01-02") returns dataset with timestamp 2020-11-01 >> report at helpdesk. We do a bit of time range checking ourselves in dfmt.copernicusmarine_get_product(), but is not called with custom dataset_id. >> fixed in 0.10.3
• sub-daily freq for dfmt.download_CMEMS results in empty files #720
• use pytest tmp_dir instead of manually creating and deleting output folders #734
• assert n/values of timesteps in download tests #743
• instead of copernicusmarine_get_dataset_id, maybe get dataset_id from 'copernicus-marine describe --include-datasets --contains <search_token>' or (once) make global lists of varnames in different datasets (combine with retrieving tstart/tstop). >> wont do

[dbazin-eo]

@veenstrajelmer w/r on:

check if start/stoptimes are in dataset, cmc.open_dataset(dataset_id="cmems_mod_glo_phy-cur_anfc_0.083deg_PT6H-i", start_datetime="1921-01-01", end_datetime="1921-01-02") returns dataset with timestamp 2020-11-01 >> report at helpdesk.

It is not clear what you would expect from the toolbox with the open_dataset() function.

For your use case, it seems that you could simply retrieve the first and last values of the time coordinate of the dataset of your interest and check it against your desired date values.

FYI, we have introduced a subset_method option in the subset() function, providing a risk-free approach, resulting in an error if a requested value is not in the coordinate range of a dataset (with hints about the valid coordinate range). We are waiting for users' feedback and then we plan to add it in the open_dataset() function (flagged in [CMCS-211]):

import copernicus_marine_client as copernicusmarine
copernicusmarine.subset(
    dataset_id="cmems_mod_glo_phy-cur_anfc_0.083deg_PT6H-i", 
    start_datetime="1921-01-01",
    end_datetime="1921-01-02",
    subset_method='strict'
)
INFO - 2023-12-19T05:11:38Z - Selected dataset version is "202211"
---------------------------------------------------------------------------
CoordinatesOutOfDatasetBounds             Traceback (most recent call last)
[...]

Anyhow, based on xarray behavior, slicing over time with the reported values would create the following dataset using xarray.open_dataset():

import xarray
from copernicusmarine.future import store
dataset = xarray.open_dataset(store.model.global.sea_water_velocity.hourly.instantaneous)
dataset.sel(time=slice("1921-01-01", "1921-01-02"))
<xarray.Dataset>
Dimensions:    (elevation: 50, latitude: 2041, longitude: 4320, time: 0)
Coordinates:
  * elevation  (elevation) float32 -5.728e+03 -5.275e+03 ... -1.541 -0.494
  * latitude   (latitude) float32 -80.0 -79.92 -79.83 ... 89.83 89.92 90.0
  * longitude  (longitude) float32 -180.0 -179.9 -179.8 ... 179.8 179.8 179.9
  * time       (time) datetime64[ns]
Data variables:
    uo         (time, elevation, latitude, longitude) float32 ...
    vo         (time, elevation, latitude, longitude) float32 ...
Attributes: (12/13)
    Conventions:                   CF-1.6
    area:                          GLOBAL
    contact:                       servicedesk.cmems@mercator-ocean.eu
    credit:                        E.U. Copernicus Marine Service Information...
    institution:                   Mercator Ocean
    licence:                       http://marine.copernicus.eu/services-portf...
    ...                            ...
    product:                       GLOBAL_ANALYSISFORECAST_PHY_001_024
    product_user_manual:           http://marine.copernicus.eu/documents/PUM/...
    quality_information_document:  http://marine.copernicus.eu/documents/QUID...
    references:                    http://marine.copernicus.eu
    source:                        MERCATOR GLO12
    title:                         Instantaneous fields for product GLOBAL_AN...

Is it what you expected?

many thanks in advance for your feedback 🙏