Skip to content

Conversation

@erikvansebille
Copy link
Member

@erikvansebille erikvansebille commented Jul 7, 2025

This PR builds further on #2053 by adding an extra dataset (MITgcm_mds) and fixing some issues in the other datasets identified through the compare_datasets() function

Full output (nono-verbose) of the compare_datasets() method is

Comparing orig_copernicusmarine and ds_copernicusmarine

  Dataset attributes differ.
    Attribute 'title' only in orig_copernicusmarine
    Attribute 'institution' only in orig_copernicusmarine
    Attribute 'contact' only in orig_copernicusmarine
    Attribute 'Conventions' only in orig_copernicusmarine
    Attribute 'source' only in orig_copernicusmarine
    Attribute 'producer' only in orig_copernicusmarine
    Attribute 'copernicusmarine_version' only in orig_copernicusmarine
    Attribute 'credit' only in orig_copernicusmarine
    Attribute 'references' only in orig_copernicusmarine
Comparing orig_copernicusmarine_globcurrent and ds_copernicusmarine_globcurrent

  Dataset attributes differ.
    Attribute 'title' only in orig_copernicusmarine_globcurrent
    Attribute 'contact' only in orig_copernicusmarine_globcurrent
    Attribute 'institution' only in orig_copernicusmarine_globcurrent
    Attribute 'subset:source' only in orig_copernicusmarine_globcurrent
    Attribute 'subset:date' only in orig_copernicusmarine_globcurrent
    Attribute 'Conventions' only in orig_copernicusmarine_globcurrent
    Attribute 'subset:productId' only in orig_copernicusmarine_globcurrent
    Attribute 'subset:datasetId' only in orig_copernicusmarine_globcurrent
    Attribute 'references' only in orig_copernicusmarine_globcurrent
Comparing orig_NEMO_MOI_U and ds_NEMO_MOI_U

  Dataset attributes differ.
    Attribute 'source' only in orig_NEMO_MOI_U
    Attribute 'institution' only in orig_NEMO_MOI_U
    Attribute 'Conventions' only in orig_NEMO_MOI_U
    Attribute 'file_name' only in orig_NEMO_MOI_U
    Attribute 'TimeStamp' only in orig_NEMO_MOI_U
    Attribute 'references' only in orig_NEMO_MOI_U
  Variable names differ:
    orig_NEMO_MOI_U vars: ['sotkeavmu15', 'sotkeavmu30', 'sotkeavmu50', 'time_counter']
    ds_NEMO_MOI_U vars: []
    Common vars: ['deptht', 'nav_lat', 'nav_lon', 'sotkeavmu1', 'vozocrtx', 'x', 'y']
Comparing orig_NEMO_MOI_V and ds_NEMO_MOI_V

  Dataset attributes differ.
    Attribute 'source' only in orig_NEMO_MOI_V
    Attribute 'institution' only in orig_NEMO_MOI_V
    Attribute 'Conventions' only in orig_NEMO_MOI_V
    Attribute 'file_name' only in orig_NEMO_MOI_V
    Attribute 'TimeStamp' only in orig_NEMO_MOI_V
    Attribute 'references' only in orig_NEMO_MOI_V
  Variable names differ:
    orig_NEMO_MOI_V vars: ['time_counter']
    ds_NEMO_MOI_V vars: []
    Common vars: ['deptht', 'nav_lat', 'nav_lon', 'vomecrty', 'x', 'y']
Comparing orig_CESM and ds_CESM

  Dataset attributes differ.
    Attribute 'title' only in orig_CESM
    Attribute 'start_time' only in orig_CESM
    Attribute 'tavg_sum' only in orig_CESM
    Attribute 'revision' only in orig_CESM
    Attribute 'history' only in orig_CESM
    Attribute 'Conventions' only in orig_CESM
    Attribute 'contents' only in orig_CESM
    Attribute 'nsteps_total' only in orig_CESM
    Attribute 'source' only in orig_CESM
    Attribute 'cell_methods' only in orig_CESM
    Attribute 'calendar' only in orig_CESM
  Dimension names differ:
    orig_CESM dims: ['d2', 'lat_aux_grid', 'moc_comp', 'moc_z', 'nchar', 'nlat', 'nlon', 'time', 'transport_comp', 'transport_reg', 'z_t', 'z_t_150m', 'z_w', 'z_w_bot', 'z_w_top']
    ds_CESM dims: ['nlat', 'nlon', 'time', 'z_t', 'z_w_top']
  Variable names differ:
    orig_CESM vars: ['ADVS', 'ADVS_ISOP', 'ADVS_SUBM', 'ADVT', 'ADVT_ISOP', 'ADVT_SUBM', 'ANGLE', 'ANGLET', 'BSF', 'CFC11', 'CFC12', 'CFC_ATM_PRESS', 'CFC_IFRAC', 'CFC_XKW', 'DIA_DEPTH', 'DIA_IMPVF_TEMP', 'DXT', 'DXU', 'DYT', 'DYU', 'HDIFB_TEMP', 'HDIFS', 'HDIFT', 'HLS_SUBM', 'HOR_DIFF', 'HT', 'HTE', 'HTN', 'HU', 'HUS', 'HUW', 'IAGE', 'IFRAC', 'INT_DEPTH', 'IOFF_F', 'KAPPA_ISOP', 'KAPPA_THIC', 'KMT', 'KMU', 'KPP_SRC_TEMP', 'KVMIX', 'KVMIX_M', 'MOC', 'N_HEAT', 'N_SALT', 'PD', 'PV', 'Q', 'QSW_3D', 'QSW_HBL', 'QSW_HTP', 'REGION_MASK', 'RHO', 'RHO_VINT', 'SALT', 'SFWF_WRST', 'SSH2', 'STF_CFC11', 'STF_CFC12', 'SU', 'SV', 'T0_Kelvin', 'TAREA', 'TAUX2', 'TAUY2', 'TBLT', 'TEMP', 'TEND_SALT', 'TEND_TEMP', 'TLAT', 'TLONG', 'TLT', 'TMXL', 'TPOWER', 'UAREA', 'UES', 'UET', 'UISOP', 'USUBM', 'UVEL2', 'VDC_S', 'VDC_T', 'VISOP', 'VNS', 'VNS_ISOP', 'VNS_SUBM', 'VNT', 'VNT_ISOP', 'VNT_SUBM', 'VSUBM', 'VVC', 'VVEL2', 'WISOP', 'WSUBM', 'WTS', 'WTT', 'WTT_ISOP', 'WTT_SUBM', 'WVEL2', 'XBLT', 'XMXL', 'cp_air', 'cp_sw', 'dTEMP_NEG_2D', 'dTEMP_POS_2D', 'days_in_norm_year', 'dz', 'dzw', 'fwflux_factor', 'grav', 'heat_to_PW', 'hflux_factor', 'lat_aux_grid', 'latent_heat_fusion', 'latent_heat_vapor', 'mass_to_Sv', 'moc_components', 'moc_z', 'momentum_factor', 'nsurface_t', 'nsurface_u', 'ocn_ref_salinity', 'omega', 'ppt_to_salt', 'radius', 'rho_air', 'rho_fw', 'rho_sw', 'salinity_factor', 'salt_to_Svppt', 'salt_to_mmday', 'salt_to_ppt', 'sea_ice_salinity', 'sflux_factor', 'sound', 'stefan_boltzmann', 'time_bound', 'transport_components', 'transport_regions', 'vonkar', 'z_t_150m', 'z_w', 'z_w_bot']
    ds_CESM vars: []
    Common vars: ['ULAT', 'ULONG', 'UVEL', 'VVEL', 'WVEL', 'time', 'z_t', 'z_w_top']
Comparing orig_MITgcm_netcdf and ds_MITgcm_netcdf

  Dataset attributes differ.
    Attribute 'MITgcm_version' only in orig_MITgcm_netcdf
    Attribute 'Ny' only in orig_MITgcm_netcdf
    Attribute 'Nx' only in orig_MITgcm_netcdf
    Attribute 'MITgcm_mnc_ver' only in orig_MITgcm_netcdf
    Attribute 'nPy' only in orig_MITgcm_netcdf
    Attribute 'nSy' only in orig_MITgcm_netcdf
    Attribute 'OLx' only in orig_MITgcm_netcdf
    Attribute 'sNy' only in orig_MITgcm_netcdf
    Attribute 'MITgcm_tag_id' only in orig_MITgcm_netcdf
    Attribute 'nSx' only in orig_MITgcm_netcdf
    Attribute 'build_user' only in orig_MITgcm_netcdf
    Attribute 'build_date' only in orig_MITgcm_netcdf
    Attribute 'sNx' only in orig_MITgcm_netcdf
    Attribute 'build_host' only in orig_MITgcm_netcdf
    Attribute 'OLy' only in orig_MITgcm_netcdf
    Attribute 'Nr' only in orig_MITgcm_netcdf
    Attribute 'nPx' only in orig_MITgcm_netcdf
    Attribute 'MITgcm_URL' only in orig_MITgcm_netcdf
  Variable names differ:
    orig_MITgcm_netcdf vars: ['Eta', 'S', 'iter']
    ds_MITgcm_netcdf vars: []
    Common vars: ['T', 'Temp', 'U', 'V', 'W', 'X', 'Xp1', 'Y', 'Yp1', 'Z', 'Zl']
Comparing orig_ERA5_wind and ds_ERA5_wind

  Dataset attributes differ.
    Attribute 'Conventions' only in orig_ERA5_wind
    Attribute 'history' only in orig_ERA5_wind
Comparing orig_FES_tides and ds_FES_tides

  Dataset attributes differ.
    Attribute 'title' only in orig_FES_tides
    Attribute 'geospatial_lon_units' only in orig_FES_tides
    Attribute 'license' only in orig_FES_tides
    Attribute 'product_version' only in orig_FES_tides
    Attribute 'date_created' only in orig_FES_tides
    Attribute 'geospatial_lon_resolution' only in orig_FES_tides
    Attribute 'cdm_data_type' only in orig_FES_tides
    Attribute 'credits' only in orig_FES_tides
    Attribute 'references' only in orig_FES_tides
    Attribute 'contact' only in orig_FES_tides
    Attribute 'summary' only in orig_FES_tides
    Attribute 'history' only in orig_FES_tides
    Attribute 'geospatial_lat_units' only in orig_FES_tides
    Attribute 'project' only in orig_FES_tides
    Attribute 'Conventions' only in orig_FES_tides
    Attribute 'geospatial_lat_resolution' only in orig_FES_tides
  Dimension names differ:
    orig_FES_tides dims: ['lat', 'lon', 'nv']
    ds_FES_tides dims: ['lat', 'lon']
  Variable names differ:
    orig_FES_tides vars: ['crs', 'lat_bnds', 'lon_bnds', 'nv']
    ds_FES_tides vars: []
    Common vars: ['Ua', 'Ug', 'lat', 'lon']
Comparing orig_hycom_espc and ds_hycom_espc

  Dataset attributes differ.
    Attribute 'generating_model' only in orig_hycom_espc
    Attribute 'input_data_source' only in orig_hycom_espc
    Attribute 'institution' only in orig_hycom_espc
    Attribute 'classification_authority' only in orig_hycom_espc
    Attribute 'field_type' only in orig_hycom_espc
    Attribute 'created_on' only in orig_hycom_espc
    Attribute 'downgrade_date' only in orig_hycom_espc
    Attribute 'area_name' only in orig_hycom_espc
    Attribute 'grid_name' only in orig_hycom_espc
    Attribute 'contact' only in orig_hycom_espc
    Attribute 'history' only in orig_hycom_espc
    Attribute 'reference' only in orig_hycom_espc
    Attribute 'Conventions' only in orig_hycom_espc
    Attribute 'NCO' only in orig_hycom_espc
    Attribute 'source' only in orig_hycom_espc
    Attribute 'distribution_statement' only in orig_hycom_espc
    Attribute 'time_origin' only in orig_hycom_espc
    Attribute 'operational_status' only in orig_hycom_espc
    Attribute 'classification_level' only in orig_hycom_espc
Comparing orig_ecco4 and ds_ecco4

  Dataset attributes differ.
    Attribute 'publisher_name' only in orig_ecco4
    Attribute 'title' only in orig_ecco4
    Attribute 'creator_email' only in orig_ecco4
    Attribute 'license' only in orig_ecco4
    Attribute 'processing_level' only in orig_ecco4
    Attribute 'date_modified' only in orig_ecco4
    Attribute 'publisher_url' only in orig_ecco4
    Attribute 'creator_name' only in orig_ecco4
    Attribute 'creator_institution' only in orig_ecco4
    Attribute 'instrument_vocabulary' only in orig_ecco4
    Attribute 'geospatial_vertical_positive' only in orig_ecco4
    Attribute 'product_time_coverage_start' only in orig_ecco4
    Attribute 'id' only in orig_ecco4
    Attribute 'product_name' only in orig_ecco4
    Attribute 'metadata_link' only in orig_ecco4
    Attribute 'comment' only in orig_ecco4
    Attribute 'geospatial_lon_resolution' only in orig_ecco4
    Attribute 'references' only in orig_ecco4
    Attribute 'date_metadata_modified' only in orig_ecco4
    Attribute 'keywords_vocabulary' only in orig_ecco4
    Attribute 'history' only in orig_ecco4
    Attribute 'geospatial_lat_units' only in orig_ecco4
    Attribute 'geospatial_lat_min' only in orig_ecco4
    Attribute 'geospatial_lon_max' only in orig_ecco4
    Attribute 'geospatial_vertical_resolution' only in orig_ecco4
    Attribute 'project' only in orig_ecco4
    Attribute 'source' only in orig_ecco4
    Attribute 'coordinates_comment' only in orig_ecco4
    Attribute 'creator_type' only in orig_ecco4
    Attribute 'author' only in orig_ecco4
    Attribute 'geospatial_vertical_units' only in orig_ecco4
    Attribute 'platform_vocabulary' only in orig_ecco4
    Attribute 'program' only in orig_ecco4
    Attribute 'geospatial_lon_units' only in orig_ecco4
    Attribute 'geospatial_lon_min' only in orig_ecco4
    Attribute 'uuid' only in orig_ecco4
    Attribute 'institution' only in orig_ecco4
    Attribute 'product_time_coverage_end' only in orig_ecco4
    Attribute 'product_version' only in orig_ecco4
    Attribute 'time_coverage_duration' only in orig_ecco4
    Attribute 'geospatial_lat_max' only in orig_ecco4
    Attribute 'geospatial_vertical_max' only in orig_ecco4
    Attribute 'geospatial_bounds_crs' only in orig_ecco4
    Attribute 'publisher_institution' only in orig_ecco4
    Attribute 'acknowledgement' only in orig_ecco4
    Attribute 'standard_name_vocabulary' only in orig_ecco4
    Attribute 'date_created' only in orig_ecco4
    Attribute 'publisher_email' only in orig_ecco4
    Attribute 'cdm_data_type' only in orig_ecco4
    Attribute 'naming_authority' only in orig_ecco4
    Attribute 'time_coverage_start' only in orig_ecco4
    Attribute 'platform' only in orig_ecco4
    Attribute 'summary' only in orig_ecco4
    Attribute 'Conventions' only in orig_ecco4
    Attribute 'publisher_type' only in orig_ecco4
    Attribute 'geospatial_vertical_min' only in orig_ecco4
    Attribute 'geospatial_lat_resolution' only in orig_ecco4
    Attribute 'date_issued' only in orig_ecco4
    Attribute 'creator_url' only in orig_ecco4
    Attribute 'keywords' only in orig_ecco4
    Attribute 'time_coverage_end' only in orig_ecco4
    Attribute 'time_coverage_resolution' only in orig_ecco4
  Dimension names differ:
    orig_ecco4 dims: ['i', 'i_g', 'j', 'j_g', 'k', 'k_l', 'k_p1', 'k_u', 'nb', 'nv', 'tile', 'time']
    ds_ecco4 dims: ['i', 'i_g', 'j', 'j_g', 'k', 'k_l', 'tile', 'time']
  Variable names differ:
    orig_ecco4 vars: ['XC_bnds', 'YC_bnds', 'Z_bnds', 'Zp1', 'Zu', 'k_p1', 'k_u', 'time_bnds']
    ds_ecco4 vars: []
    Common vars: ['UVEL', 'VVEL', 'WVEL', 'XC', 'XG', 'YC', 'YG', 'Z', 'Zl', 'i', 'i_g', 'j', 'j_g', 'k', 'k_l', 'tile', 'time']
Comparing orig_CROCO_idealized and ds_CROCO_idealized

  Dataset attributes differ.
    Attribute 'title' only in orig_CROCO_idealized
    Attribute 'Tcline_units' only in orig_CROCO_idealized
    Attribute 'dt' only in orig_CROCO_idealized
    Attribute 'ndtfast' only in orig_CROCO_idealized
    Attribute 'dtfast' only in orig_CROCO_idealized
    Attribute 'Tcoef_units' only in orig_CROCO_idealized
    Attribute 'levbfrc' only in orig_CROCO_idealized
    Attribute 'type' only in orig_CROCO_idealized
    Attribute 'gamma2' only in orig_CROCO_idealized
    Attribute 'theta_b' only in orig_CROCO_idealized
    Attribute 'T0_units' only in orig_CROCO_idealized
    Attribute 'Cs_r' only in orig_CROCO_idealized
    Attribute 'sc_w_expl' only in orig_CROCO_idealized
    Attribute 'Cs_r_expl' only in orig_CROCO_idealized
    Attribute 'theta_s_expl' only in orig_CROCO_idealized
    Attribute 'Tcline' only in orig_CROCO_idealized
    Attribute 'Scoef_units' only in orig_CROCO_idealized
    Attribute 'Akv_bak_expl' only in orig_CROCO_idealized
    Attribute 'levsfrc' only in orig_CROCO_idealized
    Attribute 'sc_r' only in orig_CROCO_idealized
    Attribute 'tnu2_expl' only in orig_CROCO_idealized
    Attribute 'T0_expl' only in orig_CROCO_idealized
    Attribute 'ntimes' only in orig_CROCO_idealized
    Attribute 'sc_r_expl' only in orig_CROCO_idealized
    Attribute 'visc2_expl' only in orig_CROCO_idealized
    Attribute 'hc_units' only in orig_CROCO_idealized
    Attribute 'theta_s' only in orig_CROCO_idealized
    Attribute 'Akv_bak_units' only in orig_CROCO_idealized
    Attribute 'Akt_bak_units' only in orig_CROCO_idealized
    Attribute 'rho0' only in orig_CROCO_idealized
    Attribute 'rdrg_expl' only in orig_CROCO_idealized
    Attribute 'S0' only in orig_CROCO_idealized
    Attribute 'Cs_w_expl' only in orig_CROCO_idealized
    Attribute 'Akt_bak_expl' only in orig_CROCO_idealized
    Attribute 'Akt_bak' only in orig_CROCO_idealized
    Attribute 'S0_units' only in orig_CROCO_idealized
    Attribute 'visc2' only in orig_CROCO_idealized
    Attribute 'his_file' only in orig_CROCO_idealized
    Attribute 'tnu2' only in orig_CROCO_idealized
    Attribute 'visc2_units' only in orig_CROCO_idealized
    Attribute 'Tcoef_expl' only in orig_CROCO_idealized
    Attribute 'Tcline_expl' only in orig_CROCO_idealized
    Attribute 'hc' only in orig_CROCO_idealized
    Attribute 'sc_w' only in orig_CROCO_idealized
    Attribute 'rdrg_units' only in orig_CROCO_idealized
    Attribute 'Akv_bak' only in orig_CROCO_idealized
    Attribute 'date' only in orig_CROCO_idealized
    Attribute 'S0_expl' only in orig_CROCO_idealized
    Attribute 'rdrg' only in orig_CROCO_idealized
    Attribute 'Tcoef' only in orig_CROCO_idealized
    Attribute 'CPP-options' only in orig_CROCO_idealized
    Attribute 'rho0_units' only in orig_CROCO_idealized
    Attribute 'gamma2_expl' only in orig_CROCO_idealized
    Attribute 'hc_expl' only in orig_CROCO_idealized
    Attribute 'R0_expl' only in orig_CROCO_idealized
    Attribute 'T0' only in orig_CROCO_idealized
    Attribute 'Scoef_expl' only in orig_CROCO_idealized
    Attribute 'R0_units' only in orig_CROCO_idealized
    Attribute 'Scoef' only in orig_CROCO_idealized
    Attribute 'R0' only in orig_CROCO_idealized
    Attribute 'SRCS' only in orig_CROCO_idealized
    Attribute 'tnu2_units' only in orig_CROCO_idealized
    Attribute 'nwrt' only in orig_CROCO_idealized
    Attribute 'rho0_expl' only in orig_CROCO_idealized
    Attribute 'rst_file' only in orig_CROCO_idealized
    Attribute 'theta_b_expl' only in orig_CROCO_idealized
    Attribute 'Cs_w' only in orig_CROCO_idealized
    Attribute 'levsfrc_expl' only in orig_CROCO_idealized
    Attribute 'levbfrc_expl' only in orig_CROCO_idealized
  Dimension names differ:
    orig_CROCO_idealized dims: ['auxil', 'eta_rho', 'eta_v', 's_rho', 's_w', 'time', 'xi_rho', 'xi_u']
    ds_CROCO_idealized dims: ['eta_rho', 'eta_v', 's_rho', 's_w', 'time', 'xi_rho', 'xi_u']
  Variable names differ:
    orig_CROCO_idealized vars: ['AKt', 'Cs_r', 'Vtransform', 'el', 'f', 'omega', 'pm', 'pn', 'rho', 'sc_r', 'sc_w', 'scrum_time', 'spherical', 'temp', 'time_step', 'ubar', 'vbar', 'xl']
    ds_CROCO_idealized vars: []
    Common vars: ['Cs_w', 'eta_rho', 'eta_v', 'h', 'hc', 's_rho', 's_w', 'time', 'u', 'v', 'w', 'x_rho', 'xi_rho', 'xi_u', 'y_rho', 'zeta']
    Attributes differ.
      Attribute 'c_grid_dynamic_range' differs for 'xi_u':
        orig_CROCO_idealized: 2:60
        ds_CROCO_idealized: 2:29
    Attributes differ.
      Attribute 'c_grid_dynamic_range' differs for 'eta_v':
        orig_CROCO_idealized: 2:50
        ds_CROCO_idealized: 2:59
    Attributes differ.
      Attribute 'c_grid_dynamic_range' differs for 'xi_rho':
        orig_CROCO_idealized: 2:61
        ds_CROCO_idealized: 2:30
    Attributes differ.
      Attribute 'c_grid_dynamic_range' differs for 'eta_rho':
        orig_CROCO_idealized: 2:51
        ds_CROCO_idealized: 2:60
Comparing orig_MITgcm_mds and ds_MITgcm_mds

  Dataset attributes differ.
    Attribute 'title' only in orig_MITgcm_mds
    Attribute 'source' only in orig_MITgcm_mds
    Attribute 'Conventions' only in orig_MITgcm_mds
    Attribute 'history' only in orig_MITgcm_mds
  Dimension names differ:
    orig_MITgcm_mds dims: ['XC', 'XG', 'YC', 'YG', 'Z', 'Zl', 'Zp1', 'Zu', 'time']
    ds_MITgcm_mds dims: ['XC', 'XG', 'YC', 'YG', 'Z', 'Zl', 'time']
  Variable names differ:
    orig_MITgcm_mds vars: ['Depth', 'PHrefC', 'PHrefF', 'Zp1', 'Zu', 'drC', 'drF', 'dxC', 'dxF', 'dxG', 'dxV', 'dyC', 'dyF', 'dyG', 'dyU', 'hFacC', 'hFacS', 'hFacW', 'iter', 'maskC', 'maskS', 'maskW', 'rA', 'rAs', 'rAw', 'rAz', 'rhoRef']
    ds_MITgcm_mds vars: []
    Common vars: ['S', 'T', 'U', 'V', 'W', 'XC', 'XG', 'YC', 'YG', 'Z', 'Zl', 'time']

@erikvansebille erikvansebille merged commit 68d6a1a into v4-dev Jul 7, 2025
8 of 9 checks passed
@erikvansebille erikvansebille deleted the circulation_models_part2 branch July 7, 2025 14:47
@github-project-automation github-project-automation bot moved this from Backlog to Done in Parcels development Jul 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants