Merge branch 'develop'

CS-SI · Feb 3, 2025 · 577d588 · 577d588
2 parents 2199529 + cfc4bca
commit 577d588
Show file tree

Hide file tree

Showing 158 changed files with 40,147 additions and 3,154 deletions.
diff --git a/.gitignore b/.gitignore
@@ -94,6 +94,7 @@ ENV/
 
 # Autogenerated docs by sphinx
 docs/plugins_reference/generated/*.rst
+docs/drivers_generated/*.rst
 
 docs/notebooks/api_user_guide/*workspace*
 docs/notebooks/tutos/*workspace*

diff --git a/BREAKING_CHANGES.rst b/BREAKING_CHANGES.rst
@@ -3,14 +3,14 @@ Breaking changes
 
 Full changelog available in `Release history <changelog.html>`_.
 
-3.0.0b3
-+++++++
+v3.0.0b3
+++++++++
 
 * :meth:`~eodag.api.core.EODataAccessGateway.download` / :class:`~eodag.types.download_args.DownloadConf` parameters
   ``outputs_prefix`` and ``outputs_extension`` renamed to ``output_dir`` and ``output_extension``.
 
-3.0.0b1
-+++++++
+v3.0.0b1
+++++++++
 
 * :meth:`~eodag.api.core.EODataAccessGateway.search` method now returns only a
   :class:`~eodag.api.search_result.SearchResult` instead of a 2 values tuple. It can optionally store the estimated
@@ -45,14 +45,14 @@ Full changelog available in `Release history <changelog.html>`_.
     # install eodag with all available providers supported
     pip install "eodag[all-providers]"
 
-2.0b1
-+++++
+v2.0b1
+++++++
 
 - STAC API compliant REST server
 - Common configuration for STAC providers
 
-1.0
-+++
+v1.0
+++++
 
 - Adds product type search functionality
 - The cli arguments are now fully compliant with opensearch geo(bbox)/time extensions
diff --git a/CHANGES.rst b/CHANGES.rst
@@ -1,6 +1,33 @@
 Release history
 ===============
 
+v3.1.0b2 (2025-02-03)
+---------------------
+
+Core features and fixes
+^^^^^^^^^^^^^^^^^^^^^^^
+
+* Assets keys uniformization using drivers (:pull:`1488`)
+* ``ssl_verify`` setting for ``get_quicklook`` (:pull:`1490`, thanks `@tromain <https://github.com/tromain>`_)
+* Queryables merged by provider priority (:pull:`1431`)
+
+Providers and product types updates
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* ``geodes_s3`` as new provider (:pull:`1506`)
+* default search timeout to 20s (:pull:`1505`)
+* ``geodes`` ``relativeOrbitNumber`` property (:pull:`1499`) and numerical queryables fix (:pull:`1507`)
+
+Miscellaneous
+^^^^^^^^^^^^^
+
+* **[docs]** Updated tutorials using ``eodag-cube`` (:pull:`1436`) and minor fixes (:pull:`1498`)(:pull:`1500`)
+* **[style]** Typing update for generics (:pull:`1486`)
+* Various minor fixes and improvements (:pull:`1471`)(:pull:`1472`)(:pull:`1473`)(:pull:`1475`)(:pull:`1477`)
+  (:pull:`1479`)(:pull:`1480`)(:pull:`1483`)(:pull:`1492`)(:pull:`1503`)(:pull:`1504`)
+* External product types reference updates (:pull:`1460`)(:pull:`1478`)(:pull:`1484`)(:pull:`1487`)(:pull:`1493`)
+  (:pull:`1494`)
+
 v3.1.0b1 (2025-01-13)
 ---------------------
 

diff --git a/README.rst b/README.rst
@@ -187,7 +187,7 @@ An eodag instance can be exposed through a STAC compliant REST api from the comm
 
 .. code-block:: bash
 
-    docker run -p 5000:5000 --rm csspace/eodag-server:3.1.0b1
+    docker run -p 5000:5000 --rm csspace/eodag-server:3.1.0b2
 
 You can also browse over your STAC API server using `STAC Browser <https://github.com/radiantearth/stac-browser>`_.
 Simply run:

diff --git a/charts/eodag-server/Chart.yaml b/charts/eodag-server/Chart.yaml
@@ -1,5 +1,5 @@
 apiVersion: v2
-appVersion: 3.1.0b1
+appVersion: 3.1.0b2
 dependencies:
 - name: common
   repository:  oci://registry-1.docker.io/bitnamicharts
@@ -15,4 +15,4 @@ name: eodag-server
 sources:
 - https://github.com/CS-SI/eodag
 type: application
-version: 3.1.0b1
+version: 3.1.0b2
diff --git a/charts/eodag-server/values.yaml b/charts/eodag-server/values.yaml
@@ -408,7 +408,7 @@ productTypes: ""
 #     processingLevel: L2
 #     keywords: MUX,CBERS,CBERS-4,L2
 #     sensorType: OPTICAL
-#     license: proprietary
+#     license: other
 #     missionStartDate: "2014-12-07T00:00:00Z"
 #     title: CBERS-4 MUX Level-2
 

diff --git a/docs/_static/eodag_overview.png b/docs/_static/eodag_overview.png
diff --git a/docs/_static/eodag_stac_client.png b/docs/_static/eodag_stac_client.png
diff --git a/docs/_static/eodag_stac_server.png b/docs/_static/eodag_stac_server.png
diff --git a/docs/_static/params_mapping_extra.csv b/docs/_static/params_mapping_extra.csv
@@ -26,7 +26,7 @@ publishedAfter,,,:green:`queryable metadata`,,,,,,,,,,,,,,,,
 publishedBefore,,,:green:`queryable metadata`,,,,,,,,,,,,,,,,
 qs,metadata only,metadata only,,metadata only,,,metadata only,,,,,,,,,,,,
 quicklook,,,metadata only,,,metadata only,,metadata only,metadata only,metadata only,,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only
-relativeOrbitNumber,,,:green:`queryable metadata`,,,,,,,,,,,,,,,,
+relativeOrbitNumber,,,:green:`queryable metadata`,,,,,,,,,,:green:`queryable metadata`,,,,,,
 services,,,,,,,,,,,,,,,metadata only,,,,
 size,,,,,,,,,,,,:green:`queryable metadata`,,,,,,,
 storageStatus,metadata only,metadata only,metadata only,metadata only,,metadata only,metadata only,metadata only,metadata only,metadata only,,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only

diff --git a/docs/_static/product_types_information.csv b/docs/_static/product_types_information.csv
diff --git a/docs/add_product_type.rst b/docs/add_product_type.rst
@@ -21,7 +21,7 @@ example:
       processingLevel: L2
       keywords: AWFI,CBERS,CBERS-4,L2
       sensorType: OPTICAL
-      license: proprietary
+      license: other
       missionStartDate: "2014-12-07T00:00:00Z"
       title: CBERS-4 AWFI Level-2
 

diff --git a/docs/api_reference/eoproduct.rst b/docs/api_reference/eoproduct.rst
@@ -19,6 +19,13 @@ Download
    EOProduct.download
    EOProduct.get_quicklook
 
+Driver
+--------
+
+.. autosummary::
+
+   EOProduct.driver
+
 Conversion
 ----------
 
@@ -36,4 +43,4 @@ Interface
 
 
 .. autoclass:: eodag.api.product._product.EOProduct
-   :members: download, get_quicklook, as_dict, from_geojson, __geo_interface__
+   :members: driver, download, get_quicklook, as_dict, from_geojson, __geo_interface__
diff --git a/docs/api_reference/utils.rst b/docs/api_reference/utils.rst
@@ -24,6 +24,12 @@ Notebook
 .. automodule:: eodag.utils.notebook
    :members:
 
+S3
+----
+
+.. automodule:: eodag.utils.s3
+   :members:
+
 Misc
 ----
 

diff --git a/docs/conf.py b/docs/conf.py
@@ -23,7 +23,7 @@
 import re
 from datetime import datetime
 from importlib.metadata import metadata
-from typing import Any, Dict, List
+from typing import Any
 
 # -- General configuration ------------------------------------------------
 
@@ -168,7 +168,7 @@
     "custom.css",
 ]
 
-html_js_files: List[Any] = []
+html_js_files: list[Any] = []
 
 # Custom sidebar templates, must be a dictionary that maps document names
 # to template names.
@@ -184,7 +184,7 @@
 
 # -- Options for LaTeX output ---------------------------------------------
 
-latex_elements: Dict[str, str] = {
+latex_elements: dict[str, str] = {
     # The paper size ('letterpaper' or 'a4paper').
     #
     # 'papersize': 'letterpaper',
@@ -281,12 +281,19 @@ def _shorten_titles(dir_path):
                         r"\1",
                         file_content,
                     )
+                    # remove long prefix from drivers titles
+                    file_content = re.sub(
+                        r"(\.html\">|\"#\">|<h1>)eodag\.api\.product\.drivers\.[a-z0-9]+\.",
+                        r"\1",
+                        file_content,
+                    )
                     # write
                     file.seek(0)
                     file.write(file_content)
-                print(f"Plugins titles shortened in {file_path}")
+                print(f"Titles shortened in {file_path}")
 
     _shorten_titles(os.path.join(app.outdir, "plugins_reference"))
+    _shorten_titles(os.path.join(app.outdir, "drivers_generated"))
 
 
 def _html_page_context(app, pagename, templatename, context, doctree):

diff --git a/docs/drivers.rst b/docs/drivers.rst
@@ -0,0 +1,46 @@
+.. module:: eodag.api.product.drivers
+
+Drivers
+=======
+
+Drivers enable additional methods to be called on the :class:`~eodag.api.product._product.EOProduct`. They are set as
+:attr:`~eodag.api.product._product.EOProduct.driver` attribute of the :class:`~eodag.api.product._product.EOProduct`
+during its initialization, using some criteria to determine the most adapted driver. The first driver having its
+associated criteria matching will be used. If no driver is found, the :class:`~eodag.api.product.drivers.base.NoDriver`
+criteria is used.
+
+
+Criteria
+^^^^^^^^
+
+.. autoclass:: eodag.api.product.drivers.DriverCriteria
+    :members:
+
+.. autodata:: DRIVERS
+    :no-value:
+.. autodata:: LEGACY_DRIVERS
+    :no-value:
+
+
+Methods available
+^^^^^^^^^^^^^^^^^
+
+.. autoclass:: eodag.api.product.drivers.base.DatasetDriver
+   :members:
+   :member-order: bysource
+
+.. autoclass:: eodag.api.product.drivers.base.AssetPatterns
+   :members:
+
+Drivers Available
+^^^^^^^^^^^^^^^^^
+
+EODAG currently advertises the following drivers:
+
+.. autosummary::
+   :toctree: drivers_generated/
+
+   base.NoDriver
+   generic.GenericDriver
+   sentinel1.Sentinel1Driver
+   sentinel2.Sentinel2Driver
diff --git a/docs/getting_started_guide/product_types.rst b/docs/getting_started_guide/product_types.rst
@@ -21,7 +21,7 @@ The example below shows the catalog entry for the product type *Sentinel 2 Level
      platformSerialIdentifier: S2A,S2B
      processingLevel: L2
      sensorType: OPTICAL
-     license: proprietary
+     license: other
      title: SENTINEL2 Level-2A
      missionStartDate: "2015-06-23T00:00:00Z"
 

diff --git a/docs/getting_started_guide/providers.rst b/docs/getting_started_guide/providers.rst
@@ -24,6 +24,7 @@ Products from the following providers are made avaiable through ``eodag``:
 * `ecmwf <https://www.ecmwf.int/>`_: European Centre for Medium-Range Weather Forecasts
 * `eumetsat_ds <https://data.eumetsat.int>`_: EUMETSAT Data Store (European Organisation for the Exploitation of Meteorological Satellites)
 * `geodes <https://geodes.cnes.fr>`_: French National Space Agency (CNES) Earth Observation portal
+* `geodes_s3 <https://geodes.cnes.fr>`_: French National Space Agency (CNES) Earth Observation portal with internal s3 Datalake
 * `hydroweb_next <https://hydroweb.next.theia-land.fr>`_: hydroweb.next thematic hub for hydrology data access
 * `meteoblue <https://content.meteoblue.com/en/business-solutions/weather-apis/dataset-api>`_: Meteoblue forecast
 * `onda <https://www.onda-dias.eu/cms/>`_: Serco DIAS

diff --git a/docs/getting_started_guide/register.rst b/docs/getting_started_guide/register.rst
@@ -165,10 +165,16 @@ Then use the consumer key as `username` and the consumer secret as `password` fr
 
 ``geodes``
 ^^^^^^^^^^
-Go to `https://geodes-portal.cnes.fr <https://https://geodes-portal.cnes.fr>`_, then login or create an account by
+Go to `https://geodes-portal.cnes.fr <https://geodes-portal.cnes.fr>`_, then login or create an account by
 clicking on ``Log in`` in the top-right corner. Once logged-in, create an API key in the user settings page, and used it
 as *apikey* in EODAG provider auth credentials.
 
+``geodes_s3``
+^^^^^^^^^^^^^
+This provider is only available for CNES users. You need to be connected to the CNES network to access the data.
+Get credentials for internal Datalake and use them as ``aws_access_key_id``, ``aws_secret_access_key`` and
+``aws_session_token`` EODAG credentials.
+
 ``hydroweb_next``
 ^^^^^^^^^^^^^^^^^
 Go to `https://hydroweb.next.theia-land.fr <https://hydroweb.next.theia-land.fr>`_, then login or create an account by

diff --git a/docs/index.rst b/docs/index.rst
@@ -39,7 +39,8 @@ types (Sentinel 1, Sentinel 2, Sentinel 3, Landsat, etc.) that can be searched a
       `earth_search <https://www.element84.com/earth-search/>`_,
       `earth_search_gcs <https://cloud.google.com/storage/docs/public-datasets>`_,
       `ecmwf <https://www.ecmwf.int>`_,
-      `geodes <https://data.eumetsat.int>`_,
+      `geodes <https://geodes.cnes.fr>`_,
+      `geodes_s3 <https://geodes.cnes.fr>`_,
       `eumetsat_ds <https://data.eumetsat.int>`_,
       `hydroweb_next <https://hydroweb.next.theia-land.fr>`_,
       `meteoblue <https://content.meteoblue.com/en/business-solutions/weather-apis/dataset-api>`_,
@@ -49,7 +50,7 @@ types (Sentinel 1, Sentinel 2, Sentinel 3, Landsat, etc.) that can be searched a
       `sara <https://copernicus.nci.org.au>`_,
       `theia <https://theia.cnes.fr/atdistrib/rocket>`_,
       `usgs <https://earthexplorer.usgs.gov>`_,
-      `usgs_satapi_aws <https://geodes.cnes.fr>`_,
+      `usgs_satapi_aws <https://landsatlook.usgs.gov>`_,
       `wekeo_cmems <https://www.wekeo.eu>`_,
       `wekeo_ecmwf <https://www.wekeo.eu>`_,
       `wekeo_main <https://www.wekeo.eu>`_
@@ -132,6 +133,7 @@ Site contents
    add_provider
    add_product_type
    plugins
+   drivers
    params_mapping
    contribute
    changelog