Remove eofs (#1621)

### What kind of change does this PR introduce? * The `eofs` library has been removed from the optional dependencies. * The empirical orthogonal functionality (`xclim.sdba.properties.first_eof` and associated test) has been removed from the code base. * CI builds no longer test `eofs` functionality. ### Does this PR introduce a breaking change? Yes. `eofs` has been completely removed since this violated its GPLv3 license. Associated functionality is reproducible by users ### Other information: Should we be offering users a note about how to migrate? I feel like it's not necessary.
Ouranosinc · Jan 25, 2024 · 36d49e9 · 36d49e9
2 parents a569e91 + 5dfa324
commit 36d49e9
Show file tree

Hide file tree

Showing 8 changed files with 23 additions and 134 deletions.
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -115,10 +115,10 @@ jobs:
  strategy:
  matrix:
  include:
- - tox-env: py38-coverage-eofs
+ - tox-env: py38-coverage
  python-version: "3.8"
  markers: -m 'not slow'
- - tox-env: py39-coverage-sbck-eofs
+ - tox-env: py39-coverage-sbck
  python-version: "3.9"
  markers: -m 'not slow'
  - tox-env: py310-coverage # No markers -- includes slow tests

diff --git a/CHANGES.rst b/CHANGES.rst
@@ -24,6 +24,7 @@ Breaking changes
 * `bump2version` has been replaced with `bump-my-version` to bump the version number using configurations set in the `pyproject.toml` file. (:issue:`1557`, :pull:`1569`).
 * `xclim`'s units registry and units formatting are now extended from `cf-xarray`. The exponent sign "^" is now never added in the ``units`` attribute. For example, square meters are given as "m2" instead of "m^2" by xclim, both are still accepted as input. (:issue:`1010`, :pull:`1590`).
 * `yamale` is now listed as a core dependency (was previously listed in the `dev` installation recipe). (:issue:`1595`, :pull:`1596`).
+* Due to a licensing limitation, the calculation of empirical orthogonal function based on `eofs` (``xclim.sdba.properties.first_eof``) has been removed from `xclim`. (:issue:`1620`, :pull:`1621`).
 
 Bug fixes
 ^^^^^^^^^

diff --git a/docs/installation.rst b/docs/installation.rst
@@ -11,7 +11,7 @@ To install `xclim` via `pip`, run this command in your terminal:
 
 .. code-block:: shell
 
- $ pip install xclim
+ $ python -m pip install xclim
 
 If you don't have `pip`_ installed, this `Python installation guide`_ can guide you through the process.
 
@@ -55,7 +55,7 @@ Both of these libraries are available on PyPI and conda-forge:
 
 .. code-block:: shell
 
- $ pip install flox clisops
+ $ python -m pip install flox clisops
  # Or, alternatively:
  $ conda install -c conda-forge flox clisops
 
@@ -70,7 +70,7 @@ For convenience, these libraries can be installed alongside `xclim` using the fo
 
 .. code-block:: shell
 
- $ pip install -r requirements_upstream.txt
+ $ python -m pip install -r requirements_upstream.txt
 
 Or, alternatively:
 
@@ -105,17 +105,8 @@ Afterwards, `SBCK` can be installed from PyPI using `pip`:
 
 .. code-block:: shell
 
- $ pip install SBCK
+ $ python -m pip install pybind11 sbck
 
-Another experimental function :py:indicator:`xclim.sdba.property.first_eof` makes use of the `eofs`_ library, which is available on both PyPI and conda-forge:
-
-.. code-block:: shell
-
- $ pip install eofs
- # or alternatively,
- $ conda install -c conda-forge eofs
-
-.. _eofs: https://ajdawson.github.io/eofs/
 .. _SBCK: https://github.com/yrobink/SBCK
 .. _Eigen3: https://eigen.tuxfamily.org/index.php
 
@@ -145,7 +136,7 @@ Once you have extracted a copy of the source, you can install it with pip:
 
 .. code-block:: shell
 
- $ pip install -e ".[dev]"
+ $ python -m pip install -e ".[dev]"
 
 Alternatively, you can also install a local development copy via `flit`_:
 
@@ -160,10 +151,10 @@ Alternatively, you can also install a local development copy via `flit`_:
 Creating a Conda environment
 ----------------------------
 
-To create a conda environment including `xclim`'s dependencies and several optional libraries (notably: `clisops`, `eigen`, `eofs`, and `flox`) and development dependencies, run the following command from within your cloned repo:
+To create a conda environment including `xclim`'s dependencies and several optional libraries (notably: `clisops`, `eigen`, `sbck`, and `flox`) and development dependencies, run the following command from within your cloned repo:
 
 .. code-block:: console
 
  $ conda env create -n my_xclim_env python=3.8 --file=environment.yml
  $ conda activate my_xclim_env
- (my_xclim_env) $ pip install -e .
+ (my_xclim_env) $ python -m pip install -e .
diff --git a/docs/references.bib b/docs/references.bib
@@ -1835,26 +1835,6 @@ @article{roy_extremeprecip_2023
  year = {2023},
 }
 
-@article{dawson_eofs_2016,
- title = {eofs: {A} {Library} for {EOF} {Analysis} of {Meteorological}, {Oceanographic}, and {Climate} {Data}},
- volume = {4},
- issn = {2049-9647},
- shorttitle = {eofs},
- url = {https://openresearchsoftware.metajnl.com/article/10.5334/jors.122/},
- doi = {10.5334/jors.122},
- abstract = {Article: eofs: A Library for EOF Analysis of Meteorological, Oceanographic, and Climate Data},
- language = {eng},
- number = {1},
- urldate = {2022-11-11},
- journal = {Journal of Open Research Software},
- author = {Dawson, Andrew},
- month = apr,
- year = {2016},
- note = {Number: 1
-Publisher: Ubiquity Press},
- pages = {e14},
-}
-
 @article{francois_multivariate_2020,
  title = {Multivariate bias corrections of climate simulations: which benefits for which losses?},
  volume = {11},

diff --git a/environment.yml b/environment.yml
@@ -27,7 +27,6 @@ dependencies:
  - xarray >=2022.06.0,<2023.11.0
  - yamale
  # Extras
- - eofs
  - flox
  # Testing and development dependencies
  - black >=22.12

diff --git a/tests/test_sdba/test_properties.py b/tests/test_sdba/test_properties.py
@@ -515,19 +515,3 @@ def test_get_measure(self, open_dataset):
 
  meas = sdba.properties.var.get_measure()(sim_var, ref_var)
  np.testing.assert_allclose(meas, [0.408327], rtol=1e-3)
-
-
-class TestEOF:
- def test_first_eof(self, open_dataset):
- pytest.importorskip("eofs")
- sim = (
- open_dataset("NRCANdaily/nrcan_canada_daily_tasmax_1990.nc")
- .tasmax.isel(lon=slice(0, 10), lat=slice(50, 60))
- .load()
- )
-
- out = sdba.properties.first_eof(sim)
- np.testing.assert_allclose(
- [out.mean(), out.max()], [0.099976, 0.103867], rtol=1e-5
- )
- assert (out.isnull() == sim.isnull().any("time")).all()
diff --git a/tox.ini b/tox.ini
@@ -5,7 +5,6 @@ env_list =
  docs
  notebooks_doctests
  offline-prefetch
-; opt-slow
  py38
  py39-upstream-doctest
  py310
@@ -58,8 +57,10 @@ allowlist_externals =
 # Requires tox-conda compatible with tox@v4.0
 ;[testenv:conda]
 ;description = Run tests with pytest under {basepython} (Anaconda distribution)
+;commands_pre =
 ;conda_channels = conda-forge
 ;conda_env = environment-dev.yml
+;deps =
 ;extras =
 
 [testenv:notebooks_doctests{-coverage,}]
@@ -68,15 +69,6 @@ commands =
  pytest --no-cov --nbval --dist=loadscope --rootdir=tests/ --ignore=docs/notebooks/example.ipynb docs/notebooks
  pytest --rootdir=tests/ --xdoctest xclim
 
-# Requires tox-conda compatible with tox@v4.0
-;[testenv:opt-{slow,not_slow}]
-;description = Run tests with optional requirements (SBCK (experimental), eofs) and pytest under {basepython} (Anaconda distribution)
-;conda_env = environment-dev.yml
-;commands =
-; pip check
-; !slow: pytest xclim -m "not slow" --durations=10
-; slow: pytest xclim --durations=10
-
 [testenv:offline{-prefetch,}{-coverage,}]
 description = Run tests with pytest under {basepython}, preventing socket connections (except for unix sockets for async support)
 commands:
@@ -112,7 +104,6 @@ deps =
  numba: llvmlite==0.42.0rc1
  coverage: coveralls
  upstream: -rrequirements_upstream.txt
- eofs: eofs
  sbck: pybind11
 install_command = python -m pip install --no-user {opts} {packages}
 download = True

diff --git a/xclim/sdba/properties.py b/xclim/sdba/properties.py
@@ -28,7 +28,6 @@
 
 from .base import Grouper, map_groups
 from .nbutils import _pairwise_haversine_and_bins
-from .processing import jitter_under_thresh
 from .utils import _pairwise_spearman, copy_all_attrs
 
 
@@ -1202,73 +1201,17 @@ def _bin_corr(corr, distance):
 )
 
 
-def _first_eof(da: xr.DataArray, *, dims=None, kind="+", thresh="1 mm/d", group="time"):
- """First Empirical Orthogonal Function.
+def first_eof():
+ """EOF Statistical Property (function removed).
 
- Through principal component analysis (PCA), compute the predominant empirical orthogonal function.
- The temporal dimension is reduced. The Eof is multiplied by the sign of its mean to ensure coherent
- signs as much as possible. Needs the eofs package to run. Based on an idea from :cite:p:`vrac_multivariate_2018`,
- using an implementation from :cite:p:`dawson_eofs_2016`.
-
- Parameters
- ----------
- da: xr.DataArray
- Data.
- dims: sequence of string, optional
- Name of the spatial dimensions. If None (default), all dimensions except "time" are used.
- kind : {'+', '*'}
- Variable "kind". If multiplicative, the zero values are set to
- very small values and the PCA is performed over the logarithm of the data.
- thresh: str
- If kind is multiplicative, this is the "zero" threshold passed to
- :py:func:`xclim.sdba.processing.jitter_under_thresh`.
- group: str
- Useless for now.
-
- Returns
- -------
- xr.DataArray, [dimensionless]
- First empirical orthogonal function
+ Warnings
+ --------
+ Due to a licensing issue, eofs-based functionality has been permanently removed.
+ Please excuse the inconvenience.
+ For more information, see: https://github.com/Ouranosinc/xclim/issues/1620
  """
- try:
- from eofs.standard import Eof
- except ImportError as err:
- raise ValueError(
- "The `first_eof` property requires the `eofs` package"
- ", which is an optional dependency of xclim."
- ) from err
-
- if dims is None:
- dims = [d for d in da.dims if d != "time"]
-
- if kind == "*":
- da = np.log(jitter_under_thresh(da, thresh=thresh))
-
- da = da - da.mean("time")
-
- def _get_eof(d):
- # Remove slices where everything is nan
- d = d[~np.isnan(d).all(axis=tuple(range(1, d.ndim)))]
- solver = Eof(d, center=False)
- eof = solver.eofs(neofs=1).squeeze()
- return eof * np.sign(np.nanmean(eof))
-
- out = xr.apply_ufunc(
- _get_eof,
- da,
- input_core_dims=[["time", *dims]],
- output_core_dims=[dims],
- dask="parallelized",
- vectorize=True,
- output_dtypes=[float],
- dask_gufunc_kwargs={"allow_rechunk": True},
+ raise RuntimeError(
+ "Due to a licensing issue, eofs-based functionality has been permanently removed. "
+ "Please excuse the inconvenience. "
+ "For more information, see: https://github.com/Ouranosinc/xclim/issues/1620"
  )
- return out.assign_attrs(units="")
-
-
-first_eof = StatisticalProperty(
- identifier="first_eof",
- aspect="spatial",
- compute=_first_eof,
- allowed_groups=["group"],
-)