Skip to content

Commit

Permalink
[PyOpenSci] Documentation and Notebook Adjustments, manage SBCK (#1338)
Browse files Browse the repository at this point in the history
### What kind of change does this PR introduce?

* Grammar and spelling mistakes within the notebooks have been
~corrected~ reduced.
* `SBCK` has been pulled from `environment.yml` for the time being until
a proper package is created.
* The Contributing guide now lists three separate subsections to help
users understand the gains from optional dependencies

### Does this PR introduce a breaking change?

Yes, but no[^1]. Changes to dependencies have been handled in such a way
that default settings are unaffected. Development and CI-based behaviour
is affected: `clisops` and `flox` are now in the `environment.yml`.

### Other information:
yrobink/SBCK-python#5
[^1]:
https://i.kym-cdn.com/entries/icons/original/000/028/596/dsmGaKWMeHXe9QuJtq_ys30PNfTGnMsRuHuo_MUzGCg.jpg
  • Loading branch information
Zeitsperre authored Mar 31, 2023
2 parents 78ff493 + f566018 commit 3935907
Show file tree
Hide file tree
Showing 22 changed files with 385 additions and 356 deletions.
3 changes: 2 additions & 1 deletion .github/workflows/upstream.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,10 @@ jobs:
run: |
conda --version
echo "micromamba: $(micromamba --version)"
- name: Install upstream versions
- name: Install upstream versions and SBCK
run: |
python -m pip install -r requirements_upstream.txt
python -m pip install "sbck @ git+https://github.com/yrobink/SBCK-python.git@master"
- name: Install xclim
run: |
python -m pip install --no-user --no-deps .
Expand Down
2 changes: 2 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ Internal changes
* `tox` now include `sbck` and `eofs` flags for easier testing of dependencies. CI builds now test against `sbck-python` @ master. (:pull:`1328`).
* `upstream` CI tests are now run on push to master, at midnight, and can also be triggered via `workflow_dispatch`. Failures from upstream build will open issues using `xarray-contrib/issue-from-pytest-log`. (:pull:`1327`).
* Warnings from set ``_version_deprecated`` within Indicators now emit ``FutureWarning`` instead of ``DeprecationWarning`` for greater visibility. (:pull:`1319`).
* The `Graphics` section of the `Usage` notebook has been expanded upon while grammar and spelling mistakes within the notebook-generated documentation have been reduced. (:issue:`1335`, :pull:`1338`, suggested from `PyOpenSci Software Review <https://github.com/pyOpenSci/software-review/issues/73>`_).
* The Contributing guide now lists three separate subsections to help users understand the gains from optional dependencies. (:issue:`1335`, :pull:`1338`, suggested from `PyOpenSci Software Review <https://github.com/pyOpenSci/software-review/issues/73>`_).

v0.41.0 (2023-02-28)
--------------------
Expand Down
5 changes: 3 additions & 2 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -327,10 +327,11 @@ When publishing on GitHub, maintainers will need to generate the release notes f
# For ReStructuredText format (offered for convenience):
$ xclim release_notes -r

When publishing to GitHub, you will still need to replace subsection headers in the Markdown (`^^^^` -> `###`) and the history published should not extend past the changes for the current version. This behaviour may eventually change.
.. note::
The changelog should not extend past those entries relevant for the current version.

.. warning::
Be warned that a published package version on PyPI can never be overwritten. Be sure to verify that the package published at https://test.pypi.org/project/xclim/ matches expectations before publishing a version on GitHub.
A published version on PyPI can never be overwritten. Be sure to verify that the package published at https://test.pypi.org/project/xclim/ matches expectations before publishing a version on GitHub.

The Manual Approach
~~~~~~~~~~~~~~~~~~~
Expand Down
24 changes: 21 additions & 3 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ xclim: Climate services library |logo|
======================================

+----------------------------+-----------------------------------------------------+
| Versions | |pypi| |conda| |
| Versions | |pypi| |conda| |versions| |
+----------------------------+-----------------------------------------------------+
| Documentation and Support | |docs| |gitter| |versions| |
| Documentation and Support | |docs| |gitter| |
+----------------------------+-----------------------------------------------------+
| Open Source | |license| |fair| |fossa| |zenodo| |
+----------------------------+-----------------------------------------------------+
Expand Down Expand Up @@ -41,10 +41,29 @@ streamflow and sea ice concentration, numerous bias-adjustment algorithms, as we
.. _xarray: https://docs.xarray.dev/
.. _dask: https://docs.dask.org/

Quick Install
-------------
`xclim` can be installed from PyPI:

.. code-block:: shell
$ pip install xclim
or from Anaconda (conda-forge):

.. code-block:: shell
$ conda install -c conda-forge xclim
Documentation
-------------
The official documentation is at https://xclim.readthedocs.io/

How to make the most of xclim: `Basic Usage Examples`_ and `In-Depth Examples`_.

.. _Basic Usage Examples: https://xclim.readthedocs.io/en/stable/notebooks/usage.html
.. _In-Depth Examples: https://xclim.readthedocs.io/en/stable/notebooks/index.html

Contributing to xclim
---------------------
xclim is in active development and is being used in production by climate services specialists around the world.
Expand All @@ -66,7 +85,6 @@ License
-------
This is free software: you can redistribute it and/or modify it under the terms of the `Apache License 2.0`_. A copy of this license is provided in the code repository (`LICENSE`_).


.. _Apache License 2.0: https://opensource.org/license/apache-2-0/
.. _LICENSE: https://github.com/Ouranosinc/xclim/blob/master/LICENSE

Expand Down
10 changes: 8 additions & 2 deletions docs/explanation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ Purpose

The primary domains that `xclim` is built for are in calculating climate indicators, performing statistical correction / bias adjustment of climate model output variables or simulations, and in performing climate model simulation ensemble statistics.

Other projects similar to xclim
===============================
Other Python projects similar to xclim
======================================

`xclim` has been developed within an ecosystem of several existing projects that deal with climate and statistical correction/downscaling and has both influenced and been influenced by their approaches:

Expand All @@ -31,11 +31,17 @@ Other projects similar to xclim
- `MetPy` is built for reading, visualizing, and performing calculations specifically on standards-compliant, operational weather data. Like `xclim`, it makes use of `xarray`.
- `xclim` adopted its standards and unit-handling approaches from `MetPy` and associated project `cf-xarray`.

* `climpred` (`climpred Source Code <https://github.com/pangeo-data/climpred>`_; `climpred Documentation <https://climpred.readthedocs.io/en/stable/index.html>`_)
- `climpred` is designed to analyze and validate weather and climate forecast data against observations, reconstructions, and simulations. Similar to `xclim`, it leverages `xarray`, `dask`, and `cf_xarray` for object handling, distributed computation, and metadata validation, respectively.

* `pyet` (`pyet Source Code <https://github.com/pyet-org/pyet>`_; `pyet Documentation <https://pyet.readthedocs.io/en/latest/>`_)
- `pyet` is a tool for calculating/estimating evapotranspiration using many different accepted methodologies and employs a similar design approach as `xclim`, based on `xarray`-natives.

* `xcdat` (`xcdat Source Code <https://github.com/xCDAT/xcdat>`_; `xcdat Documentation <https://xcdat.readthedocs.io/en/latest/>`_)

* `GeoCAT` (`GeoCAT Documentation <https://geocat.ucar.edu/>`_)
- `GeoCAT` is an ensemble of tools developed specifically for scalable data analysis and visualization of structures and unstructured gridded earth science datasets. `GeoCAT` tools rely on many of the same tools that `xclim` uses in its stack (notably, `xarray`, `dask`, and `jupyter notebooks`).

* `scikit-downscale` (`scikit-downscale Source Code <https://github.com/pangeo-data/scikit-downscale>`_, `scikit-downscale Documentation <https://scikit-downscale.readthedocs.io/en/latest/>`_)
- `scikit-downscale` offers algorithms for statistical downscaling. `xclim` drew inspiration from its fit/predict architecture API approach. The suite of downscaling algorithms offered between both projects differs.

Expand Down
68 changes: 50 additions & 18 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,23 @@ Installation

Stable release
--------------
To install xclim via pip, run this command in your terminal:

To install `xclim` via `pip`, run this command in your terminal:

.. code-block:: shell
$ pip install xclim
This is the preferred method to install xclim, as it will always install the most recent stable release.

If you don't have `pip`_ installed, this `Python installation guide`_ can guide you through the process.

.. _pip: https://pip.pypa.io/
.. _Python installation guide: https://docs.python-guide.org/starting/installation/

Anaconda release
----------------

For ease of installation across operating systems, we also offer an Anaconda Python package hosted on conda-forge.
This version tends to be updated at around the same frequency as the pip library, but can lag by a few days at times.
This version tends to be updated at around the same frequency as the PyPI-hosted library, but can lag by a few days at times.

`xclim` can be installed from conda-forge wth the following:

Expand All @@ -32,15 +32,41 @@ This version tends to be updated at around the same frequency as the pip library
.. _extra-dependencies:

Extra dependencies
Extra Dependencies
------------------

Speedups and Helper Libraries
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

To improve performance of `xclim`, we highly recommend you also install `flox`_ (see: :doc:`flox API <flox:api>`).
This package integrates into xarray and significantly improves the performance of the grouping and resampling algorithms, especially when using `dask` on large datasets.
This package seamlessly integrates into `xarray` and significantly improves the performance of the grouping and resampling algorithms, especially when using `dask` on large datasets.

For grid subsetting, we also recommend using the tools found in `clisops`_ (see: :doc:`clisops.core.subset API <clisops:api>`) for spatial manipulation of geospatial data. `clisops` began as a component of `xclim` and is designed to work alongside `xclim` and the `Pangeo`_ stack (`xarray`, `dask`, `jupyter`). In order to install `clisops`, the `GDAL`_ system libraries must be available.

On Debian/Ubuntu, `GDAL` can be installed via `apt`:

.. code-block:: shell
$ sudo apt-get install libgdal-dev
We also recommend using the subsetting tools found in `clisops`_ (see: :doc:`clisops.core.subset API <clisops:api>`) for spatial manipulation of geospatial data.
If on Anaconda Python, `GDAL` will be installed if needed as a `clisops` dependency.

Both of these libraries are available on PyPI and conda-forge:

.. code-block:: shell
$ pip install flox clisops
# Or, alternatively:
$ conda install -c conda-forge flox clisops
.. _GDAL: https://gdal.org/download.html#binaries
.. _Pangeo: https://pangeo.io/

Upstream Dependencies
^^^^^^^^^^^^^^^^^^^^^

`xclim` is regularly tested against the main development branches of a handful of key base libraries (`cftime`, `flox`, `pint`, `xarray`).
For convenience, these libraries can be installed alongside `xclim` using the following `pip`-installable recipe:
For convenience, these libraries can be installed alongside `xclim` using the following `pip`-install command:

.. code-block:: shell
Expand All @@ -55,29 +81,33 @@ Or, alternatively:
.. _flox: https://github.com/xarray-contrib/flox
.. _clisops: https://github.com/roocs/clisops

Another optional library is `SBCK`_, which provides experimental adjustment methods to extend :doc:`xclim.sdba <sdba>`.
`SBCK` is not installable directly from PyPI or conda-forge and has one complex dependency: `Eigen3`_.
Experimental SDBA Algorithms
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

`xclim` also offers support for a handful of experimental adjustment methods to extend :doc:`xclim.sdba <sdba>`, available only if some additional libraries are installed. These libraries are completely optional.

One experimental library is `SBCK`_. `SBCK` is not available from PyPI nor conda-forge, and has one complex dependency: `Eigen3`_.
As `SBCK` is compiled at installation time, a **C++** compiler (`GCC`, `Clang`, `MSVC`, etc.) must also be available.

On Debian/Ubuntu, Eigen3 can be installed from via `apt`:
On Debian/Ubuntu, `Eigen3` can be installed via `apt`:

.. code-block:: shell
$ sudo apt-get install libeigen3-dev
Eigen3 is also available on conda-forge, so one can do:
Eigen3 is also available on conda-forge, so, if already using Anaconda, one can do:

.. code-block:: shell
$ conda install -c conda-forge eigen
Afterwards, `SBCK can be installed from PyPI using `pip`:
Afterwards, `SBCK` can be installed from PyPI using `pip`:

.. code-block:: shell
$ pip install pybind11
$ pip install "sbck @ git+https://github.com/yrobink/SBCK-python.git@master"
Finally, the function :py:indicator:`xclim.sdba.property.first_eof` makes use of `eofs`_, another optional dependency, which is available on both pip and conda:
Another experimental function :py:indicator:`xclim.sdba.property.first_eof` makes use of the `eofs`_ library, which is available on both PyPI and conda-forge:

.. code-block:: shell
Expand All @@ -91,7 +121,8 @@ Finally, the function :py:indicator:`xclim.sdba.property.first_eof` makes use of

From sources
------------
.. Warning::

.. warning::
For Python3.11+ users: Many of the required scientific libraries do not currently have wheels that support the latest
python. In order to ensure that installation of xclim doesn't fail, we suggest installing the `Cython` module
before installing xclim in order to compile necessary libraries from source packages.
Expand Down Expand Up @@ -128,10 +159,11 @@ Alternatively, you can also install a local development copy via `flit`_:

Creating a Conda environment
----------------------------
To create a conda environment including all of `xclim`'s optional and development dependencies, run the following command from within your cloned repo:

To create a conda environment including `xclim`'s dependencies and several optional libraries (notably: `clisops`, `eigen`, `eofs`, and `flox`) and development dependencies, run the following command from within your cloned repo:

.. code-block:: console
$ conda create -n my_xclim_env python=3.8 --file=environment.yml
$ conda env create -n my_xclim_env python=3.8 --file=environment.yml
$ conda activate my_xclim_env
(my_xclim_env) $ pip install -e .
Loading

0 comments on commit 3935907

Please sign in to comment.