Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enh/threshold metadata #189

Merged
merged 83 commits into from
Sep 2, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
af5434b
ENH: Make rxxp and rxxptot threshold configurable
bzah Jun 17, 2022
f2dd6bb
ENH: Update metadata when a custom threshold is used
bzah Jun 17, 2022
052881d
[skip ci] big wip again
bzah Jun 30, 2022
85f14f6
it weurks
bzah Jul 4, 2022
79acf97
ENH: Make generic indices work on multiple inputs
bzah Jul 6, 2022
8ab33f1
Enh(s)
bzah Jul 8, 2022
510f85a
Cleanup
bzah Jul 8, 2022
0946e27
Improvements on `threshold`
bzah Jul 8, 2022
59f0213
Merge remote-tracking branch 'origin/master' into enh/threshold_metadata
bzah Jul 8, 2022
def38e4
MAINT: draft release notes
bzah Jul 8, 2022
6e03908
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 8, 2022
58ce894
ENH: clean output metadata
bzah Jul 20, 2022
bb335e0
ENH: Make it possible to compute CWD
bzah Jul 22, 2022
184c20a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 22, 2022
6a55a66
ENH: Make it possible to compute CWD
bzah Jul 22, 2022
fddd966
[skip ci] MAINT: Cleanup
bzah Jul 27, 2022
6103704
MAINT: Add registry for extreme mode
bzah Jul 28, 2022
47bf050
MAINT: Use Registry everywhere
bzah Jul 28, 2022
e844e17
MAINT: cleanup
bzah Aug 3, 2022
1faaab8
Merge remote-tracking branch 'origin/master' into enh/threshold_metadata
bzah Aug 3, 2022
2e6eca0
MAINT: make another cleanup
bzah Aug 3, 2022
daca7c5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 3, 2022
9fa9cea
FIX: Re-add some global attributes
bzah Aug 4, 2022
f5eb8be
ENH: Add basic implementation of generic reducer
bzah Aug 4, 2022
6517040
ENH: Add max_consecutive_occurrence reducer
bzah Aug 4, 2022
fc435cc
MAINT: clean imports
bzah Aug 4, 2022
9d0324e
ENH: Add sum_of_spell_lengths reducer
bzah Aug 5, 2022
1c58b79
ENH: Add deficit and excess reducers
bzah Aug 5, 2022
0be0516
ENH: Add fraction_of_total reducer
bzah Aug 5, 2022
cad0af5
MAINT: Clean
bzah Aug 5, 2022
8ebd3f1
FIX: Multi variable indices
bzah Aug 8, 2022
da5b754
FIX: link to custom indices doc
bzah Aug 10, 2022
3bf2b02
MAINT: Cleanup
bzah Aug 10, 2022
7e0eb43
MAINT: Add absolufy-imports for PEP8
bzah Aug 11, 2022
c049df0
ENH: Add simple reducer
bzah Aug 12, 2022
c2cbc0a
ENH: Improve metadata
bzah Aug 12, 2022
2133919
Merge remote-tracking branch 'origin/master' into enh/threshold_metadata
bzah Aug 12, 2022
665bb7e
ENH: Add rolling sum indicators
bzah Aug 12, 2022
8cbb79e
MAINT: Cleanup imports
bzah Aug 12, 2022
f19c4a0
ENH: Add rolling average indicators
bzah Aug 12, 2022
ba6a169
ENH: Improve metadata
bzah Aug 16, 2022
f40118c
ENH: Add "air" to temperature long_names
bzah Aug 16, 2022
14362ca
ENH: Add mean_of_difference generic indicator
bzah Aug 16, 2022
c60ea3f
ENH: Add difference_of_extremes generic indicator
bzah Aug 16, 2022
33dddf6
ENH: Add mean_of_absolute_one_time_step_difference generic indicator
bzah Aug 16, 2022
ce02107
ENH: Add difference_of_means generic indicator
bzah Aug 17, 2022
575fb7e
MAINT: Cleanup
bzah Aug 18, 2022
a545f52
ENH: Transform save_percentile into save_thresholds
bzah Aug 18, 2022
b0d06a3
Clean yml
bzah Aug 19, 2022
6abce9e
ENH: Proxy ECAD indices to generic indicators
bzah Aug 19, 2022
03e6863
MAINT: Format clix-meta files
bzah Aug 25, 2022
f0d5d7f
MAINT: Delete clix-meta files
bzah Aug 25, 2022
9eb9904
ENH: improve rle for spells
bzah Aug 25, 2022
acc71e7
ENH: Remove clipped_season from slice_mode
bzah Aug 25, 2022
33eb89f
FIX: indices expecting an amount
bzah Aug 25, 2022
a593329
ENH: Update generated API
bzah Aug 25, 2022
0e676fe
ENH: Update generated API
bzah Aug 25, 2022
e7284a7
ENH: Update generated API
bzah Aug 25, 2022
95ee527
MAINT: Remove xclim catalog
bzah Aug 26, 2022
6672086
MAINT: Cleanup
bzah Aug 26, 2022
dfa2a4e
ENH: Add reference to StandardIndex
bzah Aug 26, 2022
2c4d98d
ENH: step 1 - migrate user_index to generic indices
bzah Aug 26, 2022
c265d96
MAINT: Clean
bzah Aug 29, 2022
287d386
ENH: Re-add user index LogicalLink
bzah Aug 29, 2022
2e1b707
ENH: Re-add user index coef
bzah Aug 29, 2022
94cd5ac
ENH: Re-add date_event
bzah Aug 29, 2022
66d35e6
ENH: Re-add date_event (2)
bzah Aug 29, 2022
cb76220
MAINT: regenerate API
bzah Aug 29, 2022
db1a579
ENH: Add missing CF units to ecad indices
bzah Aug 30, 2022
72b24a3
ENH: Re-add '%' unit
bzah Aug 30, 2022
5986363
ENH: Add missing CMIP core variables
bzah Aug 30, 2022
e5207cb
MAINT: Remove useless error
bzah Aug 30, 2022
1a64b09
FIX: user_index mapping
bzah Aug 30, 2022
803b930
MAINT: Update some user_index tests
bzah Aug 30, 2022
50cb49a
FIX: max_of_rolling_average
bzah Aug 31, 2022
d239b68
ENH: refactor stuff
bzah Aug 31, 2022
37dbbe1
FIX: DifferenceOfMeans Indicator (a.k.a. Anomaly)
bzah Sep 1, 2022
de74418
[skip ci] DOC: Update coverage
Sep 1, 2022
6639036
FIX: ECAD indices
bzah Sep 2, 2022
4d0a4fa
FIX: difference_of_means
bzah Sep 2, 2022
380940a
MAINT: Cleanup and add integration tests
bzah Sep 2, 2022
f603d5d
[skip ci] DOC: Update coverage
Sep 2, 2022
a795078
ENH: Make it possible to use % unit on `fraction_of_total`
bzah Sep 2, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@ default_language_version:
python: python3

repos:
- repo: https://github.com/MarcoGorelli/absolufy-imports
rev: v0.3.1
hooks:
- id: absolufy-imports
name: absolufy-imports
- repo: https://github.com/asottile/pyupgrade
rev: v2.37.3
hooks:
Expand Down
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ For a detailed description of each ECA&D index, please visit: https://www.ecad.e
..
Pytest Coverage Comment:Begin

.. |coverage| image:: https://img.shields.io/badge/Coverage-93%25-brightgreen.svg
.. |coverage| image:: https://img.shields.io/badge/Coverage-92%25-brightgreen.svg
:target: https://github.com/cerfacs-globc/icclim/blob/master/README.rst#code-coverage
:alt: Code coverage

Expand Down
36 changes: 18 additions & 18 deletions doc/source/references/custom_indices.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,31 +33,31 @@ In icclim documentation we usually call them custom indices or user indices.
~~~~~~~~~~~~~~~~~~~~~~~~~
``user_index`` is a dictionary with possible keys:

+--------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|Key |Type of value |Description |
+==========================+===========================================+======================================================================================+
+------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|Key |Type of value |Description |
+========================+===========================================+======================================================================================+
|index_name |*str* |Name of custom index. |
+--------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
+------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|calc_operation |*str* |Type of calculation. See below for more details. |
+--------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|logical_operation |*str* |gt, lt, get, let or e |
+--------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
+------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|logical_operation |*str* |gt, lt, get, let or e |
+------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|thresh |*float* or *str* |In case of percentile-based index, must be string which starts with "p" (e.g. "p90"). |
+--------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|link_logical_operations |*str* |and or or |
+--------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|extreme_mode |*str* |min or max for computing min or max of running mean/sum. |
+--------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
+------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|link_logical_operations |*str* |and or or |
+------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|extreme_mode |*str* |min or max for computing min or max of running mean/sum. |
+------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|window_width |*int* |Used for computing running mean/sum. |
+--------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
+------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|coef |*float* |Constant for multiplying input data array. |
+--------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
+------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|date_event |*bool* |To keep or not the date of event. See below for more details. |
+--------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
+------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|var_type |*str* |"t" or "p". See below for more details. |
+--------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
+------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
|ref_time_range |[*datetime.datetime*, *datetime.datetime*] |Time range of reference (baseline) period for computing anomalies. |
+--------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+
+------------------------+-------------------------------------------+--------------------------------------------------------------------------------------+

Additional information about ``user_index`` keys are given below.

Expand All @@ -66,7 +66,7 @@ calc_operation key
++++++++++++++++++

======================================= ===========================================================================
value description
Value Description
======================================= ===========================================================================
``max`` maximum
``min`` minimum
Expand Down
2 changes: 2 additions & 0 deletions doc/source/references/icclim_index_api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -351,6 +351,8 @@ in *icclim* as ``hyndman_fan`` interpolation, also known as type 8.
Percentile-based indices (TX10p, TX90p, TN10p, TN90p, TG10p, TG90p, R75p, R95p and R99p) could be returned as number of days (default)
or as percentage of days (``out_unit`` = "%").

.. _custom_indices_old:

Custom indices
--------------

Expand Down
19 changes: 19 additions & 0 deletions doc/source/references/release_notes.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,25 @@
Release history
===============

6.0 (unreleased)
----------------
* [enh] Add generic indices
* [enh] Make in_files.var.threshold and threshold parameters work with string values (a value with a unit or a percentile stamp)
* [maint] **BREAKING CHANGE:** ECAD indices are no longer configurable! Use a generic index instead.
* [fix] **BREAKING CHANGE:** ECAD indices CW, CD, WW, WD were computing the precipitation percentiles on day of year values where it should have been percentiles of the whole period (excluding dry days). This has been fixed.
* [maint] icclim no longer carries a version of the clix-meta yml file. Previously it was used to generate the doc string and a few metadata of ECAD indices. It's no longer needed as we have put these metadata within StandardIndex declaration.
* [maint] **BREAKING CHANGE:** Removed the `clipped_season` option from `slice_mode`. With generic indices, `season` should work with every indices.
In particular, spell based indices (e.g. wsdi) are mapped to `max_consecutive_occurrence` generic indicator, which computes the spell length before doing the resampling operation.
So a spell that start and end outside the output frequency interval is properly accounted for its whole duration.
That's for example the case of `slice_mode="month"`, but with a spell that start in january and end in March, the whole spell length would be available in january results.
* [maint] **BREAKING CHANGE:** User index `max_nb_consecutive_events` is also mapped to `max_consecutive_occurrence`, consequently spells are also counted for their whole duration.
* [enh] Make it possible to pass a simple dictionary in `in_files`, merging together basic `in_files` and `var_name` features.
It looks like `in_files={"tasmax": "tasmax.nc", "tasmin": "tasmin.zarr"}`
* [enh] Add `min_spell_length` parameter to index API in order to control the minimum duration of spells in `sum_of_spell_lengths`.
* [enh] Add `rolling_window_width` parameter to index API in order to control the width of the rolling window in `max|min_of_rolling_sum|average`.
* [enh] Add `doy_window_width` parameter to index API in order to control the width of aggregation windows when computing doy percentiles.
* [maint] Deprecate `window_width` parameter. When filled, it is mapped to It is still mapped `doy_window_width`.

5.4.0
-----
* [fix] When giving input as a list of netcdf files, the coordinate values are now merged using the `override` strategy, thus the first file with a given dimension define this dimension for all the files.
Expand Down
7 changes: 3 additions & 4 deletions icclim/__init__.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# keep imports below to expose api in `icclim` namespace
from icclim._generated_api import * # noqa
from icclim.main import index, indice, indices # noqa
from icclim.models.constants import ICCLIM_VERSION

from ._generated_api import * # noqa
from .main import index, indice, indices # noqa
from .pre_processing.rechunk import create_optimized_zarr_store # noqa
from icclim.pre_processing.rechunk import create_optimized_zarr_store # noqa

__version__ = ICCLIM_VERSION
Loading