Skip to content

Commit

Permalink
Merge pull request #87 from ocefpaf/docs
Browse files Browse the repository at this point in the history
Docs
  • Loading branch information
kwilcox authored Oct 13, 2022
2 parents 7832dde + f17047e commit da4696b
Show file tree
Hide file tree
Showing 7 changed files with 21 additions and 20 deletions.
3 changes: 1 addition & 2 deletions .github/workflows/deploy-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
- name: Build environment
shell: bash -l {0}
run: |
micromamba create --name TEST python=3 --file requirements.txt --file tests/requirements.txt --channel conda-forge
micromamba create --name TEST python=3 --file requirements.txt --file docs/requirements.txt --channel conda-forge
micromamba activate TEST
python -m pip install -e . --no-deps --force-reinstall
Expand All @@ -41,7 +41,6 @@ jobs:
set -e
micromamba activate TEST
pushd docs
micromamba install --file requirements.txt --channel conda-forge
make clean html linkcheck
popd
Expand Down
3 changes: 3 additions & 0 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
bokeh
ipykernel
matplotlib
nbsphinx
pynco
sphinx
sphinx-autodoc-typehints
1 change: 1 addition & 0 deletions docs/source/api/ioos_qc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ ioos\_qc.config module
:members:
:undoc-members:
:show-inheritance:
:noindex:

ioos\_qc.gliders module
-----------------------
Expand Down
3 changes: 1 addition & 2 deletions docs/source/other.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@ Here is a collection of other QARTOD implementations or contacts for reference.
Python

- `Ocean Observatories Initiative (OOI) Quality Control Algorithm <https://github.com/ooici/ion-functions/tree/master/ion_functions/qc>`_
- `OOI Quality Control Algorithm Tables <https://github.com/ooi-integration/qc-lookup>`_
- `QARTOD in Practice <https://drive.google.com/open?id=0BwGScp7mjYjyNGVRUG5ZTTNPdUU>`_
- `OOI Quality Control Algorithm Tables <https://github.com/ooi-integration/qc-lookup>`_

R

Expand Down
14 changes: 7 additions & 7 deletions docs/source/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ At its core, ``ioos_qc`` is a collection of modules and methods to run various q

The following implementations are available in ``ioos_qc``:

* `IOOS QARTOD <https://ioos.noaa.gov/project/qartod/>`_ - `API </api/ioos_qc.html#module-ioos_qc.qartod>`_
* `ARGO - API </api/ioos_qc.html#module-ioos_qc.argo>`_
* `AXDS - API </api/ioos_qc.html#module-ioos_qc.axds>`_ - A collection of checks used by `Axiom Data Science <https://axiomdatascience.com>`_
* `IOOS QARTOD <https://ioos.noaa.gov/project/qartod/>`_ - :mod:`API <ioos_qc.qartod>`
* :mod:`ARGO - API <ioos_qc.argo>`
* :mod:`AXDS - API <ioos_qc.axds>` - A collection of checks used by `Axiom Data Science <https://axiomdatascience.com>`_

Basic usage
-----------
Expand Down Expand Up @@ -211,7 +211,7 @@ Usage
Streams
-------

Streams represent the data input types for running quality control tests. A user "runs" a stream of data through a collection of quality control tests defined by a Config_. A list of possible Streams can be found in the `Streams API </api/ioos_qc.html#module-ioos_qc.streams>`_.
Streams represent the data input types for running quality control tests. A user "runs" a stream of data through a collection of quality control tests defined by a Config_. A list of possible Streams can be found in the :mod:`Streams API <ioos_qc.streams>`.
All streams return a generator of QC results that contain contextual information that can be useful when using the results. You can iterate over the results generator directly or you can collect them into more familiar ``list`` or ``dict`` objects before usage. If you are
working in a streaming environment you will want to use generator result objects yourself. If you are running one-time or batch process quality checks you likely want to collect the results or use one of the Stores_ provided by ``ioos_qc``.

Expand All @@ -225,15 +225,15 @@ working in a streaming environment you will want to use generator result objects
Results
~~~~~~~

Each yielded result will be a `StreamConfigResult </api/ioos_qc.html#ioos_qc.results.StreamConfigResult>`_ or a `ContextResult </api/ioos_qc.html#ioos_qc.results.ContextResult>`_, depending on which type of Config_ object was used. Collected results are only ever of one type, a `CollectedResult </api/ioos_qc.html#ioos_qc.results.CollectedResult>`_, and only one ``CollectedResult`` will be returned after collecting Results. The benefit of using a ``CollectedResult`` is that it will piece back together all of the different ContextConfig_ objects in a Config_ and return you one result per unique ``stream_id`` and module/test combination.
Each yielded result will be a :mod:`StreamConfigResult <ioos_qc.results.StreamConfigResult>` or a :mod:`ContextResult <ioos_qc.results.ContextResult>`, depending on which type of Config_ object was used. Collected results are only ever of one type, a :mod:`CollectedResult <ioos_qc.results.CollectedResult>`, and only one ``CollectedResult`` will be returned after collecting Results. The benefit of using a ``CollectedResult`` is that it will piece back together all of the different ContextConfig_ objects in a Config_ and return you one result per unique ``stream_id`` and module/test combination.

.. note::

For example: If you had a Config_ object that contained (3) different ContextConfig_ objects (each defining a time window and test inputs) for a single variable/``stream_id``, running that ``Config`` through any ``Stream`` implementation would yield (3) different ``ContextResult`` objects. You could use them yourself to construct whatever results you wanted to manually, or you could collect those results back into a single ``CollectedResult`` object to only have to deal with one result.

.. warning::

Historically, test results were returned in a ``dict`` structure. While this is still supported it **should be considered deprecated**. The individually yielded result objects or a list of `CollectedResult objects <api/ioos_qc.html#ioos_qc.results.CollectedResult>`_ should be used in any applications, including any implementation of Stores_, going forward.
Historically, test results were returned in a ``dict`` structure. While this is still supported it **should be considered deprecated**. The individually yielded result objects or a list of :mod:`CollectedResult objects <ioos_qc.results.CollectedResult>` should be used in any applications, including any implementation of Stores_, going forward.


.. code-block:: python
Expand Down Expand Up @@ -517,7 +517,7 @@ A subset of the NumpyStream, the NetcdfStream simply extracts numpy arrays from
Stores
------

Stores represent different data formats for storing quality control Results_ from Streams_. The results from any ``Stream`` should be able to be passed into any ``Store`` implementation defined in the `Stores API </api/ioos_qc.html#module-ioos_qc.stores>`_.
Stores represent different data formats for storing quality control Results_ from Streams_. The results from any ``Stream`` should be able to be passed into any ``Store`` implementation defined in the :mod:`Stores API <ioos_qc.stores>`.

``ioos_qc`` comes with some built-in Stores_:

Expand Down
14 changes: 8 additions & 6 deletions ioos_qc/qartod.py
Original file line number Diff line number Diff line change
Expand Up @@ -469,12 +469,14 @@ def spike_test(inp: Sequence[N],
suspect_threshold: The SUSPECT threshold value, in observations units.
fail_threshold: The SUSPECT threshold value, in observations units.
method: ['average'(default),'differential'] optional input to assign the method used to detect spikes.
"average": Determine if there is a spike at data point n-1 by subtracting
the midpoint of n and n-2 and taking the absolute value of this
quantity, and checking if it exceeds a low or high threshold.
"differential": Determine if there is a spike at data point n by calculating the difference
between n and n-1 and n+1 and n variation. To considered, (n - n-1)*(n+1 - n) should
be smaller than zero (in opposite direction).
"average": Determine if there is a spike at data point n-1 by subtracting
the midpoint of n and n-2 and taking the absolute value of this
quantity, and checking if it exceeds a low or high threshold.
"differential": Determine if there is a spike at data point n by calculating the difference
between n and n-1 and n+1 and n variation. To considered, (n - n-1)*(n+1 - n) should
be smaller than zero (in opposite direction).
Returns:
A masked array of flag values equal in size to that of the input.
Expand Down
3 changes: 0 additions & 3 deletions tests/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,5 @@ dask
erddapy
flake8
isort
#nco
#pocean-core
#pynco
pytest
pytest-flake8>=1.0.4 # https://github.com/tholo/pytest-flake8/pull/59

0 comments on commit da4696b

Please sign in to comment.