Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hdc-algo utils & linspace #49

Merged
merged 6 commits into from
Feb 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/conda.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
conda:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Install Conda environment with Micromamba
uses: mamba-org/setup-micromamba@v1
Expand Down Expand Up @@ -77,7 +77,7 @@ jobs:
ls -lh pkgs/

- name: Upload results (artifact)
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: pkgs
path: pkgs
Expand Down
30 changes: 15 additions & 15 deletions .github/workflows/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
cache-path: ${{ steps.cfg.outputs.cache-path }}

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
Expand All @@ -24,7 +24,7 @@ jobs:
echo "cache-key=${cache_key}" >> $GITHUB_OUTPUT
echo "cache-path=${cache_path}" >> $GITHUB_OUTPUT
- name: cache deps
uses: actions/cache@v3
uses: actions/cache@v4
id: py_cache
with:
path: ${{ steps.cfg.outputs.cache-path }}
Expand All @@ -51,9 +51,9 @@ jobs:
- pyenv
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Get Python Env from Cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
key: ${{ needs.pyenv.outputs.cache-key }}
path: ${{ needs.pyenv.outputs.cache-path }}
Expand All @@ -74,9 +74,9 @@ jobs:
- pyenv
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Get Python Env from Cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
key: ${{ needs.pyenv.outputs.cache-key }}
path: ${{ needs.pyenv.outputs.cache-path }}
Expand All @@ -100,9 +100,9 @@ jobs:
- black
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Get Python Env from Cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
key: ${{ needs.pyenv.outputs.cache-key }}
path: ${{ needs.pyenv.outputs.cache-path }}
Expand All @@ -114,7 +114,7 @@ jobs:
env:
py_path: ${{ needs.pyenv.outputs.cache-path }}

- uses: actions/cache@v3
- uses: actions/cache@v4
id: wheels_cache
with:
path: ./wheels
Expand Down Expand Up @@ -150,7 +150,7 @@ jobs:
python setup.py sdist -d wheels

- name: Upload results (artifact)
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: wheels
path: wheels
Expand All @@ -162,9 +162,9 @@ jobs:
- black
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Get Python Env from Cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
key: ${{ needs.pyenv.outputs.cache-key }}
path: ${{ needs.pyenv.outputs.cache-path }}
Expand All @@ -190,9 +190,9 @@ jobs:
- wheels
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Get Python Env from Cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
key: ${{ needs.pyenv.outputs.cache-key }}
path: ${{ needs.pyenv.outputs.cache-path }}
Expand All @@ -205,7 +205,7 @@ jobs:
py_path: ${{ needs.pyenv.outputs.cache-path }}

- name: Get Wheels
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
with:
name: wheels
path: wheels
Expand Down
7 changes: 4 additions & 3 deletions .github/workflows/publish-s3.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,19 @@ jobs:
id-token: write
contents: read
steps:
- uses: actions/checkout@v3
- uses: actions/cache@v3
- uses: actions/checkout@v4
- uses: actions/cache@v4
id: wheels_cache
with:
path: ./wheels
key: wheels-${{ github.sha }}

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_S3_UPLOAD_ROLE }}
aws-region: eu-central-1
mask-aws-account-id: true

- name: upload wheels to S3
run: |
Expand Down
1 change: 1 addition & 0 deletions hdc/algo/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Seasonal Monitoring Algorithms."""

# isort: skip_file
from ._version import __version__

Expand Down
3 changes: 2 additions & 1 deletion hdc/algo/_version.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
"""Version only in this file."""
__version__ = "0.3.1"

__version__ = "0.4.0"
5 changes: 5 additions & 0 deletions hdc/algo/accessors.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Xarray Accesor classes."""

from typing import Iterable, List, Optional, Union
from warnings import warn

Expand Down Expand Up @@ -120,6 +121,10 @@ def end_date(self):
def raw(self):
return self._tseries.apply(lambda x: self._period_cls(x).raw).to_xarray()

@property
def linspace(self):
return self.yidx - 1


@xarray.register_dataset_accessor("dekad")
@xarray.register_dataarray_accessor("dekad")
Expand Down
1 change: 1 addition & 0 deletions hdc/algo/dekad.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Dekad helper class."""

from datetime import date, datetime, timedelta
from typing import Tuple, Union, overload

Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Numba implementations."""

from .autocorr import autocorr, autocorr_1d, autocorr_tyx
from .lroo import lroo
from .tinterpolate import tinterpolate
Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/autocorr.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Lag-1 autocorrelations."""

from numba import njit
from numba.core.types import float32, float64, int64
from numpy import isnan, zeros
Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/lroo.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Calculate the longest run of ones inside a 1d array."""

import numpy as np
from numba import guvectorize

Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/stats.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Numba accelerated statistical funtions."""

from math import erf, log, sqrt


Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/tinterpolate.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Interpolation numba functions."""

from numba import guvectorize
from numba.core.types import float64, int16, int32, uint8

Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/whit.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Numba accelerated Whittaker functions."""

# pyright: reportGeneralTypeIssues=false
# pylint: disable=C0103,C0301,E0401,R0912,R0913,R0914,R0915,too-many-lines
from math import log, pow, sqrt # pylint: disable=W0622
Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/ws2d.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Whittaker filter with differences of 2nd order for a 1d array."""

from numba import njit
from numpy import zeros

Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/ws2dgu.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Whittaker smoother with fixed lambda (S)."""

import numpy as np
from numba import guvectorize
from numba.core.types import float64, int16
Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/ws2doptv.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Whittaker filter V-curve optimization os S."""

from math import log, sqrt

import numpy as np
Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/ws2doptvp.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Whittaker filter V-curve optimization of S and asymmetric weights."""

# pyright: reportGeneralTypeIssues=false
from math import log, sqrt

Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/ws2doptvplc.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

numba implementations.
"""

# pyright: reportGeneralTypeIssues=false
from math import log, sqrt

Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/ws2dpgu.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Whittaker smoother with asymmetric smoothing and fixed lambda (S)."""

import numpy as np
from numba import guvectorize
from numba.core.types import float64, int16
Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/ws2dwcvp.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Whittaker filter V-curve optimization of S and asymmetric weights."""

# pyright: reportGeneralTypeIssues=false
import numpy as np
from numba import guvectorize, jit
Expand Down
1 change: 1 addition & 0 deletions hdc/algo/ops/zonal.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Numba accelerated zonal statistics."""

from numba import njit
import numpy as np

Expand Down
23 changes: 23 additions & 0 deletions hdc/algo/utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
"""hcd-algo utility functions."""

from typing import List, Tuple

import numpy as np


def to_linspace(x) -> Tuple[np.ndarray, List[int]]:
"""Map input array to linear space.

Returns array with linear index (0 - n-1) and list of
original keys matching the indices.
"""
keys = np.unique(x)
keys.sort()
values = np.arange(keys.size)

idx = np.searchsorted(keys, x.ravel()).reshape(x.shape)
idx[idx == keys.size] = 0
mask = keys[idx] == x.data
new_pix = np.where(mask, values[idx], 0)

return new_pix, list(keys)
2 changes: 1 addition & 1 deletion requirements_dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ twine
pytest
pytest-runner
pytest-cov
black
black>=24.1.0
pylint==2.17.2
astroid==2.15.3
pydocstyle
Expand Down
6 changes: 6 additions & 0 deletions tests/test_accessors.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Tests for xarray accessors"""

# pylint: disable=redefined-outer-name,unused-import,no-member,no-name-in-module,missing-function-docstring
# pyright: reportGeneralTypeIssues=false

Expand Down Expand Up @@ -73,6 +74,11 @@ def test_period_yidx_dekad(darr):
np.testing.assert_array_equal(darr.time.dekad.yidx, [1, 2, 3, 3, 4])


def test_period_yidx_linspace_dekad(darr):
assert isinstance(darr.time.dekad.linspace, xr.DataArray)
np.testing.assert_array_equal(darr.time.dekad.linspace, [0, 1, 2, 2, 3])


def test_period_ndays_dekad(darr):
assert isinstance(darr.time.dekad.ndays, xr.DataArray)
np.testing.assert_equal(
Expand Down
1 change: 1 addition & 0 deletions tests/test_algos.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Tests for pixel alogrithms"""

# pylint: disable=no-name-in-module,redefined-outer-name,no-value-for-parameter
# pyright: reportGeneralTypeIssues=false
import numpy as np
Expand Down
29 changes: 29 additions & 0 deletions tests/test_utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
"""Test utility functions."""

import numpy as np
import pytest

from hdc.algo.utils import to_linspace


@pytest.mark.parametrize(
"input_array, expected_output",
[
# Test case 1: Array with unique values
(
np.array([[1, 2, 3], [4, 5, 6]]),
(np.array([[0, 1, 2], [3, 4, 5]]), [1, 2, 3, 4, 5, 6]),
),
# Test case 2: Array with repeated values
(
np.array([[1, 2, 2], [3, 4, 4]]),
(np.array([[0, 1, 1], [2, 3, 3]]), [1, 2, 3, 4]),
),
# Test case 3: Empty array
(np.array([]), (np.array([]), [])),
],
)
def test_to_linspace(input_array, expected_output):
x, y = to_linspace(input_array)
np.testing.assert_equal(x, expected_output[0])
np.testing.assert_equal(y, expected_output[1])
Loading