Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Convert to numpy array before downsampling #29

Merged
merged 1 commit into from
Mar 19, 2023

Conversation

jayceslesar
Copy link
Contributor

@jayceslesar jayceslesar commented Mar 19, 2023

Was attempting to use this with non-numpy arrays in a project https://github.com/jayceslesar/hyperplot and was having a small inconvenience where needed to convert to numpy arrays as that is an underlying assumption of this library - this should make that assumption more robust. Would also be interesting to integrate this into polars using the UDf/Pipe methodology but might want a separation of concerns there as this library does what it needs to really well and polars does what it needs to really well and ideally all the magic happens on the rust side of things anyways

Essentially this would fail:

from datetime import datetime

import numpy as np
import polars as pl

from tsdownsample import LTTBDownsampler

data = {
    "timestamp": [
        datetime(year=2023, month=1, day=1),
        datetime(year=2023, month=1, day=2),
        datetime(year=2023, month=1, day=3),
    ],
    "value": [1, 2, 3],
}
df = pl.DataFrame(data)

indices = LTTBDownsampler().downsample(df["timestamp"], df["value"], n_out=3)

with

python3 testing.py

~/projects/tsdownsample on 🌱 feature/polars-support [$?] is 📦 v0.1.0 via 🐍 v3.10.6 (venv) via 🦀 v1.69.0-nightly on ☁️  jslesarbeta.team(beta-data-warehouse-test) 
❯ python3 testing.py
Traceback (most recent call last):
  File "/Users/jslesar/projects/tsdownsample/testing.py", line 18, in <module>
    indices = LTTBDownsampler().downsample(df["timestamp"], df["value"], n_out=3)
  File "/Users/jslesar/projects/tsdownsample/tsdownsample/downsampling_interface.py", line 322, in downsample
    return super().downsample(*args, n_out=n_out, parallel=parallel, **kwargs)
  File "/Users/jslesar/projects/tsdownsample/tsdownsample/downsampling_interface.py", line 107, in downsample
    x, y = self._check_valid_downsample_args(*args)
  File "/Users/jslesar/projects/tsdownsample/tsdownsample/downsampling_interface.py", line 54, in _check_valid_downsample_args
    if y.ndim != 1:
  File "/Users/jslesar/projects/tsdownsample/venv/lib/python3.10/site-packages/polars/utils/decorators.py", line 169, in _redirecting_getattr_
    return obj.__getattribute__(item)
AttributeError: 'Series' object has no attribute 'ndim'

And changing the line to indices = LTTBDownsampler().downsample(np.array(df["timestamp"]), np.array(df["value"]), n_out=3) was the fix

@jvdd
Copy link
Member

jvdd commented Mar 19, 2023

Appreciate you contributing @jayceslesar 🚀

I haven't had the chance to fully explore the hyperplot project yet, but it's definitely on my radar.

I'll further comment on the UDF/pipe & polars integration suggestions in #30

@jvdd jvdd merged commit a97dc60 into predict-idlab:main Mar 19, 2023
@jvdd jvdd mentioned this pull request Mar 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants