Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement sum for decimal type. #14652

Closed
2 tasks done
PongPong opened this issue Feb 23, 2024 · 2 comments · Fixed by #14732
Closed
2 tasks done

Implement sum for decimal type. #14652

PongPong opened this issue Feb 23, 2024 · 2 comments · Fixed by #14732
Labels
A-dtype-decimal Area: decimal data type bug Something isn't working P-low Priority: low python Related to Python Polars

Comments

@PongPong
Copy link

PongPong commented Feb 23, 2024

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

    import polars as pl

    pl.Config(activate_decimals = True)
    df = pl.DataFrame({
        'a':['1.1', '2.2']
    }).cast(pl.Decimal(3, 1))
    print(df)
    print(df.sum())
    
    shape: (2, 1)
┌──────────────┐
│ a            │
│ ---          │
│ decimal[3,1] │
╞══════════════╡
│ 1.1          │
│ 2.2          │
└──────────────┘
shape: (1, 1)
┌──────────────┐
│ a            │
│ ---          │
│ decimal[3,1] │
╞══════════════╡
│ null         │
└──────────────┘

Log output

No response

Issue description

the sum operation of decimal fields returns null

Expected behavior

this is the behavior of Pandas and PyArrow

import pandas as pd
import pyarrow as pa
pa_dtype = pa.decimal128(precision=3, scale=1)
df = pd.DataFrame({
    'a':['1.1', '2.2']
}, dtype=pd.ArrowDtype(pa_dtype))
print(df.sum())

Output:

a    3.3
dtype: decimal128(3, 1)[pyarrow]

Installed versions

--------Version info---------
Polars:               0.20.10
Index type:           UInt32
Platform:             macOS-14.3.1-arm64-arm-64bit
Python:               3.10.5 (main, Jul 14 2022, 12:56:46) [Clang 13.1.6 (clang-1316.0.21.2.5)]

----Optional dependencies----
adbc_driver_manager:  0.10.0
cloudpickle:          3.0.0
connectorx:           0.3.2
deltalake:            0.15.3
fsspec:               2023.4.0
gevent:               24.2.1
hvplot:               0.9.2
matplotlib:           3.7.2
numpy:                1.24.2
openpyxl:             <not installed>
pandas:               2.2.0
pyarrow:              15.0.0
pydantic:             2.6.1
pyiceberg:            0.6.0
pyxlsb:               <not installed>
sqlalchemy:           2.0.27
xlsx2csv:             0.8.2
xlsxwriter:           3.2.0
None
@PongPong PongPong added bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars labels Feb 23, 2024
@PongPong PongPong changed the title sum for decimal is not working *sum()* for decimal is not working Feb 23, 2024
@PongPong PongPong changed the title *sum()* for decimal is not working **sum()** for decimal is not working Feb 23, 2024
@PongPong PongPong changed the title **sum()** for decimal is not working sum() for decimal is not working Feb 23, 2024
@ritchie46
Copy link
Member

Decimal support is experimental so this isn't implemented yet.

@ritchie46 ritchie46 added the enhancement New feature or an improvement of an existing feature label Feb 25, 2024
@ritchie46 ritchie46 changed the title sum() for decimal is not working Implement sum for decimal type. Feb 25, 2024
@stinodego
Copy link
Contributor

This is actually implemented now:

import polars as pl

df = pl.DataFrame({"a": ["1.1", "2.2"]}).cast(pl.Decimal(3, 1))
result = df.select(pl.col("a").sum())
print(result)
shape: (1, 1)
┌──────────────┐
│ a            │
│ ---          │
│ decimal[3,1] │
╞══════════════╡
│ 3.3          │
└──────────────┘

However, the DataFrame.sum() method does not handle Decimals properly:

shape: (1, 1)
┌──────────────┐
│ a            │
│ ---          │
│ decimal[3,1] │
╞══════════════╡
│ null         │
└──────────────┘

So I would actually classify this as a bug.

@stinodego stinodego added A-dtype-decimal Area: decimal data type P-low Priority: low and removed enhancement New feature or an improvement of an existing feature needs triage Awaiting prioritization by a maintainer labels Feb 26, 2024
@github-project-automation github-project-automation bot moved this to Ready in Backlog Feb 26, 2024
@github-project-automation github-project-automation bot moved this from Ready to Done in Backlog Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-dtype-decimal Area: decimal data type bug Something isn't working P-low Priority: low python Related to Python Polars
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants