Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Numeric overflow when formatting Decimal type #710

Closed
sergiimk opened this issue Aug 24, 2021 · 0 comments · Fixed by #722
Closed

Numeric overflow when formatting Decimal type #710

sergiimk opened this issue Aug 24, 2021 · 0 comments · Fixed by #722
Labels

Comments

@sergiimk
Copy link
Contributor

Describe the bug
When attempting to print out records using arrow::util::pretty::print_batches I get the following panic:

thread 'main' panicked at 'attempt to subtract with overflow', /home/smikhtoniuk/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow-5.2.0/src/array/array_binary.rs:751:50
stack backtrace:
   0: rust_begin_unwind
             at /rustc/d9aa28767287670df6cf823b94629122e04442c0/library/std/src/panicking.rs:515:5
   1: core::panicking::panic_fmt
             at /rustc/d9aa28767287670df6cf823b94629122e04442c0/library/core/src/panicking.rs:92:14
   2: core::panicking::panic
             at /rustc/d9aa28767287670df6cf823b94629122e04442c0/library/core/src/panicking.rs:50:5
   3: arrow::array::array_binary::DecimalArray::value_as_string
             at /home/smikhtoniuk/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow-5.2.0/src/array/array_binary.rs:751:50
   4: arrow::util::display::make_string_from_decimal
             at /home/smikhtoniuk/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow-5.2.0/src/util/display.rs:204:29
   5: arrow::util::display::array_value_to_string
             at /home/smikhtoniuk/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow-5.2.0/src/util/display.rs:261:34
   6: arrow::util::pretty::create_table
             at /home/smikhtoniuk/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow-5.2.0/src/util/pretty.rs:74:39
   7: arrow::util::pretty::print_batches
             at /home/smikhtoniuk/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow-5.2.0/src/util/pretty.rs:42:5

Schema of the parquet file I'm reading:

message spark_schema {
  required int96 system_time;
  optional int96 event_time;
  optional binary desc (STRING);
  optional int64 delta (DECIMAL(13,4));
}

To Reproduce
I would provide the parquet file, but the dataset has sensitive info. Let me know if you can't reproduce and I'll create fake data.

Expected behavior
Decimal values print out without panics

@sergiimk sergiimk added the bug label Aug 24, 2021
sergiimk added a commit to kamu-data/arrow-rs that referenced this issue Aug 26, 2021
sergiimk added a commit to kamu-data/arrow-rs that referenced this issue Aug 26, 2021
sergiimk added a commit to kamu-data/arrow-rs that referenced this issue Aug 26, 2021
alamb pushed a commit that referenced this issue Aug 29, 2021
alamb pushed a commit that referenced this issue Sep 9, 2021
alamb added a commit that referenced this issue Sep 9, 2021
Fixes #710

Co-authored-by: Sergii Mikhtoniuk <mikhtoniuk@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant