Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decimal logical type is formatted incorrectly by print_schema #713

Closed
sergiimk opened this issue Aug 25, 2021 · 0 comments · Fixed by #721
Closed

Decimal logical type is formatted incorrectly by print_schema #713

sergiimk opened this issue Aug 25, 2021 · 0 comments · Fixed by #721
Labels

Comments

@sergiimk
Copy link
Contributor

Describe the bug
When printing out the schema of a parquet file (written by Spark) using Java implementation of parquet-schema I get:

message spark_schema {
  optional fixed_len_byte_array(9) price_usd (DECIMAL(19,4));
  ...
}

But when I use parquet::schema::printer::print_schema I get:

message spark_schema {
  OPTIONAL FIXED_LEN_BYTE_ARRAY (9) price_usd (DECIMAL4, 19);
  ...
}

Issues:

  • The missing opening bracket (minor)
  • The inversed order of precision and scale

Parquet spec doesn't seem to describe the textual representation of schema, and in Thrift file the order is scale, precision but in SQL representation of DECIMAL (which Java implementation seems to be following) the order is precision{, scale}.

To Reproduce
Use print_schema on a Parquet file containing decimals.

Expected behavior
Logical type is correctly formatted and follows conventional style DECIMAL(precision, scale)

@sergiimk sergiimk added the bug label Aug 25, 2021
sergiimk added a commit to kamu-data/arrow-rs that referenced this issue Aug 26, 2021
alamb pushed a commit that referenced this issue Aug 31, 2021
alamb pushed a commit that referenced this issue Sep 9, 2021
alamb added a commit that referenced this issue Sep 9, 2021
Fixes #713

Co-authored-by: Sergii Mikhtoniuk <mikhtoniuk@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant