We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Delta-rs version: 0.10.0
Binding:
Environment:
**Exception: Schema error: **:
I am trying to write to an already existing deltalake destination in s3 using delta
import pyarrow.parquet as pq from deltalake.writer import write_deltalake df = pq.read_table('data') storage_options = { “AWS_DEFAULT_REGION”: “eu-central-1", “AWS_ACCESS_KEY_ID”: os.environ[“AWS_ACCESS_KEY_ID”], “AWS_SECRET_ACCESS_KEY”: os.environ[“AWS_SECRET_ACCESS_KEY”], “AWS_S3_ALLOW_UNSAFE_RENAME”: “true”, } destination = "s3://some_s3_location" write_deltalake(destination, df, mode=“append”, storage_options=storage_options, partition_by=[“titles”, "train_title", "date"] )
The data gets written to s3 but somehow I get this exception Invalid data type for Delta Lake: Dictionary(Int32, Utf8)
Invalid data type for Delta Lake: Dictionary(Int32, Utf8)
Any clue why this is happening? I played around it little bit and seem there is something wrong with data type of the partition keys folders
The text was updated successfully, but these errors were encountered:
I believe this is the same issue as #1445, which will be fixed in #1481
Sorry, something went wrong.
@yefetBenTili are you still having this bug?
No branches or pull requests
Environment
Delta-rs version: 0.10.0
Binding:
Environment:
**Exception: Schema error: **:
I am trying to write to an already existing deltalake destination in s3 using delta
The data gets written to s3 but somehow I get this exception
Invalid data type for Delta Lake: Dictionary(Int32, Utf8)
Any clue why this is happening?
I played around it little bit and seem there is something wrong with data type of the partition keys folders
The text was updated successfully, but these errors were encountered: