You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What happened: write_deltalake fails when trying to add file to the transaction because it cannot encode some binary values.
Exception ignored in: 'pyarrow._dataset._filesystemdataset_write_visitor'
Traceback (most recent call last):
File "/Users/abc/repos/study_python/venv/lib/python3.9/site-packages/deltalake/writer.py", line 229, in visitor
json.dumps(stats, cls=DeltaJSONEncoder),
File "/usr/local/homebrew/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py", line 234, in dumps
return cls(
File "/usr/local/homebrew/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/local/homebrew/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/Users/abc/repos/study_python/venv/lib/python3.9/site-packages/deltalake/writer.py", line 315, in default
return obj.decode("unicode_escape")
UnicodeDecodeError: 'unicodeescape' codec can't decode byte 0x5c in position 1: \ at end of string
How to reproduce it:
import pyarrow
from deltalake.writer import write_deltalake
value = b'\x00\\'
pa_table = pyarrow.Table.from_pydict({'field_one': [value]})
write_deltalake("some_table_1234", pa_table)
The text was updated successfully, but these errors were encountered:
Environment
Delta-rs version: 0.7.0
Environment:
Bug
What happened: write_deltalake fails when trying to add file to the transaction because it cannot encode some binary values.
How to reproduce it:
The text was updated successfully, but these errors were encountered: