Remove cudf._lib.orc in favor of inlining pylibcudf #17466
Conversation
mroeschke
added
Python
github-actions
bot
added
CMake
python/cudf/cudf/utils/ioutils.py
Outdated
| # pyarrow. To make sure the cudf format is interperable | ||
| # in arrow, we use `int8` type when converting from a |
There was a problem hiding this comment.
| # pyarrow. To make sure the cudf format is interperable | |
| # in arrow, we use `int8` type when converting from a | |
| # pyarrow. To make sure the cudf format is interoperable | |
| # with arrow, we use `int8` type when converting from a |
python/cudf/cudf/utils/ioutils.py
Outdated
| try: | ||
| import ujson as json # type: ignore[import-untyped] | ||
| except ImportError: | ||
| import json |
There was a problem hiding this comment.
question: Why this? I think we're only using this to load the (tiny) metadata json dict, so it seems perhaps unnecessary?
There was a problem hiding this comment.
I'm fine with just using json; I was just porting over the equivalent behavior
| def __init__(self, Table table): | ||
| self.c_obj = table_input_metadata(table.view()) | ||
| self.column_metadata = [ | ||
|
|
||
| @property | ||
| def column_metadata(self): | ||
| return [ | ||
| ColumnInMetadata.from_libcudf(&self.c_obj.column_metadata[i], self) | ||
| for i in range(self.c_obj.column_metadata.size()) | ||
| ] | ||
|
|
There was a problem hiding this comment.
question: What is this change needed for?
There was a problem hiding this comment.
Before this was an (unused) cdef attribute, but cudf Python ORC handling modifies the column_metadata inplace, so I needed to expose this attribute in Python space.
There was a problem hiding this comment.
Can you add it as a property in types.pyi if it is not already done?
There was a problem hiding this comment.
Sure. Added in 5e27f2a
|
/merge |
rapids-bot
bot
merged commit b6f7e6e
into
rapidsai:branch-25.02
Description
Contributes to #17317
Checklist