Skip to content

Commit

Permalink
Support cupy and numpy array types in flatten_list_column_values (#251
Browse files Browse the repository at this point in the history
)

The motivation for this comes from:
- `pd.api.types.is_list_like` return value for cupy array's changes
depending on version of the pandas package.
- Normalize op in NVTabular is expected to work on dictionaries of
cupy arrays. It calls `flatten_list_column_values` if the array is list-like.
  • Loading branch information
oliverholworthy authored Mar 20, 2023
1 parent 52358b1 commit df9f026
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion merlin/core/dispatch.py
Original file line number Diff line number Diff line change
Expand Up @@ -374,9 +374,16 @@ def flatten_list_column_values(s):
return pd.Series(itertools.chain(*s))
elif cudf and isinstance(s, cudf.Series):
return s.list.leaves
elif cp and isinstance(s, cp.ndarray):
return s.flatten()
elif isinstance(s, np.ndarray):
return s.flatten()
else:
raise ValueError(
"Unsupported series type: " f"{type(s)}" " Expected either a pandas or cudf Series."
"Unsupported series type: "
f"{type(s)} "
"Expected either a pandas or cuDF Series. "
"Or a NumPy or CuPy array"
)


Expand Down

0 comments on commit df9f026

Please sign in to comment.