Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

Support ColumnarArrowEvalPython operator #394

Closed
xuechendi opened this issue Jul 6, 2021 · 1 comment
Closed

Support ColumnarArrowEvalPython operator #394

xuechendi opened this issue Jul 6, 2021 · 1 comment
Labels
enhancement New feature or request feature

Comments

@xuechendi
Copy link
Collaborator

This feature is aim to support Columnar pandas udf, previously, we will need to convert Columnar to Row to use ArrowEvalPython, which is a redundant work since we already used Arrow as our ColumnarBatch format

Add a new operator called ColumnarArrowEvalPython, which will directly write arrow buffer to socket outputstream for python, and load python replied result to Arrow based ColumnarBatch
@xuechendi xuechendi added the enhancement New feature or request label Jul 6, 2021
@github-actions github-actions bot mentioned this issue Jul 6, 2021
Merged
@zhouyuan
Copy link
Collaborator

closed in #395

@zhixingheyi-tian zhixingheyi-tian added feature request Feature request feature and removed feature request Feature request labels Aug 31, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
enhancement New feature or request feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@zhouyuan @xuechendi @zhixingheyi-tian