Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C++][Python] Expose recursive flatten for lists on list_flatten kernel function and pyarrow bindings #41183

Closed
ZhangHuiGui opened this issue Apr 13, 2024 · 1 comment
Closed

[C++][Python] Expose recursive flatten for lists on list_flatten kernel function and pyarrow bindings #41183

ZhangHuiGui opened this issue Apr 13, 2024 · 1 comment
Assignees
@ZhangHuiGui
Labels
Component: C++ Component: Python Type: enhancement
Milestone
17.0.0

Comments

@ZhangHuiGui
Copy link
Collaborator

Describe the enhancement requested

Support flatten recursively for kernel function list_flatten and related pyarrow bindings.

Two things should be included:

  1. We should support an option for deciding whether to recursively flatten a list. This option should default to false for list_flatten and pyarrow bindings.
  2. Expose the FlattenRecursively api which comes from (GH-41055: [C++] Support flatten for combining nested list related types #41092) to pyarrow related LogicalList Array bindings.

Component(s)

C++, Python
@felipecrv felipecrv changed the title [C++][Python] Support flatten recursively for kernel function list_flatten and related pyarrow bindings [C++][Python] Expose recursive flatten for lists on list_flatten kernel function and pyarrow bindings Apr 13, 2024
@github-actions github-actions bot mentioned this issue Apr 18, 2024
Merged
felipecrv pushed a commit that referenced this issue Apr 30, 2024
@ZhangHuiGui
GH-41183: [C++][Python] Expose recursive flatten for lists on list_fl…
5e986be 
…atten kernel function and pyarrow bindings (#41295)

### Rationale for this change
Expose recursive flatten for logical lists on list_flatten kernel function and pyarrow bindings.

### What changes are included in this PR?
1. Expose recursive flatten for logical lists on `list_flatten` kernel function
2. Support [Large]ListView for some kernel functions: `list_flatten`,`list_value_length`, `list_element`
3. Support recursive flatten for pyarrow bindinds and simplify [Large]ListView's pyarrow bindings
4. Refactor vector_nested_test.cc for better support [Large]ListView types.

### Are these changes tested?
Yes

### Are there any user-facing changes?
Yes.
1. Some kernel functions like: list_flatten, list_value_length, list_element would support [Large]ListView types
2. `list_flatten` and related pyarrow bindings could support flatten recursively with an ListFlattenOptions.

* GitHub Issue: #41183

Lead-authored-by: ZhangHuiGui <hugo.zhang@openpie.com>
Co-authored-by: ZhangHuiGui <2689496754@qq.com>
Signed-off-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
@felipecrv
Copy link
Contributor

Issue resolved by pull request 41295
#41295

@felipecrv felipecrv added this to the 17.0.0 milestone Apr 30, 2024
tolleybot pushed a commit to tmct/arrow that referenced this issue May 2, 2024
@ZhangHuiGui @tolleybot
apacheGH-41183: [C++][Python] Expose recursive flatten for lists on l…
e1abb14 
…ist_flatten kernel function and pyarrow bindings (apache#41295)

### Rationale for this change
Expose recursive flatten for logical lists on list_flatten kernel function and pyarrow bindings.

### What changes are included in this PR?
1. Expose recursive flatten for logical lists on `list_flatten` kernel function
2. Support [Large]ListView for some kernel functions: `list_flatten`,`list_value_length`, `list_element`
3. Support recursive flatten for pyarrow bindinds and simplify [Large]ListView's pyarrow bindings
4. Refactor vector_nested_test.cc for better support [Large]ListView types.

### Are these changes tested?
Yes

### Are there any user-facing changes?
Yes.
1. Some kernel functions like: list_flatten, list_value_length, list_element would support [Large]ListView types
2. `list_flatten` and related pyarrow bindings could support flatten recursively with an ListFlattenOptions.

* GitHub Issue: apache#41183

Lead-authored-by: ZhangHuiGui <hugo.zhang@openpie.com>
Co-authored-by: ZhangHuiGui <2689496754@qq.com>
Signed-off-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
vibhatha pushed a commit to vibhatha/arrow that referenced this issue May 25, 2024
@ZhangHuiGui @vibhatha
apacheGH-41183: [C++][Python] Expose recursive flatten for lists on l…
c9676d9 
…ist_flatten kernel function and pyarrow bindings (apache#41295)

### Rationale for this change
Expose recursive flatten for logical lists on list_flatten kernel function and pyarrow bindings.

### What changes are included in this PR?
1. Expose recursive flatten for logical lists on `list_flatten` kernel function
2. Support [Large]ListView for some kernel functions: `list_flatten`,`list_value_length`, `list_element`
3. Support recursive flatten for pyarrow bindinds and simplify [Large]ListView's pyarrow bindings
4. Refactor vector_nested_test.cc for better support [Large]ListView types.

### Are these changes tested?
Yes

### Are there any user-facing changes?
Yes.
1. Some kernel functions like: list_flatten, list_value_length, list_element would support [Large]ListView types
2. `list_flatten` and related pyarrow bindings could support flatten recursively with an ListFlattenOptions.

* GitHub Issue: apache#41183

Lead-authored-by: ZhangHuiGui <hugo.zhang@openpie.com>
Co-authored-by: ZhangHuiGui <2689496754@qq.com>
Signed-off-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ZhangHuiGui ZhangHuiGui

Labels
Component: C++ Component: Python Type: enhancement
Projects
None yet
Milestone
17.0.0
Development

No branches or pull requests
2 participants
@felipecrv @ZhangHuiGui