-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[C++][Java][FlightRPC] Add support for ordered data #34852
Labels
Breaking Change
Includes a breaking change to the API
Component: C++
Component: FlightRPC
Component: Java
Type: enhancement
Milestone
Comments
kou
added a commit
to kou/arrow
that referenced
this issue
Apr 17, 2023
kou
added a commit
to kou/arrow
that referenced
this issue
Apr 19, 2023
kou
added a commit
to kou/arrow
that referenced
this issue
Apr 20, 2023
kou
changed the title
[C++][FlightRPC] Add support for ordered data
[C++][Java][FlightRPC] Add support for ordered data
Apr 27, 2023
kou
added a commit
that referenced
this issue
May 9, 2023
…5178) ### Rationale for this change No ordering is unnecessarily limiting. Systems can and do implement distributed sorts, but they can’t reflect this in Flight RPC. ### What changes are included in this PR? These changes add `FlightInfo.ordered`. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. **This PR includes breaking changes to public APIs.** * Closes: #34852 * Closes: #35085 Lead-authored-by: Sutou Kouhei <kou@clear-code.com> Co-authored-by: David Li <li.davidm96@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
kou
changed the title
[C++][Java][FlightRPC] Add support for ordered data
[C++][Java][Go][FlightRPC] Add support for ordered data
May 9, 2023
kou
changed the title
[C++][Java][Go][FlightRPC] Add support for ordered data
[C++][Java][FlightRPC] Add support for ordered data
May 9, 2023
liujiacheng777
pushed a commit
to LoongArch-Python/arrow
that referenced
this issue
May 11, 2023
…ta (apache#35178) ### Rationale for this change No ordering is unnecessarily limiting. Systems can and do implement distributed sorts, but they can’t reflect this in Flight RPC. ### What changes are included in this PR? These changes add `FlightInfo.ordered`. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. **This PR includes breaking changes to public APIs.** * Closes: apache#34852 * Closes: apache#35085 Lead-authored-by: Sutou Kouhei <kou@clear-code.com> Co-authored-by: David Li <li.davidm96@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ArgusLi
pushed a commit
to Bit-Quill/arrow
that referenced
this issue
May 15, 2023
…ta (apache#35178) ### Rationale for this change No ordering is unnecessarily limiting. Systems can and do implement distributed sorts, but they can’t reflect this in Flight RPC. ### What changes are included in this PR? These changes add `FlightInfo.ordered`. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. **This PR includes breaking changes to public APIs.** * Closes: apache#34852 * Closes: apache#35085 Lead-authored-by: Sutou Kouhei <kou@clear-code.com> Co-authored-by: David Li <li.davidm96@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
rtpsw
pushed a commit
to rtpsw/arrow
that referenced
this issue
May 16, 2023
…ta (apache#35178) ### Rationale for this change No ordering is unnecessarily limiting. Systems can and do implement distributed sorts, but they can’t reflect this in Flight RPC. ### What changes are included in this PR? These changes add `FlightInfo.ordered`. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. **This PR includes breaking changes to public APIs.** * Closes: apache#34852 * Closes: apache#35085 Lead-authored-by: Sutou Kouhei <kou@clear-code.com> Co-authored-by: David Li <li.davidm96@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
7 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Breaking Change
Includes a breaking change to the API
Component: C++
Component: FlightRPC
Component: Java
Type: enhancement
Describe the enhancement requested
Based on the proposal in https://docs.google.com/document/d/1jhPyPZSOo2iy0LqIJVUs9KWPyFULVFJXTILDfkadx2g/edit# .
See also the discussion thread: https://lists.apache.org/thread/247z3t06mf132nocngc1jkp3oqglz7jp
Currently, the endpoints within a FlightInfo explicitly have no ordering:
This is unnecessarily limiting. Systems can and do implement distributed sorts, but they can’t reflect this in Flight RPC. And users are asking; see this Stack Overflow question.
Proposal:
Add a flag to
FlightInfo
:If set, the client may assume that the data is sorted in the same order as the endpoints. Otherwise, the client cannot make any assumptions (as before).
Component(s)
C++, Java,FlightRPC
The text was updated successfully, but these errors were encountered: