Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BigQuery: order not preserved when downloading ORDER BY query results to dataframe with BQ Storage API #7759

Closed
tswast opened this issue Apr 19, 2019 · 1 comment · Fixed by #7793
Closed

BigQuery: order not preserved when downloading ORDER BY query results to dataframe with BQ Storage API #7759

tswast opened this issue Apr 19, 2019 · 1 comment · Fixed by #7793
Assignees
@tswast
Labels
api: bigquery Issues related to the BigQuery API. api: bigquerystorage Issues related to the BigQuery Storage API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@tswast
Copy link
Contributor

tswast commented Apr 19, 2019

Since data is downloaded in parallel, order might not be preserved. I believe if you explicitly request only a single stream, order is preserved. (Internal bug 130887748 to confirm this).
@tswast tswast added api: bigquery Issues related to the BigQuery API. api: bigquerystorage Issues related to the BigQuery Storage API. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Apr 19, 2019
@tswast tswast self-assigned this Apr 19, 2019
@tswast
Copy link
Contributor Author

tswast commented Apr 19, 2019

Idea: Add a _preserve_order property to RowIterator. If true, only use one stream. Defualt to false. Set to true in QueryJob.result() if the query string contains "ORDER BY" (case insensitive, any amount of whitespace between ORDER and BY).

@yoshi-automation yoshi-automation added the triage me I really want to be triaged. label Apr 20, 2019
@tswast tswast mentioned this issue Apr 22, 2019
Closed
@tswast tswast added priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. and removed triage me I really want to be triaged. labels Apr 22, 2019
@tswast tswast mentioned this issue Apr 23, 2019
Merged
@yoshi-automation yoshi-automation added 🚨 This issue needs some love. and removed 🚨 This issue needs some love. labels Apr 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tswast tswast

Labels
api: bigquery Issues related to the BigQuery API. api: bigquerystorage Issues related to the BigQuery Storage API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Milestone
No milestone
2 participants
@tswast @yoshi-automation