Skip to content

ENH: Read BigQuery query table without a query #266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
tswast opened this issue Mar 28, 2019 · 2 comments · Fixed by #443
Closed

ENH: Read BigQuery query table without a query #266

tswast opened this issue Mar 28, 2019 · 2 comments · Fixed by #443
Assignees
@tswast
Labels
api: bigquery Issues related to the googleapis/python-bigquery-pandas API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@tswast
Copy link
Collaborator

tswast commented Mar 28, 2019

It can cost a lot to run a SELECT * FROM my_table query, and yet from an API perspective, this query is completely unnecessary, as one can list the rows from the table directly for free with tabledata.list. I propose

df = pandas_gbq.read_gbq("my-project.my_dataset.my_table")

which will download a table directly (using bigquery_client.list_rows().to_dataframe()). Since all queries must contain some kind of whitespace and table IDs cannot contain any whitespace, we can use that to disambiguate query versus table ID.
@tswast tswast added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Mar 28, 2019
@max-sixty
Copy link
Contributor

I didn't know about that - I had thought you could run a preview, but not download a whole table that way.

Not opposed to it. I'm not sure it's that helpful given than a large enough table to be expensive to query is also impractical to download over JSON into memory.
But that's not an argument to not have the functionality, am +0 !

@tswast tswast mentioned this issue Apr 16, 2019
Closed
@tswast tswast mentioned this issue Aug 26, 2019
Closed
2 tasks
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-pandas API. label Jul 17, 2021
@tswast tswast self-assigned this Dec 6, 2021
@tswast
Copy link
Collaborator Author

tswast commented Dec 6, 2021

I think this feature would help with testing #365, as I realized I'm using the google-cloud-bigquery bigquery_client.list_rows(table_id).to_dataframe(...) pattern in #442 which doesn't actually test the full round trip via pandas-gbq.

@tswast tswast mentioned this issue Dec 6, 2021
Merged
4 tasks
gcf-merge-on-green bot pushed a commit that referenced this issue Dec 22, 2021
@tswast
feat: accepts a table ID, which downloads the table without a query (#…

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode
bf0e863 
…443)

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery-pandas/issues/new/choose) before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)

Fixes #266  🦕
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tswast tswast

Labels
api: bigquery Issues related to the googleapis/python-bigquery-pandas API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: accepts a table ID, which downloads the table without a query tswast/python-bigquery-pandas
2 participants
@tswast @max-sixty