Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature request: to_gbq with a bigframes.pandas.DataFrame should call .to_gbq() from BigQuery DataFrames not download and re-upload #824

Open
tswast opened this issue Nov 5, 2024 · 0 comments
Assignees
Labels
api: bigquery Issues related to the googleapis/python-bigquery-pandas API.

Comments

@tswast
Copy link
Collaborator

tswast commented Nov 5, 2024

Is your feature request related to a problem? Please describe.

Mixing pandas-gbq and BigFrames can be a recipe for confusion and increased costs. Sending a bigframes.pandas.DataFrame to pandas_gbq.to_gbq results in many queries as pandas-gbq tries to determine the appropriate schema to use for upload.

Describe the solution you'd like

Ideally, if bigframes is installed, pandas-gbq can detect if someone is trying to use a BigQuery DataFrames object with it and call the .to_gbq() method on the object, which would avoid unnecessary queries, a download, and a re-upload of the data.

Describe alternatives you've considered

  • Make BigFrames.pandas.DataFrame less compatible with the pandas interface to avoid confusion by pandas-gbq.
  • Still download and re-upload but skip the schema detection, which should be unnecessary with a BigFrames DataFrame.

Additional context

See internal issue 374235961

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-pandas API. label Nov 5, 2024
@shollyman shollyman assigned chalmerlowe and Linchin and unassigned whuffman36 Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-pandas API.
Projects
None yet
Development

No branches or pull requests

4 participants