feat(bigquery): add create_bqstorage_client param to to_dataframe and to_arrow
#9573
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
googlebot
added
the
cla: yes
|
feedback during informal review: be clearer we're only using default settings when constructing the client on user behalf, and that to do more you'll still want to supply your own client. |
tswast
changed the title
use_bqstorage_api param to to_dataframe and `…create_bqstorage_client param to to_dataframe and to_arrow
tswast
force-pushed
the
bq-public-data
branch
from
dcbf7dd
to
119f19b
Compare
plamut
reviewed
Nov 4, 2019
There was a problem hiding this comment.
Looks good, I only have a few minor comments.
Making it more clear that auto-creating a BQ storage client behind the scenes only uses the default settings would indeed be beneficial, as @shollyman pointed out.
plamut
approved these changes
Nov 5, 2019
|
|
||
| download_public_data_sandbox.download_public_data_sandbox(client) | ||
| out, _ = capsys.readouterr() | ||
| out, err = capsys.readouterr() |
|
Per #9457 I should close the client transport if we create it in the method. |
tswast
added
the
do not merge
tswast
changed the title
create_bqstorage_client param to to_dataframe and to_arrowcreate_bqstorage_client param to to_dataframe and to_arrow
… and `to_arrow` When the `create_bqstorage_client` parameter is set to `True`, the BigQuery client constructs a BigQuery Storage API client for you. This removes the need for boilerplate code to manually construct both clients explitly with the same credentials. Does this make the `bqstorage_client` parameter unnecessary? In most cases, yes, but there are a few cases where we'll want to continue using it. * When partner tools use `to_dataframe`, they should continue to use `bqstorage_client` so that they can set the correct amended user-agent strings. * When a developer needs to override the default API endpoint for the BQ Storage API, they'll need to manually supply a `bqstorage_client`.
tswast
force-pushed
the
bq-public-data
branch
from
75baaad
to
bf00d45
Compare
tswast
changed the title
create_bqstorage_client param to to_dataframe and to_arrowcreate_bqstorage_client param to to_dataframe and to_arrow
This was referenced Jan 29, 2020
[CHANGE ME] Re-generated container to pick up changes in the API or client library generator.
#10256
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When the
create_bqstorage_clientparameter is set toTrue, the BigQueryclient constructs a BigQuery Storage API client for you. This removes
the need for boilerplate code to manually construct both clients
explicitly with the same credentials.
Does this make the
bqstorage_clientparameter unnecessary? In mostcases, yes, but there are a few cases where we'll want to continue using
it. Specifically, when partner tools use
to_dataframe, they shouldcontinue to use
bqstorage_clientso that they can set the correctamended user-agent strings.
bqstorage_clientis also needed for regionalAPI endpoints.
TODO
owns_bqstorage_clientto track ifbqstorage_clientneeds its transport closed.owns_bqstorage_clientis true.max_resultsis set.