feat: add Client.query_and_wait which directly returns a RowIterator of results
#1722
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
product-auto-label
bot
added
size: m
chalmerlowe
reviewed
Nov 16, 2023
| timeout: Optional[float], | ||
| job_retry: retries.Retry, | ||
| ) -> table.RowIterator: | ||
| """Initiate a query using jobs.query with jobCreationMode=JOB_CREATION_REQUIRED. |
There was a problem hiding this comment.
Should this be JOB_CREATION_OPTIONAL?
It seems that way based on the descriptions above.
There was a problem hiding this comment.
You're correct. I'll update.
…_is_completely_cached
4 tasks
…_is_completely_cached
…or` of results Set the `QUERY_PREVIEW_ENABLED=TRUE` environment variable to use this with the new JOB_CREATION_OPTIONAL mode (currently in preview).
tswast
force-pushed
the
issue589-query_and_wait
branch
from
93d47e7
to
c16e4be
Compare
product-auto-label
bot
added
size: l
tswast
changed the base branch from
main
to
issue589-RowIterator._is_completely_cached
Base automatically changed from
issue589-RowIterator._is_completely_cached
to
main
November 21, 2023 15:13
product-auto-label
bot
added
size: xl
mmladenovski
approved these changes
Dec 5, 2023
| elif page_size is not None or max_results is not None: | ||
| request_body["maxResults"] = page_size or max_results | ||
|
|
||
| if os.getenv("QUERY_PREVIEW_ENABLED", "").casefold() == "true": |
There was a problem hiding this comment.
@tswast , in our previous discussion, I only suggested for this environment variable to use a list of project IDs for a more fine-grained control. We have discussed it with @shollyman , and for the Java client we decided to go with a boolean. I am fine either way, staying consistent with a boolean or doing a more fine-grained project id list (if required by python client usage).
This was being reset in some cases when the rows were all available in the first page of results.
4 tasks
chalmerlowe
reviewed
Dec 8, 2023
| if default_job_config is None: | ||
| return job_config | ||
|
|
||
| # anything that's not defined on the incoming |
There was a problem hiding this comment.
This comment is a bit unclear. Can we clean this up a bit?
chalmerlowe
reviewed
Dec 8, 2023
| # should be filled in with the default | ||
| # the incoming therefore has precedence | ||
| # | ||
| # Note that _fill_from_default doesn't mutate the receiver |
There was a problem hiding this comment.
For my own edification, what do we mean by "receiver"?
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
chalmerlowe
approved these changes
Dec 8, 2023
This was referenced Dec 19, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Set the
QUERY_PREVIEW_ENABLED=TRUEenvironment variable to use this with the new JOB_CREATION_OPTIONAL mode (currently in preview).Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Note: Currently in DRAFT mode as I realized that some of the necessary work to speed things up overlaps with the
query(api_method="QUERY")work (#1723), so I'll work on modifying that first before moving on to the next step of a "jobless" API.Towards #589 🦕