Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PollingFuture default timeout value object() incompatible with QueryJob typeguard in _reload_query_results #1459

Closed
gbmarc1 opened this issue Jan 10, 2023 · 0 comments · Fixed by #1541
Assignees
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@gbmarc1
Copy link

gbmarc1 commented Jan 10, 2023

google.api_core.future.polling.PollingFuture default timeout value object() incompatible with google.cloud.bigquery.job.query.QueryJob typeguard in _reload_query_results.

A refactor in google-api-core introduced the global _DEFAULT_VALUE in the PollingFuture class ->
here.
This change is incompatible with google-cloud-bigquery typeguard here and becomes invalid and enters the if context which results in a error on this line.

TypeError: unsupported operand type(s) for -: 'object' and 'float'

The typeguard is based on the fact that the default value is None and not object().

As discussed here with @vam-google,

The default value cannot be None and the previous behavior of api-core was a bug. The gist of it is the need to distinguish between infinite timeout (None) and not specified timeout (_DEFAULT_TIMEOUT).

Stack trace

/usr/local/lib/python3.9/site-packages/feast/infra/offline_stores/bigquery.py:583: in block_until_done
--
  | if bq_job.exception():
  | /usr/local/lib/python3.9/site-packages/google/api_core/future/polling.py:282: in exception
  | self._blocking_poll(timeout=timeout)
  | /usr/local/lib/python3.9/site-packages/google/cloud/bigquery/job/query.py:1245: in _blocking_poll
  | super(QueryJob, self)._blocking_poll(timeout=timeout, **kwargs)
  | /usr/local/lib/python3.9/site-packages/google/api_core/future/polling.py:137: in _blocking_poll
  | polling(self._done_or_raise)(retry=retry)
  | /usr/local/lib/python3.9/site-packages/google/api_core/retry.py:349: in retry_wrapped_func
  | return retry_target(
  | /usr/local/lib/python3.9/site-packages/google/api_core/retry.py:191: in retry_target
  | return target()
  | /usr/local/lib/python3.9/site-packages/google/cloud/bigquery/job/query.py:1358: in _done_or_raise
  | self._reload_query_results(retry=retry, timeout=transport_timeout)
  | _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
  |  
  | self = QueryJob<project=..., location=US, id=...>
  | retry = None, timeout = <object object at 0x7fa46677c5c0>
  |  
  | def _reload_query_results(
  | self, retry: "retries.Retry" = DEFAULT_RETRY, timeout: float = None
  | ):
  | """Refresh the cached query results.
  |  
  | Args:
  | retry (Optional[google.api_core.retry.Retry]):
  | How to retry the call that retrieves query results.
  | timeout (Optional[float]):
  | The number of seconds to wait for the underlying HTTP transport
  | before using ``retry``.
  | """
  | if self._query_results and self._query_results.complete:
  | return
  |  
  | # Since the API to getQueryResults can hang up to the timeout value
  | # (default of 10 seconds), set the timeout parameter to ensure that
  | # the timeout from the futures API is respected. See:
  | # https://github.com/GoogleCloudPlatform/google-cloud-python/issues/4135
  | timeout_ms = None
  | if self._done_timeout is not None:
  | # Subtract a buffer for context switching, network latency, etc.
  | >           api_timeout = self._done_timeout - _TIMEOUT_BUFFER_SECS
  | E           TypeError: unsupported operand type(s) for -: 'object' and 'float'
  |  
  | /usr/local/lib/python3.9/site-packages/google/cloud/bigquery/job/query.py:1329: TypeError

Thanks!

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Jan 10, 2023
@tswast tswast added the type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. label Jan 18, 2023
@brijk7 brijk7 added the priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. label Apr 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants