Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigQuery: Support WHERE IN or WHERE ANY clause in DB-API #9177

Closed
haibin opened this issue Sep 5, 2019 · 4 comments · Fixed by #9189
Closed

BigQuery: Support WHERE IN or WHERE ANY clause in DB-API #9177

haibin opened this issue Sep 5, 2019 · 4 comments · Fixed by #9189
Assignees
Labels
api: bigquery Issues related to the BigQuery API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@haibin
Copy link

haibin commented Sep 5, 2019

BigQuery DB-API throws the following error when I pass parameters to the Cursor.execute() for WHERE IN or WHERE ANY clause. Any idea if it will be supported?

  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/cursor.py", line 159, in execute
    query_parameters = _helpers.to_query_parameters(parameters)
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/_helpers.py", line 117, in to_query_parameters
    return to_query_parameters_list(parameters)
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/_helpers.py", line 84, in to_query_parameters_list
    return [scalar_to_query_parameter(value) for value in parameters]
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/_helpers.py", line 84, in <listcomp>
    return [scalar_to_query_parameter(value) for value in parameters]
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/_helpers.py", line 69, in scalar_to_query_parameter
    name, value
google.cloud.bigquery.dbapi.exceptions.ProgrammingError: encountered parameter None with value ['Apple.com', 'Coles'] of unexpected type

@plamut plamut added api: bigquery Issues related to the BigQuery API. type: question Request for information or clarification. Not an issue. labels Sep 5, 2019
@plamut
Copy link
Contributor

plamut commented Sep 5, 2019

@haibin Thanks for the report!

Would you mind providing more details, such as the BigQuery client version, and a reproducible code sample, if possible? That and any other info would be really helpful with investigating the cause, thanks!

@haibin
Copy link
Author

haibin commented Sep 5, 2019

google-cloud-bigquery version: 1.19.0

from google.cloud import bigquery

from google.cloud.bigquery import dbapi

client = bigquery.Client()
conn = dbapi.Connection(client)
curr = conn.cursor()

query = """
  SELECT name, state
  FROM `bigquery-public-data.usa_names.usa_1910_2013`
  WHERE state = %s
  LIMIT 2
"""
curr.execute(query, ('NY', ))
result = curr.fetchall()
print(result)

query = """
  SELECT name, state
  FROM `bigquery-public-data.usa_names.usa_1910_2013`
  WHERE state IN %s
  LIMIT 2
"""
curr.execute(query, (('NY', 'TX'), ))
result = curr.fetchall()
print(result)

Output

[Row(('Mildred', 'NY'), {'name': 0, 'state': 1}), Row(('Irene', 'NY'), {'name': 0, 'state': 1})]
Traceback (most recent call last):
  File "hello_bq.py", line 25, in <module>
    curr.execute(query, (('NY', 'TX'), ))
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/cursor.py", line 159, in execute
    query_parameters = _helpers.to_query_parameters(parameters)
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/_helpers.py", line 117, in to_query_parameters
    return to_query_parameters_list(parameters)
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/_helpers.py", line 84, in to_query_parameters_list
    return [scalar_to_query_parameter(value) for value in parameters]
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/_helpers.py", line 84, in <listcomp>
    return [scalar_to_query_parameter(value) for value in parameters]
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/_helpers.py", line 69, in scalar_to_query_parameter
    name, value
google.cloud.bigquery.dbapi.exceptions.ProgrammingError: encountered parameter None with value ('NY', 'TX') of unexpected type

@fhoffa
Copy link

fhoffa commented Sep 6, 2019

Alternative approach in the meantime:

query = """
  SELECT name, state
  FROM `bigquery-public-data.usa_names.usa_1910_2013`
  WHERE state IN UNNEST(SPLIT(%s))
  LIMIT 2
"""
curr.execute(query, ('NY,TX', ))

@plamut
Copy link
Contributor

plamut commented Sep 8, 2019

I can confirm that the issue is reproducible.

@plamut plamut added priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. and removed type: question Request for information or clarification. Not an issue. labels Sep 8, 2019
@plamut plamut self-assigned this Sep 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants