Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document how to construct external tables #773

Closed
machow opened this issue Jul 15, 2021 · 8 comments · Fixed by #1420 or #1658
Closed

Document how to construct external tables #773

machow opened this issue Jul 15, 2021 · 8 comments · Fixed by #1420 or #1658
Assignees
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: docs Improvement to the documentation for an API.

Comments

@machow
Copy link

machow commented Jul 15, 2021

Is your feature request related to a problem? Please describe.

Currently, the document on creating a table is clear on how to initialize a Table object with a specific fields schema. For example, here is a code snippet from the docs:

image

However, a challenge I faced with these docs is knowing how to create a Table with, e.g., an ExternalConfig set.

Based on the source for Table / unit tests, is the correct approach using a property setter?:

from google.cloud import bigquery
client bigquery.Client()

table = bigquery.Table(...)

# use property setter
table.external_data_configuration = bigquery.ExternalConfig(...)

# edit: it looks like ExternalConfig also needs to use setters, so might include, e.g..
external_config = bigquery.ExternalConfig("NEWLINE_DELIMITED_JSON")
external_config.source_uris = [...]

Describe the solution you'd like

Can we add an example to the docs so that readers learn these two pieces?:

  • Table attributes like schema can be set on __init__ (or via setters)
  • Other Table attributes must be set using setters

It seems like two places users may look for this info is in the Table API doc, and in the usage/tables - creating a table doc. Also, apologies if it's already somewhere in the docs and I missed it 😅.

Describe alternatives you've considered

Since Table can be constructed using .from_api_repr(), I wondered if this is a more common approach for construction? In that case it seems like having a nudge in the docs on using this path for e.g. constructing an external table would be super helpful!

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Jul 15, 2021
@plamut plamut added the type: docs Improvement to the documentation for an API. label Jul 16, 2021
@tswast
Copy link
Contributor

tswast commented Jul 20, 2021

It could also be good to include such a sample here as well: https://cloud.google.com/bigquery/external-table-definition

@tswast
Copy link
Contributor

tswast commented Jul 20, 2021

There is a Python sample here: https://cloud.google.com/bigquery/external-data-cloud-storage#python

@tswast
Copy link
Contributor

tswast commented Jan 30, 2023

Looks like we already had such a code sample here:

# [START bigquery_query_external_gcs_perm]

It appears on cloud.google.com here: https://cloud.google.com/bigquery/docs/external-data-cloud-storage#create_an_external_table

Admittedly, that was a bit difficult for me to find, but it's there. It does need an update, though.

The Python code sample on https://cloud.google.com/bigquery/docs/external-table-definition is a bit misleading, so I propose we remove that one.

@tswast
Copy link
Contributor

tswast commented Feb 1, 2023

  1. Move https://github.com/googleapis/python-bigquery/blob/main/samples/create_table_external_data_configuration.py to samples/snippets (and its corresponding test)

  2. Update the region tag. It should be bigquery_query_external_gcs_perm

  3. Update the docs here: https://cloud.google.com/bigquery/docs/external-data-cloud-storage

  4. Delete the old sample from snippets.py

    # [START bigquery_query_external_gcs_perm]

@meredithslota
Copy link
Contributor

meredithslota commented Aug 17, 2023

PR above addresses (1) and (2) steps in Tim's comment. Edit will reopen PR, there's a few more changes I want to make.

@meredithslota
Copy link
Contributor

There are no Python samples in https://cloud.google.com/bigquery/docs/external-table-definition anymore (removed in internal change 506395040). Submitted a new PR to tackle (1) and (2) above.

@meredithslota meredithslota added the priority: p3 Desirable enhancement or fix. May not be included in next release. label Aug 18, 2023
@meredithslota
Copy link
Contributor

Sent cl/565816568 to update the sample location as per (3) above. Pending.

@meredithslota
Copy link
Contributor

CL to update sample location now submitted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: docs Improvement to the documentation for an API.
Projects
None yet
7 participants