-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix new partitioned table #3381
Fix new partitioned table #3381
Conversation
…w-partitioned-table
[This is a re-do of the pull request googleapis#3366 to fix the author email address to use my google.com email] I moved the partitioning_type check into the if/elif block to avoid a ValueError that should not have been thrown. See: https://cloud.google.com/bigquery/docs/creating-partitioned-tables#creat ing_a_partitioned_table and the "API" example. Before my change, the following code failed with the ValueError thrown by this function, but with my change, I was able to successfully create the partitioned table: destination_table = dataset.table(table_shortname) destination_partition = dataset.table(table_shortname+'$'+partition_date) if not destination_table.exists(): destination_table.partitioning_type = 'DAY' destination_table.create()
@dhermes @jonparrott |
Thanks @richkadel. This LGTM. @tswast Any issues? |
@richkadel Your code snippet above never uses |
raise ValueError("Set either 'view_query' or 'schema'.") | ||
elif self.partitioning_type is None: | ||
raise ValueError( | ||
"Set either 'view_query' or 'schema' or 'partitioning_type'.") |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
@dhermes re: @richkadel Your code snippet above never uses destination_partition. Is there a typo somewhere? No typo. The code that uses the partition itself is not shown. I left the line in to show that the table name used in the "create" call is the table name without the partition suffix. Once the table is created then you can use destination_partition value as the "job_destination" setting to materialize data into just that partition. |
@richkadel Gotcha, I just made an edit to make that clearer. |
As discussed with team, I removed the ValueError and any assumptions about the legitimate combinations of values in the request. ValueError unit test was removed, and the unused schema in `test_create_w_alternate_client` that was suppressed due to the `elif` in `_build_resource`, was removed so the expected value still matches the remaining inputs.
Updates have been pushed and all tests pass. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!
OK @jonparrott also thinks this looks just fine. |
[This is a re-do of the pull request #3366 to fix the author email address to use my
google.com
email]I moved the
partitioning_type
check into theif
/elif
block to avoid aValueError
that should not have been thrown.See:
https://cloud.google.com/bigquery/docs/creating-partitioned-tables#creating_a_partitioned_table and the "API" example.
Before my change, the following code failed with the
ValueError
thrown by this function, but with my change, I was able to successfully create the partitioned table: