Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Destination bigquery: pull in cdk update for refreshes bugfix #42504

Merged
merged 1 commit into from
Aug 20, 2024

Conversation

edgao
Copy link
Contributor

@edgao edgao commented Jul 24, 2024

mostly just updates to make it compile against latest CDK. The big thing is adding the ability to fetch the generation ID of the final table in BQDestinationHandler.

@edgao edgao requested a review from a team as a July 24, 2024 17:59
Copy link

vercel bot commented Jul 24, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 19, 2024 6:12pm

Copy link
Contributor Author

edgao commented Jul 24, 2024

@edgao edgao force-pushed the edgao/refreshes_recover_from_real_raw_table branch from d59f5d8 to 3d9cf10 Compare July 24, 2024 18:19
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch from 9eb5cfc to cea9896 Compare July 24, 2024 18:20
@edgao edgao force-pushed the edgao/refreshes_recover_from_real_raw_table branch from 3d9cf10 to 341ae0d Compare July 26, 2024 17:45
@edgao edgao force-pushed the edgao/refreshes_recover_from_real_raw_table branch 2 times, most recently from 8a53e44 to 3922cf7 Compare July 26, 2024 22:44
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch from cea9896 to deec8a0 Compare July 26, 2024 22:47
@octavia-squidington-iii octavia-squidington-iii removed the area/connectors Connector related issues label Jul 26, 2024
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch from deec8a0 to 84a849b Compare July 26, 2024 22:55
@octavia-squidington-iii octavia-squidington-iii added the area/connectors Connector related issues label Jul 26, 2024
@edgao edgao force-pushed the edgao/refreshes_recover_from_real_raw_table branch from 3922cf7 to b9bf10d Compare July 29, 2024 16:33
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch from 84a849b to bf40a10 Compare July 29, 2024 16:34
@edgao edgao force-pushed the edgao/refreshes_recover_from_real_raw_table branch from b9bf10d to 087b55b Compare August 2, 2024 17:25
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch from bf40a10 to 6d39d04 Compare August 2, 2024 17:25
@edgao edgao force-pushed the edgao/refreshes_recover_from_real_raw_table branch from 087b55b to 7fdef81 Compare August 2, 2024 19:59
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch from 6d39d04 to cefde10 Compare August 2, 2024 19:59
@edgao edgao force-pushed the edgao/refreshes_recover_from_real_raw_table branch from 7fdef81 to ad24e27 Compare August 6, 2024 17:11
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch from cefde10 to 8853128 Compare August 6, 2024 17:11
@edgao edgao force-pushed the edgao/refreshes_recover_from_real_raw_table branch from ad24e27 to 1d5f768 Compare August 9, 2024 22:46
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch from 8853128 to 1d826e0 Compare August 9, 2024 22:46
@octavia-squidington-iv octavia-squidington-iv requested a review from a team August 13, 2024 22:28
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch from d266507 to 5c35ee7 Compare August 14, 2024 15:31
@edgao edgao force-pushed the edgao/more_refresh_tests branch from 19305f5 to 23a01c7 Compare August 14, 2024 18:19
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch from 5c35ee7 to 823d610 Compare August 14, 2024 18:19
@edgao
Copy link
Contributor Author

edgao commented Aug 14, 2024

realized that (a) bigquery doesn't have coverage on generation ID migration on an overwrite stream, and (b) there's probably a bug in how we handle that migration (discovered during snowflake+redshift implementation)

copying over redshift's overwrite test case + fixing the bug.

@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch 2 times, most recently from a6ef76a to da301df Compare August 15, 2024 15:23
@octavia-squidington-iv octavia-squidington-iv requested a review from a team August 15, 2024 15:24
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch from da301df to ae4ddcf Compare August 15, 2024 16:07
@edgao edgao force-pushed the edgao/more_refresh_tests branch from 23a01c7 to 5fb7304 Compare August 15, 2024 16:26
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch 2 times, most recently from 4496d75 to c1a4857 Compare August 15, 2024 17:55
@octavia-squidington-iii octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Aug 15, 2024
@edgao edgao force-pushed the edgao/more_refresh_tests branch from 5fb7304 to 95ac402 Compare August 16, 2024 20:10
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch from c1a4857 to 27fd278 Compare August 16, 2024 20:10
Base automatically changed from edgao/more_refresh_tests to edgao/refreshes_recover_from_real_raw_table August 19, 2024 16:27
@edgao edgao force-pushed the edgao/refreshes_recover_from_real_raw_table branch 2 times, most recently from 7270022 to 840e346 Compare August 19, 2024 17:15
Base automatically changed from edgao/refreshes_recover_from_real_raw_table to master August 19, 2024 18:06
@edgao edgao force-pushed the edgao/bigquery_refreshes_bugfix branch from 27fd278 to 3cf24d5 Compare August 19, 2024 18:07
@edgao edgao merged commit b20365a into master Aug 20, 2024
34 checks passed
@edgao edgao deleted the edgao/bigquery_refreshes_bugfix branch August 20, 2024 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/destination/bigquery
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants