Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revamp incremental by period and make it work for BQ #42

Merged
merged 6 commits into from
Apr 25, 2024

Conversation

b-per
Copy link
Contributor

@b-per b-per commented Apr 15, 2024

  • updated some of the integration testing for the insert_by_period materialization
  • fix some existing quoting
  • add support for BQ
  • make sure that integration tests are passing on Postgres + Snowflake + BigQuery (screenshots below)

Postgres:
image

Snowflake:
image

BQ:
image

@dbeatty10 dbeatty10 added the quoting Issues related to dbt's quoting behavior label Apr 15, 2024
Comment on lines 127 to 128
-- finally, drop the existing/backup relation after the commit
{# {{ drop_relation_if_exists(backup_relation) }} #}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this meant to be commented out?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I can remove it entirely. This comes from the OG table materialization but we don't define backup_relation anywhere in this one.

insert_by_period/macros/get_period_sql.sql Show resolved Hide resolved
insert_by_period/macros/get_period_sql.sql Show resolved Hide resolved
Copy link
Contributor

@dbeatty10 dbeatty10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per discussion (primary here):

I didn't personally check how this behaves in dbt-bigquery with different datetime / timestamp fields or different time zone settings.

But approving in the sprit of "experimental features" "not intended as ready-to-use packages".

On that note, here's something we should probably add to the README:

Tip

We recommend clone and install as a local package, or just copy-paste.
But if you choose to ignore that recommendation and use subdirectory within packages.yml to install directly from this repo instead:

To avoid breaking behavior changes affecting your dbt project, you'll probably want to
specify a 40-character hash as the revision in addition to configuring the subdirectory. That way, you won't accidentally receive breaking changes.

@dbeatty10 dbeatty10 merged commit c845674 into main Apr 25, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
quoting Issues related to dbt's quoting behavior
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants