Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for on_schema_change #229

Merged
merged 2 commits into from
Oct 18, 2021
Merged

Add support for on_schema_change #229

merged 2 commits into from
Oct 18, 2021

Conversation

jtcohen6
Copy link
Contributor

@jtcohen6 jtcohen6 commented Oct 14, 2021

resolves #198
resolves #226

Support for:

  • ignore
  • fail
  • append_new_columns
  • sync_all_columns, dropping columns is not supported on Spark/Delta

I'm planning to rebase this against main. After merging, we'll backport to 0.21.latest, since this should have been included to coincide with the introduction of on_schema_change for other adapters in v0.21

Checklist

  • I have signed the CLA
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • I have updated the CHANGELOG.md and added information about my change to the "dbt next" section.

@cla-bot cla-bot bot added the cla:yes label Oct 14, 2021
@jtcohen6 jtcohen6 force-pushed the feature/on-schema-change branch from 8a25095 to 792981b Compare October 14, 2021 19:28
@jtcohen6 jtcohen6 changed the title WIP support for on_schema_change Add support for on_schema_change Oct 14, 2021
@jtcohen6 jtcohen6 marked this pull request as ready for review October 14, 2021 19:52
@jtcohen6 jtcohen6 force-pushed the feature/on-schema-change branch from 792981b to dc0ce42 Compare October 14, 2021 20:03
@jtcohen6 jtcohen6 changed the base branch from 0.21.latest to main October 14, 2021 20:03
@jtcohen6
Copy link
Contributor Author

jtcohen6 commented Oct 14, 2021

The tests pass against 0.21.latest. They started failing after I rebased against main.

In order to get this working, we need to upgrade main to at least v0.21: #231

What we really want is to get this repo upgraded to v1.0.0b1: #227

@jtcohen6 jtcohen6 force-pushed the feature/on-schema-change branch from dc0ce42 to b0c7eae Compare October 15, 2021 15:10
@jtcohen6 jtcohen6 requested a review from leahwicz October 15, 2021 15:28
@jtcohen6 jtcohen6 merged commit 8869b7a into main Oct 18, 2021
@jtcohen6 jtcohen6 deleted the feature/on-schema-change branch October 18, 2021 08:14
jtcohen6 added a commit that referenced this pull request Oct 18, 2021
* Add support for on_schema_change

* Add changelog note
jtcohen6 added a commit that referenced this pull request Oct 18, 2021
* Fix --store-failures (#233)

* Fix --store-failures

* Add chnagelog entry

* Maybe this?

* Fix indentation

* Class inheritance, silly

* Add support for on_schema_change (#229)

* Add support for on_schema_change

* Add changelog note

* add persist_doc call to incremental model with tests (#234)

* add persist_doc call to incremental model with tests

* update changelog

* add whatis to test for incremental

* Reorg changelog

Co-authored-by: Binh Pham <ebinhpham@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incremental models do not fail when schema changes Support new on_schema_change config
2 participants