Handle hard deletes in snapshots

[clrcrl]

resolves #249

Related Discourse post

Description

When a record is deleted from a snapshot query result set, dbt_valid_to will be updated in the target snapshot as the current timestamp.

Checklist

• I have signed the CLA
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have updated the CHANGELOG.md and added information about my change to the "dbt next" section.

[jrandrews]

Will there be any options to disable this behavior if we want and have snapshots just ignore deletes?

I ask because we've occasionally had issues where our source tables had all the rows truncated out of them but new rows were not inserted (thanks BigQuery!) I think what would happen in that scenario is that all the rows in our snapshot would be end-dated because the source table would have no rows in it, correct? We'd like to avoid that behavior.

Also, what would snapshots do if a row was end-dated because of being hard-deleted in the source, but then subsequently a new row with the same common PK value showed in up the source? E.g. in our scenario if all the rows in the snapshotwith NULL end-dates were first end-dated because the source was truncated but not reloaded, but then the next go-around the source table had been repopulated again?

[clrcrl]

So the context here is that I wrote this code with a community member the other day, and didn't want to lose it! I don't think we're going to merge this as-is, it needs a lot more testing. For now, just leaving it open as a draft PR

[tayloramurphy]

@clrcrl what is the effect of switching the join type? I see that the macro doesn't even exist anymore and is now just snapshot_staging_table (based on what I see in https://github.com/fishtown-analytics/dbt/blob/dev/marian-anderson/core/dbt/include/global_project/macros/materializations/snapshot/snapshot.sql#L26)

If we add the timestamp with deletes and override this internal macro, does it break anything else?

[clrcrl]

Here's a Loom on the topic

And the example data that I'm talking through

[clrcrl]

Closing as there's no intent to merge this at this time.