scd2
validity column values based on column in source data
#1776
Labels
enhancement
New feature or request
scd2
validity column values based on column in source data
#1776
Feature description
There are currently two options for validity column (i.e. "valid from" and "valid to") values:
boundary_timestamp
argumentI would like to see a third option:
In some cases an
updated_at
-style column in the source data "naturally" provides a good value for the validity columns.It then makes sense to also detect changes based on this column, i.e. to base the "row hash" on a composite of the natural key and
updated_at
(i.e. the surrogate key).Are you a dlt user?
Yes, I'm already a dlt user.
Use case
User asked for it: #1601 (comment)
Proposed solution
Extend public API with
natural_key
andboundary_timestamp_column
arguments:Related issues
#1601 (comment) (specifically option 1)
The text was updated successfully, but these errors were encountered: