-
Notifications
You must be signed in to change notification settings - Fork 314
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: use append vs merge option from backend config #3965
feat: use append vs merge option from backend config #3965
Conversation
2bce617
to
367bd51
Compare
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #3965 +/- ##
==========================================
- Coverage 71.64% 71.26% -0.38%
==========================================
Files 372 370 -2
Lines 54977 54083 -894
==========================================
- Hits 39389 38544 -845
+ Misses 13262 13231 -31
+ Partials 2326 2308 -18
☔ View full report in Codecov by Sentry. |
@lvrach @achettyiitr hey guys, please review also the flow diagram and the outstanding questions (see PR description). |
7f68b03
to
a8cba84
Compare
a8cba84
to
a1b912e
Compare
Description
Adding support for opt-in deduplication.
Linear Ticket
< Linear Link >
E2E Warehouse run
https://github.com/rudderlabs/e2e/actions/runs/6587296799
Updated Flow Diagram
Outstanding questions
shouldMerge
method inmssql
as well here like we do in postgres? if we add it, is it OK for existing customers that are usingmssql
?shouldMerge
there as well?TODOs
CanAppend
with a generic statement in the public docs saying that even though the user didn't opt-in withenableMerge
we might merge anyway due to other conditions e.g. if they connected an extract source, we replay the data, etc...loadTableStrategy
and replace that withenableMerge
Security