Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

non-destructive dbt run #137

Closed
jthandy opened this issue Sep 10, 2016 · 3 comments
Closed

non-destructive dbt run #137

jthandy opened this issue Sep 10, 2016 · 3 comments
Assignees

Comments

@jthandy
Copy link
Member

jthandy commented Sep 10, 2016

In production, we want runs to be as non-destructive as possible. If a given project has 4 incremental models and 50 views and no "table" models, that means all dbt production runs can avoid tearing down any database objects. There are two changes we need to make to actually get that to happen:

  1. We need to avoid rebuilding dependent views if dbt run specifies only incremental models. Currently, these views get rebuilt even though there is no need for this.
  2. We need to have a graph selection mechanism for model materialization so that we can select only incremental models (or only tables, etc.) from the graph.

This combination will result in a significantly cleaner production run; tearing down objects in production should be done only as needed.

@drewbanin drewbanin modified the milestone: 0.5.1 Release Sep 28, 2016
@drewbanin drewbanin self-assigned this Sep 28, 2016
@drewbanin drewbanin modified the milestone: 0.5.1 Release Oct 18, 2016
@drewbanin drewbanin changed the title allow dbt to run only incremental tables tables only dbt run Oct 18, 2016
@drewbanin
Copy link
Contributor

drewbanin commented Oct 18, 2016

leave views alone, truncate + re-insert into tables, run incremental models. Truncation is to avoid dropping dependent views

This is for production use-cases where we don't want to trash everything and rebuild from scratch

@drewbanin
Copy link
Contributor

see also: #140

@drewbanin drewbanin added this to the 0.5.2 Release milestone Oct 18, 2016
@drewbanin
Copy link
Contributor

dbt run --non-destructive

@drewbanin drewbanin changed the title tables only dbt run non-destructive dbt run Dec 13, 2016
yu-iskw pushed a commit to yu-iskw/dbt that referenced this issue Aug 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants