Skip to content

Commit

Permalink
populate_adapter_cache for everyone
Browse files Browse the repository at this point in the history
Added the populate_adapter_cache to the GraphRunnableTask before_run method
 - Previously only Run used it
  • Loading branch information
Jacob Beck committed Apr 14, 2020
1 parent 6424b65 commit ac427bd
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 6 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
- Users can now use jinja as arguments to tests. Test arguments are rendered in the native context and injected into the test execution context directly. ([#2149](https://github.com/fishtown-analytics/dbt/issues/2149), [#2220](https://github.com/fishtown-analytics/dbt/pull/2220))
- Added support for `db_groups` and `autocreate` flags in Redshift configurations. ([#1995](https://github.com/fishtown-analytics/dbt/issues/1995), [#2262](https://github.com/fishtown-analytics/dbt/pull/2262))
- Users can supply paths as arguments to `--models` and `--select`, either explicitily by prefixing with `path:` or implicitly with no prefix. ([#454](https://github.com/fishtown-analytics/dbt/issues/454), [#2258](https://github.com/fishtown-analytics/dbt/pull/2258))
- dbt now builds the relation cache for "dbt compile" and "dbt ls" as well as "dbt run" ([#1705](https://github.com/fishtown-analytics/dbt/issues/1705), [#2319](https://github.com/fishtown-analytics/dbt/pull/2319))

### Fixes
- When a jinja value is undefined, give a helpful error instead of failing with cryptic "cannot pickle ParserMacroCapture" errors ([#2110](https://github.com/fishtown-analytics/dbt/issues/2110), [#2184](https://github.com/fishtown-analytics/dbt/pull/2184))
Expand Down
6 changes: 1 addition & 5 deletions core/dbt/task/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,9 +88,6 @@ def index_offset(self, value: int) -> int:
def raise_on_first_error(self):
return False

def populate_adapter_cache(self, adapter):
adapter.set_relations_cache(self.manifest)

def get_hook_sql(self, adapter, hook, idx, num_hooks, extra_context):
compiled = compile_node(adapter, self.config, hook, self.manifest,
extra_context)
Expand Down Expand Up @@ -195,9 +192,8 @@ def print_results_line(self, results, execution_time):
.format(stat_line=stat_line, execution=execution))

def before_run(self, adapter, selected_uids):
super().before_run(adapter, selected_uids)
with adapter.connection_named('master'):
self.create_schemas(adapter, selected_uids)
self.populate_adapter_cache(adapter)
self.safe_run_hooks(adapter, RunHookType.Start, {})

def after_run(self, adapter, results):
Expand Down
7 changes: 6 additions & 1 deletion core/dbt/task/runnable.py
Original file line number Diff line number Diff line change
Expand Up @@ -340,11 +340,16 @@ def _mark_dependent_errors(self, node_id, result, cause):
for dep_node_id in self.linker.get_dependent_nodes(node_id):
self._skipped_children[dep_node_id] = cause

def populate_adapter_cache(self, adapter):
adapter.set_relations_cache(self.manifest)

def before_hooks(self, adapter):
pass

def before_run(self, adapter, selected_uids):
pass
with adapter.connection_named('master'):
self.create_schemas(adapter, selected_uids)
self.populate_adapter_cache(adapter)

def after_run(self, adapter, results):
pass
Expand Down

0 comments on commit ac427bd

Please sign in to comment.