Code-based mechanism for configuring models

[drewbanin]

dbt should provide a mechanism for configuring sets of models through code. This configuration/selection should happen in Python, as it's a much more capable and flexible mechanism for describing business logic than Jinja.

A potential interface for this might look like:

models.select(package='snowplow', materialized='incremental').onComplete([redshift.vacuum, redshift.analyze) }}

This snippet would be responsible for adding post-hooks to all incremental models in the Snowplow package.

Some more examples:

# Turn off all snowplow models
models.select(package="snowplow").config(enabled=False) }}

# Enable the `snowplow_page_views` model in the `snowplow` package
models.select(fqn="snowplow.snowplow_page_views").config(enabled=True) }}

# Configure all `view` models in the Snowplow package to be `ephemeral`
models.select(package="snowplow", materialized"view").config(materialized='ephemeral') }}

# Drops all models in the snowplow package where the set of `tags` includes `temporary`
models.select(package="snowplow", hasTags=["temporary"]).onRunEnd(dbt.drop_relation) }}

Altogether, this would be used in a script like:

#!/usr/bin/env python

import dbt

project = dbt.load_project()
models = project.models()

models_to_run = models.select(package="snowplow", children=True, parents=True)
results = models_to_run.run()

This is sort of a vignette of what model selection/configuration in code could look like. Alternative proposals are welcomed!

[drewbanin]

See #1842