Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for dynamic schema name generation for artifacts #153

Closed
ehmartens opened this issue Aug 3, 2022 · 0 comments · Fixed by #188
Closed

Add support for dynamic schema name generation for artifacts #153

ehmartens opened this issue Aug 3, 2022 · 0 comments · Fixed by #188
Assignees
Labels
enhancement New feature or request

Comments

@ehmartens
Copy link
Contributor

ehmartens commented Aug 3, 2022

This request was initially surfaced here in dbt slack.

Hi there. It is possible to use Jinja inside dbt_project to dynamically build the schema name for different environments when used in a package variable? I'm using dbt_artifacts and want the schema to be adt (stands for audit) in prod, but <target.user>_adt in dev.

The goal here is for the dbt_artifacts_schema (currently set in vars) to be dynamic, based on user name and environment.

@jaypeedevlin and I played around with using the generate_schema_name macro to do this. It works fine in create_dbt_artifacts_tables() to replace the logic in the set database_name (here) with logic that utilizes the generate_database_name() macro. The problem arises when we attempt to update the database in the sources.yml file (here). The macro isn't acceptable in this file, and therefore this solution isn't feasible.

Two proposed approaches for tackling this would be:

  1. Make the source tables in to dbt models so that we can utilize the generate_schema_name() macro
  2. Make some assumptions about how users will structure their projects and create vars that reflect those assumptions. (For instance, a prod and dev location for the artifacts.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants