diff --git a/CHANGELOG.md b/CHANGELOG.md index 124ea15..c861e46 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,23 @@ - [@handle](https://github.com/handle) (#XXX) ---> +# Unreleased +## New features +- Addition of the [create_base_models](macros/create_base_models.sql) +This macro generates a series of terminal commands (appended w) bash script which creates a new file in your dbt project based off the results of the [generate_base_model](macros/generate_base_model.sql) macro. Therefore, instead of outputting in the terminal, it will create the file for you. + +## Quality of life +- Addition of the [base_model_creation](bash_scripts/base_model_creation.sh) bash script which allows users to input multiple tables as a list and generate a terminal command that will combine **all** [create_base_models](macros/create_base_models.sql) commands. This way, you can generate base models for all your sources at once. + +## Contributors: +- [@fivetran-joemarkiewicz](https://github.com/fivetran-joemarkiewicz) (#83) + +# dbt-codegen v0.9.0 + +# dbt-codegen v0.8.1 + +# dbt-codegen v0.8.0 + # dbt-codegen v0.7.0 ## 🚨 Breaking change diff --git a/README.md b/README.md index 4d8e1af..0a78f52 100644 --- a/README.md +++ b/README.md @@ -13,12 +13,18 @@ Macros that generate dbt code, and log it to the command line. - [generate_base_model (source)](#generate_base_model-source) - [Arguments:](#arguments-1) - [Usage:](#usage-1) - - [generate_model_yaml (source)](#generate_model_yaml-source) + - [create_base_models (source)](#create_base_models-source) - [Arguments:](#arguments-2) - [Usage:](#usage-2) - - [generate_model_import_ctes (source)](#generate_model_import_ctes-source) + - [base_model_creation (source)](#base_model_creation-source) - [Arguments:](#arguments-3) - [Usage:](#usage-3) + - [generate_model_yaml (source)](#generate_model_yaml-source) + - [Arguments:](#arguments-4) + - [Usage:](#usage-4) + - [generate_model_import_ctes (source)](#generate_model_import_ctes-source) + - [Arguments:](#arguments-5) + - [Usage:](#usage-5) # Installation instructions New to dbt packages? Read more about them [here](https://docs.getdbt.com/docs/building-a-dbt-project/package-management/). @@ -140,6 +146,37 @@ select * from renamed 4. Paste the output in to a model, and refactor as required. +## create_base_models ([source](macros/create_base_models.sql)) +This macro generates a series of terminal commands (appended with the `&&` to allow for subsequent execution) that execute the [base_model_creation](#base_model_creation-source) bash script. This bash script will write the output of the [generate_base_model](#generate_base_model-source) macro into a new model file in your local dbt project. +>**Note**: This macro is not compatible with the dbt Cloud IDE. + +### Arguments: +* `source_name` (required): The source you wish to generate base model SQL for. +* `tables` (required): A list of all tables you want to generate the base models for. + +### Usage: +1. Create a source for the table you wish to create a base model on top of. +2. Copy the macro into a statement tab into your local IDE, and run your code + +```sql +dbt run-operation codegen.create_base_models --args '{source_name: my-source, tables: ["this-table","that-table"]}' +``` +## base_model_creation ([source](bash_scripts/base_model_creation.sh)) +This bash script when executed from your local IDE will create model files in your dbt project instance that contain the outputs of the [generate_base_model](macros/generate_base_model.sql) macro. +>**Note**: This macro is not compatible with the dbt Cloud IDE. + +### Arguments: +* `source_name` (required): The source you wish to generate base model SQL for. +* `tables` (required): A list of all tables you want to generate the base models for. + +### Usage: +1. Create a source for the table you wish to create a base model on top of. +2. Copy the macro into a statement tab into your local IDE, and run your code + +```bash +source dbt_packages/codegen/bash_scripts/base_model_creation.sh "source_name" ["this-table","that-table"] +``` + ## generate_model_yaml ([source](macros/generate_model_yaml.sql)) This macro generates the YAML for a list of model(s), which you can then paste into a schema.yml file. diff --git a/bash_scripts/base_model_creation.sh b/bash_scripts/base_model_creation.sh new file mode 100644 index 0000000..65eb9c0 --- /dev/null +++ b/bash_scripts/base_model_creation.sh @@ -0,0 +1,5 @@ +#!/bin/bash + +echo "" > models/stg_$1__$2.sql + +dbt run-operation codegen.generate_base_model --args '{"source_name": "'$1'", "table_name": "'$2'"}' | tail -n +3 >> models/stg_$1__$2.sql \ No newline at end of file diff --git a/macros/create_base_models.sql b/macros/create_base_models.sql new file mode 100644 index 0000000..d5b448c --- /dev/null +++ b/macros/create_base_models.sql @@ -0,0 +1,16 @@ +{% macro create_base_models(source_name, tables) %} + +{% set source_name = ""~ source_name ~"" %} + +{% set zsh_command_models = "source dbt_packages/codegen/bash_scripts/base_model_creation.sh """~ source_name ~""" " %} + +{%- set models_array = [] -%} + +{% for t in tables %} + {% set help_command = zsh_command_models + t %} + {{ models_array.append(help_command) }} +{% endfor %} + +{{ log("Run these commands in your shell to generate the models:\n" ~ models_array|join(' && \n'), info=True) }} + +{% endmacro %}