Feature/hackathon model generator (#83)

* feature/hackathon-model-generator

* documentation updates

* final bash script changes

* readme updates

* changelog entry

* Remove `columns_array` and `&&` and add some light commentary

Co-authored-by: Doug Beatty <doug.beatty@dbtlabs.com>
Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com>

CHANGELOG.md

@@ -10,6 +10,23 @@
 - [@handle](https://github.com/handle) (#XXX)
 --->
 
+# Unreleased
+## New features
+- Addition of the [create_base_models](macros/create_base_models.sql)
+This macro generates a series of terminal commands (appended w) bash script which creates a new file in your dbt project based off the results of the [generate_base_model](macros/generate_base_model.sql) macro. Therefore, instead of outputting in the terminal, it will create the file for you.
+
+## Quality of life
+- Addition of the [base_model_creation](bash_scripts/base_model_creation.sh) bash script which allows users to input multiple tables as a list and generate a terminal command that will combine **all** [create_base_models](macros/create_base_models.sql) commands. This way, you can generate base models for all your sources at once.
+
+## Contributors:
+- [@fivetran-joemarkiewicz](https://github.com/fivetran-joemarkiewicz) (#83)
+
+# dbt-codegen v0.9.0
+
+# dbt-codegen v0.8.1
+
+# dbt-codegen v0.8.0
+
 # dbt-codegen v0.7.0
 
 ## 🚨 Breaking change

README.md

@@ -13,12 +13,18 @@ Macros that generate dbt code, and log it to the command line.
   - [generate_base_model (source)](#generate_base_model-source)
     - [Arguments:](#arguments-1)
     - [Usage:](#usage-1)
-  - [generate_model_yaml (source)](#generate_model_yaml-source)
+  - [create_base_models (source)](#create_base_models-source)
     - [Arguments:](#arguments-2)
     - [Usage:](#usage-2)
-  - [generate_model_import_ctes (source)](#generate_model_import_ctes-source)
+  - [base_model_creation (source)](#base_model_creation-source)
     - [Arguments:](#arguments-3)
     - [Usage:](#usage-3)
+  - [generate_model_yaml (source)](#generate_model_yaml-source)
+    - [Arguments:](#arguments-4)
+    - [Usage:](#usage-4)
+  - [generate_model_import_ctes (source)](#generate_model_import_ctes-source)
+    - [Arguments:](#arguments-5)
+    - [Usage:](#usage-5)
 
 # Installation instructions
 New to dbt packages? Read more about them [here](https://docs.getdbt.com/docs/building-a-dbt-project/package-management/).
@@ -140,6 +146,37 @@ select * from renamed
 
 4. Paste the output in to a model, and refactor as required.
 
+## create_base_models ([source](macros/create_base_models.sql))
+This macro generates a series of terminal commands (appended with the `&&` to allow for subsequent execution) that execute the [base_model_creation](#base_model_creation-source) bash script. This bash script will write the output of the [generate_base_model](#generate_base_model-source) macro into a new model file in your local dbt project.
+>**Note**: This macro is not compatible with the dbt Cloud IDE.
+
+### Arguments:
+* `source_name` (required): The source you wish to generate base model SQL for.
+* `tables` (required): A list of all tables you want to generate the base models for.
+
+### Usage:
+1. Create a source for the table you wish to create a base model on top of.
+2. Copy the macro into a statement tab into your local IDE, and run your code
+
+```sql
+dbt run-operation codegen.create_base_models --args '{source_name: my-source, tables: ["this-table","that-table"]}'
+```
+## base_model_creation ([source](bash_scripts/base_model_creation.sh))
+This bash script when executed from your local IDE will create model files in your dbt project instance that contain the outputs of the [generate_base_model](macros/generate_base_model.sql) macro.
+>**Note**: This macro is not compatible with the dbt Cloud IDE.
+
+### Arguments:
+* `source_name` (required): The source you wish to generate base model SQL for.
+* `tables` (required): A list of all tables you want to generate the base models for.
+
+### Usage:
+1. Create a source for the table you wish to create a base model on top of.
+2. Copy the macro into a statement tab into your local IDE, and run your code
+
+```bash
+source dbt_packages/codegen/bash_scripts/base_model_creation.sh "source_name" ["this-table","that-table"]
+```
+
 ## generate_model_yaml ([source](macros/generate_model_yaml.sql))
 This macro generates the YAML for a list of model(s), which you can then paste into a
 schema.yml file.

bash_scripts/base_model_creation.sh

@@ -0,0 +1,5 @@
+#!/bin/bash
+
+echo "" > models/stg_$1__$2.sql
+
+dbt run-operation codegen.generate_base_model --args '{"source_name": "'$1'", "table_name": "'$2'"}' | tail -n +3 >> models/stg_$1__$2.sql

macros/create_base_models.sql

@@ -0,0 +1,16 @@
+{% macro create_base_models(source_name, tables) %}
+
+{% set source_name = ""~ source_name ~"" %}
+
+{% set zsh_command_models = "source dbt_packages/codegen/bash_scripts/base_model_creation.sh """~ source_name ~""" " %}
+
+{%- set models_array = [] -%}
+
+{% for t in tables %}
+    {% set help_command = zsh_command_models + t %}
+    {{ models_array.append(help_command) }}
+{% endfor %}
+
+{{ log("Run these commands in your shell to generate the models:\n" ~ models_array|join(' && \n'), info=True) }}
+
+{% endmacro %}