Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/generate multiple models yaml #85

Merged
merged 12 commits into from
Nov 23, 2022
16 changes: 11 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,11 +126,11 @@ select * from renamed
4. Paste the output in to a model, and refactor as required.

## generate_model_yaml ([source](macros/generate_model_yaml.sql))
This macro generates the YAML for a model, which you can then paste into a
This macro generates the YAML for a list of model(s), which you can then paste into a
schema.yml file.

### Arguments:
* `model_name` (required): The model you wish to generate YAML for.
* `model_names` (required): The model(s) you wish to generate YAML for.
* `upstream_descriptions` (optional, default=False): Whether you want to include descriptions for identical column names from upstream models.

### Usage:
Expand All @@ -139,17 +139,23 @@ schema.yml file.

```
{{ codegen.generate_model_yaml(
model_name='customers'
model_names=['customers']
) }}
```

Alternatively, call the macro as an [operation](https://docs.getdbt.com/docs/using-operations):

```
$ dbt run-operation generate_model_yaml --args '{"model_name": "customers"}'
$ dbt run-operation generate_model_yaml --args '{"model_names": ["customers"]}'
```

3. The YAML for a base model will be logged to the command line
or

```
$ dbt run-operation generate_model_yaml --args '{"model_names": ["customers", "orders"]}'
angelica-lastra marked this conversation as resolved.
Show resolved Hide resolved
```

3. The YAML for a base model(s) will be logged to the command line

```
version: 2
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
) %}

{% set actual_source_yaml = codegen.generate_model_yaml(
model_name='model_struct'
model_names=['model_struct']
)
%}

Expand Down
2 changes: 1 addition & 1 deletion integration_tests/tests/test_generate_model_yaml.sql
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{% set actual_model_yaml = codegen.generate_model_yaml(
model_name='data__a_relation'
model_names=['data__a_relation']
)
%}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{% set actual_model_yaml = codegen.generate_model_yaml(
model_names=['data__a_relation','data__b_relation']
)
%}

{% set expected_model_yaml %}
version: 2

models:
- name: data__a_relation
description: ""
columns:
- name: col_a
description: ""

- name: col_b
description: ""

- name: data__b_relation
description: ""
columns:
- name: col_a
description: ""

- name: col_b
description: ""

{% endset %}

{{ assert_equal (actual_model_yaml | trim, expected_model_yaml | trim) }}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{% set actual_model_yaml = codegen.generate_model_yaml(
model_name='child_model',
model_names=['child_model'],
upstream_descriptions=True
)
%}
Expand Down
37 changes: 22 additions & 15 deletions macros/generate_model_yaml.sql
Original file line number Diff line number Diff line change
Expand Up @@ -17,24 +17,31 @@
{% do return(model_yaml) %}
{% endmacro %}

{% macro generate_model_yaml(model_name, upstream_descriptions=False) %}
{% macro generate_model_yaml(model_names=[], upstream_descriptions=False) %}

{% set model_yaml=[] %}
{% set column_desc_dict = codegen.build_dict_column_descriptions(model_name) if upstream_descriptions else {} %}
{% set model_yaml=[] %}

{% do model_yaml.append('version: 2') %}
{% do model_yaml.append('') %}
{% do model_yaml.append('models:') %}
{% do model_yaml.append(' - name: ' ~ model_name | lower) %}
{% do model_yaml.append(' description: ""') %}
{% do model_yaml.append(' columns:') %}

{% set relation=ref(model_name) %}
{%- set columns = adapter.get_columns_in_relation(relation) -%}
{% do model_yaml.append('version: 2') %}
{% do model_yaml.append('') %}
{% do model_yaml.append('models:') %}

{% for column in columns %}
{% set model_yaml = codegen.generate_column_yaml(column, model_yaml, column_desc_dict) %}
{% endfor %}
{% if model_names is string %}
{{ exceptions.raise_compiler_error("The name argument to ref() must be a list, got <class 'string'>: " ~ number) }}
angelica-lastra marked this conversation as resolved.
Show resolved Hide resolved
{% else %}
{% for model in model_names %}
{% do model_yaml.append(' - name: ' ~ model | lower) %}
{% do model_yaml.append(' description: ""') %}
{% do model_yaml.append(' columns:') %}

{% set relation=ref(model) %}
{%- set columns = adapter.get_columns_in_relation(relation) -%}
{% set column_desc_dict = codegen.build_dict_column_descriptions(model) if upstream_descriptions else {} %}

{% for column in columns %}
{% set model_yaml = codegen.generate_column_yaml(column, model_yaml, column_desc_dict) %}
angelica-lastra marked this conversation as resolved.
Show resolved Hide resolved
{% endfor %}
{% endfor %}
{% endif %}

{% if execute %}

Expand Down