-
Notifications
You must be signed in to change notification settings - Fork 951
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fast follows for Unit Testing (#4975)
## What are you changing in this pull request and why? Fast follows for unit tests: - Breaks the reference page into sections in a folder. - Adds resource links to the /docs page on unit tests. - The Methods page is updated to reflect unit tests. - Links to information about the --empty flag. - Various other edits and clean ups. ## Checklist - [x] Review the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md) so my content adheres to these guidelines. - [x] For [docs versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#about-versioning), review how to [version a whole page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version) and [version a block of content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content). - [x] Add a checklist item for anything that needs to happen before this PR is merged, such as "needs technical review" or "change base branch."
- Loading branch information
Showing
9 changed files
with
254 additions
and
163 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
60 changes: 60 additions & 0 deletions
60
website/docs/reference/resource-properties/data-formats.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
--- | ||
title: "Supported data formats for unit tests" | ||
sidebar_label: "Data formats" | ||
--- | ||
|
||
Currently, mock data for unit testing in dbt supports two formats: | ||
|
||
- `dict` (default): Inline dictionary values. | ||
- `csv`: Inline CSV values or a CSV file. | ||
|
||
We will support more in the future, so watch our [upgrade guides](/docs/dbt-versions/core-upgrade) and this page for updates. | ||
|
||
The `dict` data format is the default if no `format` is defined. | ||
|
||
`dict` requires an inline dictionary for `rows`: | ||
|
||
```yml | ||
|
||
unit_tests: | ||
- name: test_my_model | ||
model: my_model | ||
given: | ||
- input: ref('my_model_a') | ||
format: dict | ||
rows: | ||
- {id: 1, name: gerda} | ||
- {id: 2, b: michelle} | ||
|
||
``` | ||
|
||
When using the `csv` format, you can use either an inline CSV string for `rows`: | ||
|
||
```yml | ||
|
||
unit_tests: | ||
- name: test_my_model | ||
model: my_model | ||
given: | ||
- input: ref('my_model_a') | ||
format: csv | ||
rows: | | ||
id,name | ||
1,gerda | ||
2,michelle | ||
``` | ||
Or, you can provide the name of a CSV file in the `tests/fixtures` directory (or the configured `test-paths` location) of your project for `fixture`: | ||
|
||
```yml | ||
unit_tests: | ||
- name: test_my_model | ||
model: my_model | ||
given: | ||
- input: ref('my_model_a') | ||
format: csv | ||
fixture: my_model_a_fixture | ||
``` |
34 changes: 34 additions & 0 deletions
34
website/docs/reference/resource-properties/unit-test-input.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
--- | ||
title: "Input for unit tests" | ||
sidebar_label: "Input" | ||
--- | ||
|
||
Use inputs in your unit tests to reference a specific model or source for the test: | ||
|
||
- For `input:`, use a string that represents a `ref` or `source` call: | ||
- `ref('my_model')` or `ref('my_model', v='2')` or `ref('dougs_project', 'users')` | ||
- `source('source_schema', 'source_name')` | ||
- Optionally use for seeds: | ||
- If you don’t supply an input for a seed, we will use the seed _as_ the input. | ||
- If you do supply an input for a seed, we will use that input instead. | ||
- Use “empty” inputs by setting rows to an empty list `rows: []` | ||
|
||
```yml | ||
|
||
unit_tests: | ||
- name: test_is_valid_email_address # this is the unique name of the test | ||
model: dim_customers # name of the model I'm unit testing | ||
given: # the mock data for your inputs | ||
- input: ref('stg_customers') | ||
rows: | ||
- {email: cool@example.com, email_top_level_domain: example.com} | ||
- {email: cool@unknown.com, email_top_level_domain: unknown.com} | ||
- {email: badgmail.com, email_top_level_domain: gmail.com} | ||
- {email: missingdot@gmailcom, email_top_level_domain: gmail.com} | ||
- input: ref('top_level_email_domains') | ||
rows: | ||
- {tld: example.com} | ||
- {tld: gmail.com} | ||
... | ||
|
||
``` |
67 changes: 67 additions & 0 deletions
67
website/docs/reference/resource-properties/unit-test-overrides.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
--- | ||
title: "Unit test overrides" | ||
sidebar_label: "Overrides" | ||
--- | ||
|
||
When configuring your unit test, you can override the output of [macros](/docs/build/jinja-macros#macros), [project variables](/docs/build/project-variables), or [environment variables](/docs/build/environment-variables) for a given unit test. | ||
|
||
```yml | ||
|
||
- name: test_my_model_overrides | ||
model: my_model | ||
given: | ||
- input: ref('my_model_a') | ||
rows: | ||
- {id: 1, a: 1} | ||
- input: ref('my_model_b') | ||
rows: | ||
- {id: 1, b: 2} | ||
- {id: 2, b: 2} | ||
overrides: | ||
macros: | ||
type_numeric: override | ||
invocation_id: 123 | ||
vars: | ||
my_test: var_override | ||
env_vars: | ||
MY_TEST: env_var_override | ||
expect: | ||
rows: | ||
- {macro_call: override, var_call: var_override, env_var_call: env_var_override, invocation_id: 123} | ||
|
||
``` | ||
|
||
## Macros | ||
|
||
You can override the output of any macro in your unit test defition. | ||
|
||
If the model you're unit testing uses these macros, you must override them: | ||
- [`is_incremental`](/docs/build/incremental-models#understanding-the-is_incremental-macro): If you're unit testing an incremental model, you must explicity set `is_incremental` to `true` or `false`. See more docs on unit testing incremental models [here](/docs/build/unit-tests#unit-testing-incremental-models). | ||
|
||
```yml | ||
|
||
unit_tests: | ||
- name: my_unit_test | ||
model: my_incremental_model | ||
overrides: | ||
macros: | ||
# unit test this model in "full refresh" mode | ||
is_incremental: false | ||
... | ||
|
||
``` | ||
|
||
- [`dbt_utils.star`](/blog/star-sql-love-letter): If you're unit testing a model that uses the `star` macro, you must explicity set `star` to a list of columns. This is because the `star` only accepts a [relation](/reference/dbt-classes#relation) for the `from` argument; the unit test mock input data is injected directly into the model SQL, replacing the `ref('')` or `source('')` function, causing the `star` macro to fail unless overidden. | ||
|
||
```yml | ||
unit_tests: | ||
- name: my_other_unit_test | ||
model: my_model_that_uses_star | ||
overrides: | ||
macros: | ||
# explicity set star to relevant list of columns | ||
star: col_a,col_b,col_c | ||
... | ||
``` |
34 changes: 34 additions & 0 deletions
34
website/docs/reference/resource-properties/unit-testing-versions.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
--- | ||
title: "Unit testing versioned SQL models" | ||
sidebar_label: "Versions" | ||
--- | ||
|
||
If your model has multiple versions, the default unit test will run on _all_ versions of your model. To specify version(s) of your model to unit test, use `include` or `exclude` for the desired versions in your model versions config: | ||
|
||
```yaml | ||
|
||
# my test_is_valid_email_address unit test will run on all versions of my_model | ||
unit_tests: | ||
- name: test_is_valid_email_address | ||
model: my_model | ||
... | ||
|
||
# my test_is_valid_email_address unit test will run on ONLY version 2 of my_model | ||
unit_tests: | ||
- name: test_is_valid_email_address | ||
model: my_model | ||
versions: | ||
include: | ||
- 2 | ||
... | ||
|
||
# my test_is_valid_email_address unit test will run on all versions EXCEPT 1 of my_model | ||
unit_tests: | ||
- name: test_is_valid_email_address | ||
model: my_model | ||
versions: | ||
exclude: | ||
- 1 | ||
... | ||
|
||
``` |
Oops, something went wrong.