Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fast follows for Unit Testing #4975

Merged
merged 26 commits into from
Feb 27, 2024
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
f1d3506
Adding data types to unit testing
matthewshaver Feb 22, 2024
8ba002f
Apply suggestions from code review
matthewshaver Feb 22, 2024
79da9b4
Updating formatting
matthewshaver Feb 22, 2024
d4e48d1
Merge branch 'unit-testing-ff' of https://github.com/dbt-labs/docs.ge…
matthewshaver Feb 22, 2024
3a7a7d9
type to format
matthewshaver Feb 22, 2024
2568718
Separating pages
matthewshaver Feb 23, 2024
68abb18
Merge branch 'current' into unit-testing-ff
matthewshaver Feb 23, 2024
5657bf0
Fixing sidebar entries
matthewshaver Feb 23, 2024
c81ac4a
Merge branch 'unit-testing-ff' of https://github.com/dbt-labs/docs.ge…
matthewshaver Feb 23, 2024
d58d4ba
Updating pages with more info
matthewshaver Feb 26, 2024
386d3e5
Update website/docs/docs/build/unit-tests.md
matthewshaver Feb 26, 2024
3c93943
More FF changes
matthewshaver Feb 26, 2024
10f63a1
add overrides to sidebar
matthewshaver Feb 26, 2024
b50fc30
Missed closing block
matthewshaver Feb 26, 2024
ab84d9d
Merge branch 'current' into unit-testing-ff
matthewshaver Feb 26, 2024
24437d5
Update website/docs/reference/commands/build.md
matthewshaver Feb 26, 2024
921a27a
Update website/docs/reference/commands/build.md
matthewshaver Feb 26, 2024
5d52b19
Update website/docs/docs/build/unit-tests.md
matthewshaver Feb 26, 2024
0c8141b
Update website/docs/reference/commands/build.md
matthewshaver Feb 26, 2024
f2197db
Additional changes
matthewshaver Feb 26, 2024
0b6c2a0
Merge branch 'unit-testing-ff' of https://github.com/dbt-labs/docs.ge…
matthewshaver Feb 26, 2024
627ded7
Merge branch 'current' into unit-testing-ff
matthewshaver Feb 26, 2024
840f1bd
Apply suggestions from code review
matthewshaver Feb 27, 2024
9752645
Update unit-tests.md
matthewshaver Feb 27, 2024
43a7d2c
Apply suggestions from code review
runleonarun Feb 27, 2024
dee0678
Merge branch 'current' into unit-testing-ff
matthewshaver Feb 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion website/docs/docs/build/unit-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Now, we are introducing a new type of test to dbt - unit tests. In software prog
- We currently only support unit testing SQL models.
- We currently only support adding unit tests to models in your _current_ project.
- If your model has multiple versions, by default the unit test will run on *all* versions of your model. Read [unit testing versioned models](#unit-testing-versioned-models) for more information.
- Unit tests must be defined in a YML file in your `models/` directory.

Read the [reference doc](/reference/resource-properties/unit-tests) for more details about formatting your unit tests.

Expand Down Expand Up @@ -124,7 +125,7 @@ You only have to define the mock data for the columns you care about. This enabl

The direct parents of the model that you’re unit testing (in this example, `stg_customers` and `top_level_email_domains`) need to exist in the warehouse before you can execute the unit test.

Use the `--empty` flag to build an empty version of the models to save warehouse spend.
Use the [`--empty`](/reference/commands/build#the---empty-flag) flag to build an empty version of the models to save warehouse spend.

```bash

Expand Down Expand Up @@ -326,3 +327,12 @@ unit_tests:
```

There is currently no way to unit test whether the dbt framework inserted/merged the records into your existing model correctly, but [we're investigating support for this in the future](https://github.com/dbt-labs/dbt-core/issues/8664).

## Additional resources

- [Unit testing reference page](/reference/resource-properties/unit-tests)
- [Supported data formats for mock data](/reference/resource-properties/data-formats)
- [Unit testing versioned models](/reference/resource-properties/unit-testing-versions)
- [Unit test inputs](/reference/resource-properties/unit-test-input)
- [Unit test overrides](/reference/resource-properties/unit-test-overrides)
- [The `--empty` flag](/reference/commands/build#the---empty-flag)
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
17 changes: 15 additions & 2 deletions website/docs/reference/commands/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The `dbt build` command will:

In DAG order, for selected resources or an entire project.

### Details
## Details

**Artifacts:** The `build` task will write a single [manifest](/reference/artifacts/manifest-json) and a single [run results artifact](/reference/artifacts/run-results-json). The run results will include information about all models, tests, seeds, and snapshots that were selected to build, combined into one file.

Expand All @@ -26,15 +26,28 @@ In DAG order, for selected resources or an entire project.
**Flags:** The `build` task supports all the same flags as `run`, `test`, `snapshot`, and `seed`. For flags that are shared between multiple tasks (e.g. `--full-refresh`), `build` will use the same value for all selected resource types (e.g. both models and seeds will be full refreshed).

<VersionBlock firstVersion="1.8">

### The `--empty` flag

The `build` command supports the `--empty` flag for building schema-only dry runs. The `--empty` flag limits the refs and sources to zero rows. dbt will still execute the model SQL against the target data warehouse but will avoid expensive reads of input data. This validates dependencies and ensures your models will build properly.

## Unit tests
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

When `dbt build` is executed with unit tests, the models will be processed according to their lineage and dependencies. The tests will be executed as follows:
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

- [Unit tests](/docs/build/unit-tests) are run on a SQL model.
- The model is materialized.
- [Data tests](/docs/build/data-tests) are run on the model.

This saves on warehouse spend because the model won't be materialized unless the unit tests pass successfully.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

Unit tests and data tests can be selected using `--select test_type:unit` or `--select test_type:data` for `dbt build` (same for the `--exclude` flag).

</VersionBlock>

### Examples


```
$ dbt build
Running with dbt=0.21.0-b2
Expand Down
24 changes: 22 additions & 2 deletions website/docs/reference/node-selection/methods.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,15 +156,35 @@ dbt ls -s config.transient:true
### The "test_type" method
The `test_type` method is used to select tests based on their type, `singular` or `generic`:
<VersionBlock lastVersion="1.7">
The `test_type` method is used to select tests based on their type, `singular` or `generic`:
```bash
dbt test --select "test_type:generic" # run all generic tests
dbt test --select "test_type:singular" # run all singular tests
```
</VersionBlock>
<VersionBlock firstVersion="1.8">
The `test_type` method is used to select tests based on their type:
- [Unit tests](/docs/build/unit-tests)
- [Data tests](/docs/build/data-tests):
- [Singular](/docs/build/data-tests#singular-data-tests)
- [Generic](/docs/build/data-tests#generic-data-tests)
```bash
dbt test --select "test_type:unit" # run all unit tests
dbt test --select "test_type:data" # run all data tests
dbt test --select "test_type:generic" # run all generic data tests
dbt test --select "test_type:singular" # run all singular data tests
```
</VersionBlock>
### The "test_name" method
Expand Down
60 changes: 60 additions & 0 deletions website/docs/reference/resource-properties/data-formats.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
title: "Supported data formats for unit tests"
sidebar_label: "Data formats"
---

Currently, mock data for unit testing in dbt supports two formats:

- `dict` (default): Inline dictionary values.
- `csv`: Inline CSV values or a CSV file.

We will support more in the future, so watch our [upgrade guides](/docs/dbt-versions/core-upgrade) and this page for updates.

The `dict` data format is the default if no `format` is defined.

`dict` requires an inline dictionary for `rows`:

```yml

unit_tests:
- name: test_my_model
model: my_model
given:
- input: ref('my_model_a')
format: dict
rows:
- {id: 1, name: gerda}
- {id: 2, b: michelle}

```

When using the `csv` format, you can use either an inline CSV string for `rows`:

```yml

unit_tests:
- name: test_my_model
model: my_model
given:
- input: ref('my_model_a')
format: csv
rows: |
id,name
1,gerda
2,michelle
```
Or, you can provide the name of a CSV file in the `tests/fixtures` directory (or the configured `test-paths` location) of your project for `fixture`:

```yml
unit_tests:
- name: test_my_model
model: my_model
given:
- input: ref('my_model_a')
format: csv
fixture: my_model_a_fixture
```
34 changes: 34 additions & 0 deletions website/docs/reference/resource-properties/unit-test-input.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
title: "Input for unit tests"
sidebar_label: "Input"
---

Inputs in unit testing are used to reference a specific model or source for the test:
runleonarun marked this conversation as resolved.
Show resolved Hide resolved

- `input:` string that represents a `ref` or `source` call:
runleonarun marked this conversation as resolved.
Show resolved Hide resolved
- `ref('my_model')` or `ref('my_model', v='2')` or `ref('dougs_project', 'users')`
- `source('source_schema', 'source_name')`
- `input:` is optional for seeds:
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
- If you don’t supply an input for a seed, we will use the seed _as_ the input.
- If you do supply an input for a seed, we will use that input instead.
- You can also have “empty” inputs, by setting rows to an empty list `rows: []`
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

```yml
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

unit_tests:
- name: test_is_valid_email_address # this is the unique name of the test
model: dim_customers # name of the model I'm unit testing
given: # the mock data for your inputs
- input: ref('stg_customers')
rows:
- {email: cool@example.com, email_top_level_domain: example.com}
- {email: cool@unknown.com, email_top_level_domain: unknown.com}
- {email: badgmail.com, email_top_level_domain: gmail.com}
- {email: missingdot@gmailcom, email_top_level_domain: gmail.com}
- input: ref('top_level_email_domains')
rows:
- {tld: example.com}
- {tld: gmail.com}
...

```
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
title: "Unit test overrides"
sidebar_label: "Overrides"
---

When configuring your unit test, you can override the output of [macros](/docs/build/jinja-macros#macros), [project variables](/docs/build/project-variables), or [environment variables](/docs/build/environment-variables) for a given unit test.

```yml

- name: test_my_model_overrides
model: my_model
given:
- input: ref('my_model_a')
rows:
- {id: 1, a: 1}
- input: ref('my_model_b')
rows:
- {id: 1, b: 2}
- {id: 2, b: 2}
overrides:
macros:
type_numeric: override
invocation_id: 123
vars:
my_test: var_override
env_vars:
MY_TEST: env_var_override
expect:
rows:
- {macro_call: override, var_call: var_override, env_var_call: env_var_override, invocation_id: 123}

```
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
title: "Unit testing versioned SQL models"
sidebar_label: "Versions"
---

If your model has multiple versions, the default unit test will run on _all_ versions of your model. To specify version(s) of your model to unit test, use `include` or `exclude` for the desired versions in your model versions config:

```yaml

# my test_is_valid_email_address unit test will run on all versions of my_model
unit_tests:
- name: test_is_valid_email_address
model: my_model
...

# my test_is_valid_email_address unit test will run on ONLY version 2 of my_model
unit_tests:
- name: test_is_valid_email_address
model: my_model
versions:
include:
- 2
...

# my test_is_valid_email_address unit test will run on all versions EXCEPT 1 of my_model
unit_tests:
- name: test_is_valid_email_address
model: my_model
versions:
exclude:
- 1
...

```
Loading
Loading