Skip to content

Commit

Permalink
Fast follows for Unit Testing (#4975)
Browse files Browse the repository at this point in the history
## What are you changing in this pull request and why?

Fast follows for unit tests:
- Breaks the reference page into sections in a folder.
- Adds resource links to the /docs page on unit tests.
- The Methods page is updated to reflect unit tests.
- Links to information about the --empty flag.
- Various other edits and clean ups. 

## Checklist

- [x] Review the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
so my content adheres to these guidelines.
- [x] For [docs
versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#about-versioning),
review how to [version a whole
page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version)
and [version a block of
content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content).
- [x] Add a checklist item for anything that needs to happen before this
PR is merged, such as "needs technical review" or "change base branch."
  • Loading branch information
matthewshaver authored Feb 27, 2024
2 parents 8e2e363 + dee0678 commit 80ec1e3
Show file tree
Hide file tree
Showing 9 changed files with 254 additions and 163 deletions.
11 changes: 10 additions & 1 deletion website/docs/docs/build/unit-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Now, we are introducing a new type of test to dbt - unit tests. In software prog
- We currently only support unit testing SQL models.
- We currently only support adding unit tests to models in your _current_ project.
- If your model has multiple versions, by default the unit test will run on *all* versions of your model. Read [unit testing versioned models](#unit-testing-versioned-models) for more information.
- Unit tests must be defined in a YML file in your `models/` directory.

Read the [reference doc](/reference/resource-properties/unit-tests) for more details about formatting your unit tests.

Expand Down Expand Up @@ -124,7 +125,7 @@ You only have to define the mock data for the columns you care about. This enabl

The direct parents of the model that you’re unit testing (in this example, `stg_customers` and `top_level_email_domains`) need to exist in the warehouse before you can execute the unit test.

Use the `--empty` flag to build an empty version of the models to save warehouse spend.
Use the [`--empty`](/reference/commands/build#the---empty-flag) flag to build an empty version of the models to save warehouse spend.

```bash

Expand Down Expand Up @@ -326,3 +327,11 @@ unit_tests:
```

There is currently no way to unit test whether the dbt framework inserted/merged the records into your existing model correctly, but [we're investigating support for this in the future](https://github.com/dbt-labs/dbt-core/issues/8664).

## Additional resources

- [Unit testing reference page](/reference/resource-properties/unit-tests)
- [Supported data formats for mock data](/reference/resource-properties/data-formats)
- [Unit testing versioned models](/reference/resource-properties/unit-testing-versions)
- [Unit test inputs](/reference/resource-properties/unit-test-input)
- [Unit test overrides](/reference/resource-properties/unit-test-overrides)
17 changes: 15 additions & 2 deletions website/docs/reference/commands/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The `dbt build` command will:

In DAG order, for selected resources or an entire project.

### Details
## Details

**Artifacts:** The `build` task will write a single [manifest](/reference/artifacts/manifest-json) and a single [run results artifact](/reference/artifacts/run-results-json). The run results will include information about all models, tests, seeds, and snapshots that were selected to build, combined into one file.

Expand All @@ -26,15 +26,28 @@ In DAG order, for selected resources or an entire project.
**Flags:** The `build` task supports all the same flags as `run`, `test`, `snapshot`, and `seed`. For flags that are shared between multiple tasks (e.g. `--full-refresh`), `build` will use the same value for all selected resource types (e.g. both models and seeds will be full refreshed).

<VersionBlock firstVersion="1.8">

### The `--empty` flag

The `build` command supports the `--empty` flag for building schema-only dry runs. The `--empty` flag limits the refs and sources to zero rows. dbt will still execute the model SQL against the target data warehouse but will avoid expensive reads of input data. This validates dependencies and ensures your models will build properly.

## Tests

When `dbt build` is executed with unit tests applied, the models will be processed according to their lineage and dependencies. The tests will be executed as follows:

- [Unit tests](/docs/build/unit-tests) are run on a SQL model.
- The model is materialized.
- [Data tests](/docs/build/data-tests) are run on the model.

This saves on warehouse spend as the model will only be materialized if the unit tests pass successfully.

Unit tests and data tests can be selected using `--select test_type:unit` or `--select test_type:data` for `dbt build` (same for the `--exclude` flag).

</VersionBlock>

### Examples


```
$ dbt build
Running with dbt=0.21.0-b2
Expand Down
24 changes: 22 additions & 2 deletions website/docs/reference/node-selection/methods.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,15 +156,35 @@ dbt ls -s config.transient:true
### The "test_type" method
The `test_type` method is used to select tests based on their type, `singular` or `generic`:
<VersionBlock lastVersion="1.7">
The `test_type` method is used to select tests based on their type, `singular` or `generic`:
```bash
dbt test --select "test_type:generic" # run all generic tests
dbt test --select "test_type:singular" # run all singular tests
```
</VersionBlock>
<VersionBlock firstVersion="1.8">
The `test_type` method is used to select tests based on their type:
- [Unit tests](/docs/build/unit-tests)
- [Data tests](/docs/build/data-tests):
- [Singular](/docs/build/data-tests#singular-data-tests)
- [Generic](/docs/build/data-tests#generic-data-tests)
```bash
dbt test --select "test_type:unit" # run all unit tests
dbt test --select "test_type:data" # run all data tests
dbt test --select "test_type:generic" # run all generic data tests
dbt test --select "test_type:singular" # run all singular data tests
```
</VersionBlock>
### The "test_name" method
Expand Down
60 changes: 60 additions & 0 deletions website/docs/reference/resource-properties/data-formats.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
title: "Supported data formats for unit tests"
sidebar_label: "Data formats"
---

Currently, mock data for unit testing in dbt supports two formats:

- `dict` (default): Inline dictionary values.
- `csv`: Inline CSV values or a CSV file.

We will support more in the future, so watch our [upgrade guides](/docs/dbt-versions/core-upgrade) and this page for updates.

The `dict` data format is the default if no `format` is defined.

`dict` requires an inline dictionary for `rows`:

```yml

unit_tests:
- name: test_my_model
model: my_model
given:
- input: ref('my_model_a')
format: dict
rows:
- {id: 1, name: gerda}
- {id: 2, b: michelle}

```

When using the `csv` format, you can use either an inline CSV string for `rows`:

```yml

unit_tests:
- name: test_my_model
model: my_model
given:
- input: ref('my_model_a')
format: csv
rows: |
id,name
1,gerda
2,michelle
```
Or, you can provide the name of a CSV file in the `tests/fixtures` directory (or the configured `test-paths` location) of your project for `fixture`:

```yml
unit_tests:
- name: test_my_model
model: my_model
given:
- input: ref('my_model_a')
format: csv
fixture: my_model_a_fixture
```
34 changes: 34 additions & 0 deletions website/docs/reference/resource-properties/unit-test-input.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
title: "Input for unit tests"
sidebar_label: "Input"
---

Use inputs in your unit tests to reference a specific model or source for the test:

- For `input:`, use a string that represents a `ref` or `source` call:
- `ref('my_model')` or `ref('my_model', v='2')` or `ref('dougs_project', 'users')`
- `source('source_schema', 'source_name')`
- Optionally use for seeds:
- If you don’t supply an input for a seed, we will use the seed _as_ the input.
- If you do supply an input for a seed, we will use that input instead.
- Use “empty” inputs by setting rows to an empty list `rows: []`

```yml

unit_tests:
- name: test_is_valid_email_address # this is the unique name of the test
model: dim_customers # name of the model I'm unit testing
given: # the mock data for your inputs
- input: ref('stg_customers')
rows:
- {email: cool@example.com, email_top_level_domain: example.com}
- {email: cool@unknown.com, email_top_level_domain: unknown.com}
- {email: badgmail.com, email_top_level_domain: gmail.com}
- {email: missingdot@gmailcom, email_top_level_domain: gmail.com}
- input: ref('top_level_email_domains')
rows:
- {tld: example.com}
- {tld: gmail.com}
...

```
67 changes: 67 additions & 0 deletions website/docs/reference/resource-properties/unit-test-overrides.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
title: "Unit test overrides"
sidebar_label: "Overrides"
---

When configuring your unit test, you can override the output of [macros](/docs/build/jinja-macros#macros), [project variables](/docs/build/project-variables), or [environment variables](/docs/build/environment-variables) for a given unit test.

```yml

- name: test_my_model_overrides
model: my_model
given:
- input: ref('my_model_a')
rows:
- {id: 1, a: 1}
- input: ref('my_model_b')
rows:
- {id: 1, b: 2}
- {id: 2, b: 2}
overrides:
macros:
type_numeric: override
invocation_id: 123
vars:
my_test: var_override
env_vars:
MY_TEST: env_var_override
expect:
rows:
- {macro_call: override, var_call: var_override, env_var_call: env_var_override, invocation_id: 123}

```

## Macros

You can override the output of any macro in your unit test defition.

If the model you're unit testing uses these macros, you must override them:
- [`is_incremental`](/docs/build/incremental-models#understanding-the-is_incremental-macro): If you're unit testing an incremental model, you must explicity set `is_incremental` to `true` or `false`. See more docs on unit testing incremental models [here](/docs/build/unit-tests#unit-testing-incremental-models).

```yml

unit_tests:
- name: my_unit_test
model: my_incremental_model
overrides:
macros:
# unit test this model in "full refresh" mode
is_incremental: false
...

```

- [`dbt_utils.star`](/blog/star-sql-love-letter): If you're unit testing a model that uses the `star` macro, you must explicity set `star` to a list of columns. This is because the `star` only accepts a [relation](/reference/dbt-classes#relation) for the `from` argument; the unit test mock input data is injected directly into the model SQL, replacing the `ref('')` or `source('')` function, causing the `star` macro to fail unless overidden.

```yml
unit_tests:
- name: my_other_unit_test
model: my_model_that_uses_star
overrides:
macros:
# explicity set star to relevant list of columns
star: col_a,col_b,col_c
...
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
title: "Unit testing versioned SQL models"
sidebar_label: "Versions"
---

If your model has multiple versions, the default unit test will run on _all_ versions of your model. To specify version(s) of your model to unit test, use `include` or `exclude` for the desired versions in your model versions config:

```yaml

# my test_is_valid_email_address unit test will run on all versions of my_model
unit_tests:
- name: test_is_valid_email_address
model: my_model
...

# my test_is_valid_email_address unit test will run on ONLY version 2 of my_model
unit_tests:
- name: test_is_valid_email_address
model: my_model
versions:
include:
- 2
...

# my test_is_valid_email_address unit test will run on all versions EXCEPT 1 of my_model
unit_tests:
- name: test_is_valid_email_address
model: my_model
versions:
exclude:
- 1
...

```
Loading

0 comments on commit 80ec1e3

Please sign in to comment.