diff --git a/website/docs/docs/build/unit-tests.md b/website/docs/docs/build/unit-tests.md index 9828788dd67..d4124120d22 100644 --- a/website/docs/docs/build/unit-tests.md +++ b/website/docs/docs/build/unit-tests.md @@ -24,6 +24,7 @@ Now, we are introducing a new type of test to dbt - unit tests. In software prog - We currently only support unit testing SQL models. - We currently only support adding unit tests to models in your _current_ project. - If your model has multiple versions, by default the unit test will run on *all* versions of your model. Read [unit testing versioned models](#unit-testing-versioned-models) for more information. +- Unit tests must be defined in a YML file in your `models/` directory. Read the [reference doc](/reference/resource-properties/unit-tests) for more details about formatting your unit tests. @@ -124,7 +125,7 @@ You only have to define the mock data for the columns you care about. This enabl The direct parents of the model that you’re unit testing (in this example, `stg_customers` and `top_level_email_domains`) need to exist in the warehouse before you can execute the unit test. -Use the `--empty` flag to build an empty version of the models to save warehouse spend. +Use the [`--empty`](/reference/commands/build#the---empty-flag) flag to build an empty version of the models to save warehouse spend. ```bash @@ -326,3 +327,11 @@ unit_tests: ``` There is currently no way to unit test whether the dbt framework inserted/merged the records into your existing model correctly, but [we're investigating support for this in the future](https://github.com/dbt-labs/dbt-core/issues/8664). + +## Additional resources + +- [Unit testing reference page](/reference/resource-properties/unit-tests) +- [Supported data formats for mock data](/reference/resource-properties/data-formats) +- [Unit testing versioned models](/reference/resource-properties/unit-testing-versions) +- [Unit test inputs](/reference/resource-properties/unit-test-input) +- [Unit test overrides](/reference/resource-properties/unit-test-overrides) \ No newline at end of file diff --git a/website/docs/reference/commands/build.md b/website/docs/reference/commands/build.md index c83f669fe26..a8ccdd8fc5d 100644 --- a/website/docs/reference/commands/build.md +++ b/website/docs/reference/commands/build.md @@ -12,7 +12,7 @@ The `dbt build` command will: In DAG order, for selected resources or an entire project. -### Details +## Details **Artifacts:** The `build` task will write a single [manifest](/reference/artifacts/manifest-json) and a single [run results artifact](/reference/artifacts/run-results-json). The run results will include information about all models, tests, seeds, and snapshots that were selected to build, combined into one file. @@ -26,15 +26,28 @@ In DAG order, for selected resources or an entire project. **Flags:** The `build` task supports all the same flags as `run`, `test`, `snapshot`, and `seed`. For flags that are shared between multiple tasks (e.g. `--full-refresh`), `build` will use the same value for all selected resource types (e.g. both models and seeds will be full refreshed). - + ### The `--empty` flag The `build` command supports the `--empty` flag for building schema-only dry runs. The `--empty` flag limits the refs and sources to zero rows. dbt will still execute the model SQL against the target data warehouse but will avoid expensive reads of input data. This validates dependencies and ensures your models will build properly. +## Tests + +When `dbt build` is executed with unit tests applied, the models will be processed according to their lineage and dependencies. The tests will be executed as follows: + +- [Unit tests](/docs/build/unit-tests) are run on a SQL model. +- The model is materialized. +- [Data tests](/docs/build/data-tests) are run on the model. + +This saves on warehouse spend as the model will only be materialized if the unit tests pass successfully. + +Unit tests and data tests can be selected using `--select test_type:unit` or `--select test_type:data` for `dbt build` (same for the `--exclude` flag). + ### Examples + ``` $ dbt build Running with dbt=0.21.0-b2 diff --git a/website/docs/reference/node-selection/methods.md b/website/docs/reference/node-selection/methods.md index 549bc5d45e1..f95b2603f45 100644 --- a/website/docs/reference/node-selection/methods.md +++ b/website/docs/reference/node-selection/methods.md @@ -156,15 +156,35 @@ dbt ls -s config.transient:true ### The "test_type" method -The `test_type` method is used to select tests based on their type, `singular` or `generic`: - + +The `test_type` method is used to select tests based on their type, `singular` or `generic`: ```bash dbt test --select "test_type:generic" # run all generic tests dbt test --select "test_type:singular" # run all singular tests ``` + + + + +The `test_type` method is used to select tests based on their type: + +- [Unit tests](/docs/build/unit-tests) +- [Data tests](/docs/build/data-tests): + - [Singular](/docs/build/data-tests#singular-data-tests) + - [Generic](/docs/build/data-tests#generic-data-tests) + + +```bash +dbt test --select "test_type:unit" # run all unit tests +dbt test --select "test_type:data" # run all data tests +dbt test --select "test_type:generic" # run all generic data tests +dbt test --select "test_type:singular" # run all singular data tests +``` + + ### The "test_name" method diff --git a/website/docs/reference/resource-properties/data-formats.md b/website/docs/reference/resource-properties/data-formats.md new file mode 100644 index 00000000000..47eb013a340 --- /dev/null +++ b/website/docs/reference/resource-properties/data-formats.md @@ -0,0 +1,60 @@ +--- +title: "Supported data formats for unit tests" +sidebar_label: "Data formats" +--- + +Currently, mock data for unit testing in dbt supports two formats: + +- `dict` (default): Inline dictionary values. +- `csv`: Inline CSV values or a CSV file. + +We will support more in the future, so watch our [upgrade guides](/docs/dbt-versions/core-upgrade) and this page for updates. + +The `dict` data format is the default if no `format` is defined. + +`dict` requires an inline dictionary for `rows`: + +```yml + +unit_tests: + - name: test_my_model + model: my_model + given: + - input: ref('my_model_a') + format: dict + rows: + - {id: 1, name: gerda} + - {id: 2, b: michelle} + +``` + +When using the `csv` format, you can use either an inline CSV string for `rows`: + +```yml + +unit_tests: + - name: test_my_model + model: my_model + given: + - input: ref('my_model_a') + format: csv + rows: | + id,name + 1,gerda + 2,michelle + +``` + +Or, you can provide the name of a CSV file in the `tests/fixtures` directory (or the configured `test-paths` location) of your project for `fixture`: + +```yml + +unit_tests: + - name: test_my_model + model: my_model + given: + - input: ref('my_model_a') + format: csv + fixture: my_model_a_fixture + +``` \ No newline at end of file diff --git a/website/docs/reference/resource-properties/unit-test-input.md b/website/docs/reference/resource-properties/unit-test-input.md new file mode 100644 index 00000000000..d9c04e7c7b6 --- /dev/null +++ b/website/docs/reference/resource-properties/unit-test-input.md @@ -0,0 +1,34 @@ +--- +title: "Input for unit tests" +sidebar_label: "Input" +--- + +Use inputs in your unit tests to reference a specific model or source for the test: + +- For `input:`, use a string that represents a `ref` or `source` call: + - `ref('my_model')` or `ref('my_model', v='2')` or `ref('dougs_project', 'users')` + - `source('source_schema', 'source_name')` +- Optionally use for seeds: + - If you don’t supply an input for a seed, we will use the seed _as_ the input. + - If you do supply an input for a seed, we will use that input instead. +- Use “empty” inputs by setting rows to an empty list `rows: []` + +```yml + +unit_tests: + - name: test_is_valid_email_address # this is the unique name of the test + model: dim_customers # name of the model I'm unit testing + given: # the mock data for your inputs + - input: ref('stg_customers') + rows: + - {email: cool@example.com, email_top_level_domain: example.com} + - {email: cool@unknown.com, email_top_level_domain: unknown.com} + - {email: badgmail.com, email_top_level_domain: gmail.com} + - {email: missingdot@gmailcom, email_top_level_domain: gmail.com} + - input: ref('top_level_email_domains') + rows: + - {tld: example.com} + - {tld: gmail.com} +... + +``` \ No newline at end of file diff --git a/website/docs/reference/resource-properties/unit-test-overrides.md b/website/docs/reference/resource-properties/unit-test-overrides.md new file mode 100644 index 00000000000..48d03187c9f --- /dev/null +++ b/website/docs/reference/resource-properties/unit-test-overrides.md @@ -0,0 +1,67 @@ +--- +title: "Unit test overrides" +sidebar_label: "Overrides" +--- + +When configuring your unit test, you can override the output of [macros](/docs/build/jinja-macros#macros), [project variables](/docs/build/project-variables), or [environment variables](/docs/build/environment-variables) for a given unit test. + +```yml + + - name: test_my_model_overrides + model: my_model + given: + - input: ref('my_model_a') + rows: + - {id: 1, a: 1} + - input: ref('my_model_b') + rows: + - {id: 1, b: 2} + - {id: 2, b: 2} + overrides: + macros: + type_numeric: override + invocation_id: 123 + vars: + my_test: var_override + env_vars: + MY_TEST: env_var_override + expect: + rows: + - {macro_call: override, var_call: var_override, env_var_call: env_var_override, invocation_id: 123} + +``` + +## Macros + +You can override the output of any macro in your unit test defition. + +If the model you're unit testing uses these macros, you must override them: + - [`is_incremental`](/docs/build/incremental-models#understanding-the-is_incremental-macro): If you're unit testing an incremental model, you must explicity set `is_incremental` to `true` or `false`. See more docs on unit testing incremental models [here](/docs/build/unit-tests#unit-testing-incremental-models). + + ```yml + + unit_tests: + - name: my_unit_test + model: my_incremental_model + overrides: + macros: + # unit test this model in "full refresh" mode + is_incremental: false + ... + + ``` + + - [`dbt_utils.star`](/blog/star-sql-love-letter): If you're unit testing a model that uses the `star` macro, you must explicity set `star` to a list of columns. This is because the `star` only accepts a [relation](/reference/dbt-classes#relation) for the `from` argument; the unit test mock input data is injected directly into the model SQL, replacing the `ref('')` or `source('')` function, causing the `star` macro to fail unless overidden. + + ```yml + + unit_tests: + - name: my_other_unit_test + model: my_model_that_uses_star + overrides: + macros: + # explicity set star to relevant list of columns + star: col_a,col_b,col_c + ... + + ``` \ No newline at end of file diff --git a/website/docs/reference/resource-properties/unit-testing-versions.md b/website/docs/reference/resource-properties/unit-testing-versions.md new file mode 100644 index 00000000000..67236317650 --- /dev/null +++ b/website/docs/reference/resource-properties/unit-testing-versions.md @@ -0,0 +1,34 @@ +--- +title: "Unit testing versioned SQL models" +sidebar_label: "Versions" +--- + +If your model has multiple versions, the default unit test will run on _all_ versions of your model. To specify version(s) of your model to unit test, use `include` or `exclude` for the desired versions in your model versions config: + +```yaml + +# my test_is_valid_email_address unit test will run on all versions of my_model +unit_tests: + - name: test_is_valid_email_address + model: my_model + ... + +# my test_is_valid_email_address unit test will run on ONLY version 2 of my_model +unit_tests: + - name: test_is_valid_email_address + model: my_model + versions: + include: + - 2 + ... + +# my test_is_valid_email_address unit test will run on all versions EXCEPT 1 of my_model +unit_tests: + - name: test_is_valid_email_address + model: my_model + versions: + exclude: + - 1 + ... + +``` \ No newline at end of file diff --git a/website/docs/reference/resource-properties/unit-tests.md b/website/docs/reference/resource-properties/unit-tests.md index 8f05cb021b5..45ab428c5d0 100644 --- a/website/docs/reference/resource-properties/unit-tests.md +++ b/website/docs/reference/resource-properties/unit-tests.md @@ -11,6 +11,13 @@ Unit tests validate your SQL modeling logic on a small set of static inputs befo To run only your unit tests, use the command: `dbt test --select test_type:unit` +## Before you begin + +- We currently only support unit testing SQL models. +- We currently only support adding unit tests to models in your _current_ project. +- If your model has multiple versions, by default the unit test will run on *all* versions of your model. Read [unit testing versioned models](#unit-testing-versioned-models) for more information. +- Unit tests must be defined in a YML file in your `models/` directory. + ```yml @@ -52,165 +59,8 @@ unit_tests: - -## About writing unit tests - -Unit tests are currently limited to testing SQL models and only models in your current project. - -### Versions -If your model has multiple versions, the default unit test will run on *all* versions of your model. To specify version(s) of your model to unit test, use `include` or `exclude` for the desired versions in your model versions config: - -```yaml - -# my test_is_valid_email_address unit test will run on all versions of my_model -unit_tests: - - name: test_is_valid_email_address - model: my_model - ... - -# my test_is_valid_email_address unit test will run on ONLY version 2 of my_model -unit_tests: - - name: test_is_valid_email_address - model: my_model - versions: - include: - - 2 - ... - -# my test_is_valid_email_address unit test will run on all versions EXCEPT 1 of my_model -unit_tests: - - name: test_is_valid_email_address - model: my_model - versions: - exclude: - - 1 - ... - -``` - -### Format - -When using `format: dict` you must supply an in-line dictionary for `rows:` (this is the default, if you don’t specify a `format`) - -```yml - -unit_tests: - - name: test_my_model - model: my_model - given: - - input: ref('my_model_a') - format: dict - rows: - - {id: 1, name: gerda} - - {id: 2, b: michelle} - ... -``` - -When `format: csv`, can either supply: - - An inline csv string for `rows:` - - ```yaml - unit_tests: - - name: test_my_model - model: my_model - given: - - input: ref('my_model_a') - format: csv - rows: | - id,name - 1,gerda - 2,michelle - ... - ``` - - - - The name of a csv file in the `tests/fixtures` directory in your project (or the directory configured for [test-paths](https://docs.getdbt.com/reference/project-configs/test-paths)) for `fixture`: - - ```yaml - unit_tests: - - name: test_my_model - model: my_model - given: - - input: ref('my_model_a') - format: csv - fixture: my_model_a_fixture - ... - ``` - - ```csv - # tests/fixtures/my_model_a_fixture.csv - 1,gerda - 2,michelle - ``` - -### Input - -- `input:` string that represents a `ref` or `source` call: - - `ref('my_model')` or `ref('my_model', v='2')` or `ref('dougs_project', 'users')` - - `source('source_schema', 'source_name')` -- `input:` is optional for seeds: - - If you don’t supply an input for a seed, we will use the seed *as* the input. - - If you do supply an input for a seed, we will use that input instead. -- You can also have “empty” inputs, by setting rows to an empty list `rows: []` - -### Overrides - -When configuring your unit test, you can override the output of [macros](/docs/build/jinja-macros#macros), [project variables](/docs/build/project-variables), or [environment variables](/docs/build/environment-variables) for a given unit test. - -```yml - -unit_tests: - - name: my_favorite_unit_test - model: my_favorite_model - overrides: - macros: - is_incremental: false - current_timestamp: "date('2023-01-15')" - vars: - my_name: grace - platforms: ['web', 'mobile'] - env_vars: - DBT_ENVIRONMENT_NAME: prod - ... - -``` - -#### Macros - -You can override the output of any macro in your unit test defition. - -There are some macros you _must_ override if the model you're unit testing uses them: - - [`is_incremental`](/docs/build/incremental-models#understanding-the-is_incremental-macro): If you're unit testing an incremental model, you must explicity set `is_incremental` to `true` or `false`. See more docs on unit testing incremental models [here](/docs/build/unit-tests#unit-testing-incremental-models). - - ```yml - - unit_tests: - - name: my_unit_test - model: my_incremental_model - overrides: - macros: - # unit test this model in "full refresh" mode - is_incremental: false - ... - - ``` - - - [`dbt_utils.star`](/blog/star-sql-love-letter): If you're unit testing a model that uses the `star` macro, you must explicity set `star` to a list of columns. This is because the `star` only accepts a [relation](/reference/dbt-classes#relation) for the `from` argument; the unit test mock input data is injected directly into the model SQL, replacing the `ref('')` or `source('')` function, causing the `star` macro to fail unless overidden. - - ```yml - - unit_tests: - - name: my_other_unit_test - model: my_model_that_uses_star - overrides: - macros: - # explicity set star to relevant list of columns - star: col_a,col_b,col_c - ... - - ``` - ## Examples + ```yml unit_tests: diff --git a/website/sidebars.js b/website/sidebars.js index a76d7089390..422c964adf8 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -842,6 +842,10 @@ const sidebarSettings = { label: "For unit tests", items: [ "reference/resource-properties/unit-tests", + "reference/resource-properties/unit-test-input", + "reference/resource-properties/unit-testing-versions", + "reference/resource-properties/unit-test-overrides", + "reference/resource-properties/data-formats", ], }, {