Feature: Add tests to dbt docs UI and references

[tayloramurphy]

Feature

Feature description

Specific requests:

• Full documentation for tests
• Tests and Macros visible in UI via sidebar
• Reference-able using explicit links or Jinja references
• Jinja references within models

Per a discussion in slack, I'd like to see custom data tests within the dbt docs UI. At a minimum having the tests viewable via the sidebar would be great (possibly along with dbt_project.yml and profiles.yml).

I'd also like to be able to link to the tests either using the explicit link [my_link_label](/#!/model/model.my_package_name.my_model_name) or by using {{ tests('assert_my_data') }}.

To add on to that, I would love if I could reference a test within a dbt model to indicate that it's relevant to that part of the model. For example, I have a CTE here

...
sub_months AS (

    SELECT
      account_number,
      cohort_month,
      subscription_name,
      subscription_name_slugify,
      subscription_slug_for_counting
    FROM {{ ref('zuora_base_mrr') }}
    GROUP BY 1, 2, 3, 4, 5

),
...

and it’d be cool to reference the test w/in the CTE:

...
sub_months AS (

    SELECT
      account_number,
      cohort_month,
      subscription_name,
      subscription_name_slugify,
      subscription_slug_for_counting
    FROM {{ ref('zuora_base_mrr') }}
    GROUP BY 1, 2, 3, 4, 5
   {{ test('assert_unique_combination_account') }}
),
...

And all that test jinja would do is create a clickable link (or modal) to pop-up the relevant test / documentation.

Who will this benefit?

Any team writing a non-zero amount of tests (which should be all teams, right?).

[drewbanin]

Thanks for the writeup @tayloramurphy! I think it's a really good and straightforward idea to expose custom data tests in the dbt docs sidebar.

Custom data tests can reference zero or models, so I'm not exactly sure how to best expose that in the docs ui for a model. Maybe just a list like:

Tested by:
 - [assert_unique_combination_account](#)
 - [assert_unique_other_thing_too](#)

^ I imagine we can expose a macro that will generate correct urls for different resources, so you won't need to hassle with building the url manually everywhere.

I like your idea of colocating the existence of the test in the model definition. Would it instead make sense to more explicitly define what is being tested in the custom data test? I can imagine something like:

-- tests/assert_unique_combination_account.sql

{% docs assert_unique_combination_account %}
  More information about what is being tested here
{% enddocs %}

{{ document_test_for("model_name", "assert_unique_combination_account") }}
{{ document_test_for("other_model", "assert_unique_combination_account") }}

select * from ...

I think we'd probably want to make this syntax more closely match the existing documentation syntax, but just use this example for the general idea. If we do this, then we'd be able to link custom data tests to the models that they actually test; not just those that are refd in the SQL. Further, we'd have rich content to expose in the models section of the docs ui.

Let me know what you think about all of this!

[tayloramurphy]

+1 on everything you suggested.

Another possible iteration could be to link to specific fields as well. So on the columns listing you might see:

column	...	tests
account_id	...	U N C

Where the C is for custom and when you expand there could be a link to the test as well.

But generally, anything that makes the tests a first class citizen with the same documentation level as models is moving in the right direction. Thanks Drew!