Configurable description in yaml config for generic tests

[kwigley]

Describe the feature

Add the ability to define descriptions in yaml files. Support the ability to use the rendered values of the test args ({{ model }} and {{ column_name }} by default)

Consider the following example:

# macros/reusable_tests.sql
{% test not_null(model, column_name), adapter = 'default' %}

    select * from {{ model }}
    where {{ column_name }} is null

{% endtest %}

# macros/schema.yml
tests:
 - name: not_null
   adapter: default
   description: "Assert that {{ column_name }} is not null in {{ model }}"
   args:
     - name: model
       description: "The model to test"
     - name: column_name
       description: "The column or expression to test"

Additional context in #1173.

[jtcohen6]

In an ideal world, all arguments to the generic test could be in scope when resolving the description. So, for instance, {{ values }} for an accepted_values test. If that's too tricky, I think {{ model }} + {{ column_name }} make for an excellent first cut.

Note that one-off/bespoke tests should also be able to have descriptions defined (#2578), but those wouldn't have the same kinds of variables in context—just doc blocks, like any other resource description.

[jtcohen6]

We've discussed using this description, if available / in lieu of the test unique_id, to construct the failure message for stdout logging, and to store that same message in run_results.json for better metadata. In concert with the changes in #3258 + #3321, we've imagined:

Completed with 1 error and 1 warning:

Failure in test: Ensure column `fun` in `my_model` meets my custom criteria
  Found 1 failing record, expected 0.

  compiled SQL at target/compiled/my_project/models/schema.yml/schema_test/my_custom_generic_my_model_fun.sql

Warning in test: my_custom_bespoke
  Calculated failure is 1, which is >0.

  compiled SQL at target/compiled/my_project/tests/my_custom_bespoke.sql

Done. PASS=0 WARN=1 ERROR=1 SKIP=0 TOTAL=2

@kwigley Is that something you'd want to tackle in this issue / #3302, or should we open up a separate issue?

[jtcohen6]

Update on the thought above: I'm less sure about whether we need to include that synthesized message in run_results.json, since the description will be in the manifest.json, and the two can be joined together on the unique_id.

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

[joellabes]

I still want this! Mostly for the aliasing reasons discussed in #3815

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest; add a comment to notify the maintainers.

[dbeatty10]

Re-labeling as a duplicate. Here are the related issues:

• Allow instances of generic data tests to be documented #2578
• [CT-3323] Allow singular tests to be documented #9005