0.19.1 unable to infer dependencies for custom tests

[bmwilly]

Describe the bug

I have a macro test_valid_code defining a custom test that uses {{ ref('model') }}. For dbt 0.19.1 (this only fails for 0.19.1), dbt compile gives

Encountered an error:
Runtime Error
  Compilation Error in test valid_code (models/model.yml)
    dbt was unable to infer all dependencies for the model "valid_code_model".
    This typically happens when ref() is placed within a conditional block.

    To fix this, add the following hint to the top of the model "valid_code_model":

    -- depends_on: {{ ref('model') }}

    > in macro test_valid_code (macros/tests/test_valid_code.sql)
    > called by test valid_code_model (models/model.yml)
    > called by test valid_code_model (models/model.yml)

Trying to add -- depends_on: {{ ref('model') }} to the model and/or test file doesn't help.

Expected behavior

dbt {compile,run} to not give any errors.

System information

Which database are you using dbt with?

• postgres
• redshift
• bigquery
• snowflake
• other (specify: ____________)

The output of dbt --version:

$ dbt --version
installed version: 0.19.1
   latest version: 0.19.1

Up to date!

Plugins:
  - bigquery: 0.19.1
  - snowflake: 0.19.1
  - redshift: 0.19.1
  - postgres: 0.19.1

The operating system you're using:

macOS 11.2.3

The output of python --version:

Python 3.8.5

[jtcohen6]

@bmwilly Thanks for the report! It does look like this represents a regression from v0.19.0.

Could you clarify what {{ ref('model') }} represents here? Is that the model that the test is being defined on? If so, this could just be {{ model }} instead (as the built-in schema tests have it).

If this represents a different model that is being directly referenced from within your schema test definition, then I've been able to reproduce this error:

{% macro test_valid_code(model) %}

    select count(*) from (

        select * from {{ model }}
        union all
        select * from {{ ref('model_b') }}
        
    )

{% endmacro %}

Compilation Error in test valid_code_model_a_ (models/schema.yml)
  dbt was unable to infer all dependencies for the model "valid_code_model_a_".
  This typically happens when ref() is placed within a conditional block.

  To fix this, add the following hint to the top of the model "valid_code_model_a_":

  -- depends_on: {{ ref('model_b') }}

  > in macro test_valid_code (macros/test_valid_code.sql)
  > called by test valid_code_model_a_ (models/schema.yml)
  > called by test valid_code_model_a_ (models/schema.yml)

Here's one way to work around this in the meantime, so that the dependency is still detected:

{% macro test_valid_code(model, other_model) %}

    select count(*) from (

        select * from {{ model }}
        union all
        select * from {{ other_model }}
        
    ) validation_errors

{% endmacro %}

version: 2

models:
  - name: model_a
    tests:
      - valid_code:
          other_model: "{{ ref('model_b') }}"

The latter construction is what I'd recommend for generic/schema tests, anyhow. That said, we'll investigate and see if we can figure a way to restore the previous behavior.

[bmwilly]

@jtcohen6 thanks for the quick response and for the understanding from my not very clear report!

If this represents a different model that is being directly referenced from within your schema test definition, then I've been able to reproduce this error:

yes, I did mean a different model that is being directly referenced in the schema test definition, similar to the example you give.

I was able to fix this using your workaround — thanks!

Feel free to close this issue; I'll leave it open in case you want to use it to track restoring the previous functionality.

[jtcohen6]

@bmwilly We're including a fix for this in v0.19.2, for which we just cut a release candidate. Any chance you could test locally and confirm that it fixes the issue you described above?

pip install dbt==0.19.2rc1
# or
brew install dbt@0.19.2-rc1

[bmwilly]

@jtcohen6 seems to work, thanks!

[MartinNowak]

pip install dbt==0.19.2rc1
# or
brew install dbt@0.19.2-rc1

@jtcohen6 works if the test is in the same package, but apparently still fails with tests from another package.

[jtcohen6]

Thanks for the report @MartinNowak! I wonder if you're seeing the same error as the one reported over in #3324?

[MartinNowak]

Thanks for the report @MartinNowak! I wonder if you're seeing the same error as the one reported over in #3324?

Thx for reproducing and appending, indeed it's sth. in between the two problems.
#3324 (comment)