[CT-1040] [Bug] Selectors fail on dbt deps

[danieldiamond]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

dbt deps fails when a selector using inheritance is implemented. Error occurring at this line.

The definition is

{'method': 'selector', 'value': ''}

i.e. the value is not being read for the selector method.

I believe it is because default: "{{ target.type == 'snowflake' | as_bool }}" yaml is not being read at time of dbt deps

Expected Behavior

selectors should be

Steps To Reproduce

1. Add selectors.yaml with inheritance e.g.

selectors:
  - name: snowflake
    default: "{{ target.type == 'snowflake' | as_bool }}"
...
  - name: pr_run
    description: PR Jobs
    definition:
      union:
        - intersection:
          - method: selector
            value: "{{ target.type }}"
          - "+state:modified+"

2. Run dbt deps

Relevant log output

Traceback (most recent call last):
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/config/selectors.py", line 38, in selectors_from_dict
    selectors = parse_from_selectors_definition(selector_file)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/graph/cli.py", line 260, in parse_from_selectors_definition
    "definition": parse_from_definition(
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/graph/cli.py", line 240, in parse_from_definition
    return parse_union_definition(definition, result=result)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/graph/cli.py", line 157, in parse_union_definition
    include, exclude = _parse_include_exclude_subdefs(
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/graph/cli.py", line 150, in _parse_include_exclude_subdefs
    parse_from_definition(definition, result=result))
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/graph/cli.py", line 242, in parse_from_definition
    return parse_intersection_definition(definition, result=result)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/graph/cli.py", line 171, in parse_intersection_definition
    include, exclude = _parse_include_exclude_subdefs(
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/graph/cli.py", line 150, in _parse_include_exclude_subdefs
    parse_from_definition(definition, result=result))
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/graph/cli.py", line 244, in parse_from_definition
    return parse_dict_definition(definition, result=result)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/graph/cli.py", line 198, in parse_dict_definition
    raise ValidationException(
dbt.exceptions.ValidationException: Runtime Error
  Existing selector definition for  not found.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/main.py", line 129, in main
    results, succeeded = handle_and_check(args)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/main.py", line 192, in handle_and_check
    task, res = run_from_args(parsed)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/main.py", line 219, in run_from_args
    task = parsed.cls.from_args(args=parsed)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/task/deps.py", line 91, in from_args
    return super().from_args(args)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/task/base.py", line 108, in from_args
    config = cls.ConfigType.from_args(args)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/config/runtime.py", line 581, in from_args
    project, profile = cls.collect_parts(args)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/config/runtime.py", line 209, in collect_parts
    project = partial.render(project_renderer)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/config/project.py", line 286, in render
    return self.create_project(rendered)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/config/project.py", line 405, in create_project
    selectors = selector_config_from_data(rendered.selectors_dict)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/config/selectors.py", line 109, in selector_config_from_data
    selectors = SelectorConfig.selectors_from_dict(selectors_data)
  File "/Users/danieldiamond/.virtualenvs/dev-dbt/lib/python3.9/site-packages/dbt/config/selectors.py", line 50, in selectors_from_dict
    raise DbtSelectorsError(
dbt.exceptions.DbtSelectorsError: Runtime Error
  Could not read selector file data: Runtime Error
    Existing selector definition for  not found.
22:42:35  Encountered an error:
Runtime Error
  Could not read selector file data: Runtime Error
    Existing selector definition for  not found.

### Environment

```markdown
- OS: MacOS Monterey 12.4
- Python: 3.9.12
- dbt: 1.2.0

Which database adapter are you using with dbt?

snowflake

Additional Context

#4827

[danieldiamond]

Interestingly, I've thrown a breakpoint() at the offending line. When running dbt deps it appears that target dictionary is not being read - hence the empty value. But when running dbt debug, the target is being read.

[danieldiamond]

cc'ing @gshank

[jtcohen6]

@danieldiamond This is indeed because profiles.yml is not read during dbt deps & dbt clean, and so target is set to {}. (That's been true since dbt Core v1.0.x.) As a result, {{ target.type }} returns None/Undefined.

[danieldiamond]

Thanks for the prompt reply @jtcohen6. What's the best way to proceed? Are we able to disable reading selector definitions at these moments? Should I rather add a default value in the jinja as a redundant fallback?

[jtcohen6]

Should I rather add a default value in the jinja as a redundant fallback?

I think this would give you the quickest resolution, yep.

Given that node selection is not relevant for deps or clean, it would strike me as sensible to skip parsing that file for those tasks.

[jtcohen6]

Proposed resolution: Do not parse selectors.yml for deps and clean tasks, because the information is never needed

[danieldiamond]

hey @jtcohen6 sorry I should've followed up, I went with

value: "{{ target.type|default('snowflake', true) }}"

as a workaround for now. but I agree that we should not parse selectors.yml for deps or clean tasks. if you point me in the right direction I'd be more than happy to create a PR with the necessary changes

[emmyoop]

Check ff it's an UnsetProfileConfig and if it is we can skip rendering the selectors.

Additionally, there seems to be some dead code in selector config object. render_from_dict and from_path (need to verify before removing).