CT-729 Include schema model config in unrendered config

[gshank]

resolves #5338

Description

Unrendered_config originally only included config from models. When we added config for models in schema files, we didn't include that in unrendered_config. Unrendered_config still doesn't include model config from dbt_project.yml, since the code explicitly avoided that.

Checklist

• I have read the contributing guide and understand what's expected of me
• I have signed the CLA
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have opened an issue to add/update docs, or docs changes are not required/relevant for this PR
• I have run changie new to create a changelog entry

[gshank]

@jtcohen6 I didn't find any actual documentation about what unrendered_config is supposed to contain. The code goes to some trouble to not include the config from dbt_project.yml. I'm not exactly clear on why that is. Could you cast some light on it?

[jtcohen6]

@gshank My impression is that unrendered_config stores any raw Jinja expressions in dbt_project.yml. So for instance:

# dbt_project.yml
models:
  +enabled: "{{ target.name == 'dev' }}"

The entry for a model node in the manifest should show up like:

"config": {
        "enabled": true,
        "alias": null,
        "schema": null,
        "database": null,
        "tags": [],
        "meta": {},
        "materialized": "view",
        "persist_docs": {},
        "quoting": {},
        "column_types": {},
        "full_refresh": null,
        "unique_key": null,
        "on_schema_change": "ignore",
        "grants": {},
        "packages": [],
        "post-hook": [],
        "pre-hook": []
      },
...
"unrendered_config": {
        "enabled": "{{ target.name == 'dev' }}"
      },

dbt should use the "unrendered" form of enabled for comparison, instead of the rendered form, which inevitably differs across target environments (even though it hasn't changed).

Docs on this behavior: https://docs.getdbt.com/reference/node-selection/state-comparison-caveats#false-positives

I believe we are updating unrendered_config with values from dbt_project.yml, it's just happening during config resolution here:

dbt-core/core/dbt/parser/base.py

Lines 289 to 291 in e48f7ab

	# unrendered_config is used to compare the original database/schema/alias
	# values and to handle 'same_config' and 'same_contents' calls
	parsed_node.unrendered_config = config.build_config_dict(rendered=False)

[gshank]

If you look at "build_config_dict" in core/dbt/context/context_config.py it uses different renderers for rendered and unrendered. The "unrendered" flavor uses UnrenderedConfigGenerator. If you compare that to ContextConfigGenerator, the unrendered config generator explicitly skips loading the config from dbt_project.yml.

[gshank]

It looks like the issue is not that the dbt_project.yml config isn't processed, but that the unrendered_config dictionaries aren't merged, so that we lose dbt_project.yml config items in unrendered_config that are overwritten by later config. The actual config in the node is correct, because that part is merged properly.

[iknox-fa]

Removing myself as a reviewer on this one since I don't know enough about parsing to be useful here.

[jtcohen6]

@gshank Did we decide if the behavior change here was the right one? Or better than the status quo, at least? I remember discussing that we don't have any way to access the unrendered form of patch (.yml) configs for the model, but that is the most correct set of information to include in the model's unrendered_config.

[gshank]

My recollection is that not including the dbt_project configs when re-rendering was actually a bug.