fix(dbt): add use_identifiers option and avoid duplicate descriptions #3179
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checklist
This PR adds an additional option
use_identifiers
to the dbt configuration to use nodes identifiers instead of names if those are defined, and default back to names if not.Previously this was achieved by setting
load_schemas=False
however there are cases when we want to load schemas while using node identifiers instead of names (for ex. dbt does not write column descriptions to Snowflake on views* and those need to be ingested withload_schemas=True
, while using identifiers to preserve the lineage).*see https://github.com/dbt-labs/dbt/issues/3291
Also:
load_catalog
toload_schemas
in dbt.py for consistency with the config file