Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-1924] Support 'contract' config outside model yaml file #6756

Closed
Tracked by #6747
MichelleArk opened this issue Jan 26, 2023 · 1 comment · Fixed by #7148
Closed
Tracked by #6747

[CT-1924] Support 'contract' config outside model yaml file #6756

MichelleArk opened this issue Jan 26, 2023 · 1 comment · Fixed by #7148
Assignees

Comments

@MichelleArk
Copy link
Contributor

MichelleArk commented Jan 26, 2023

In the first cut of dbt constraints, constraints_enabled (= “this model is contracted and should be treated as such”, to be renamed to contract) can only be set in model yaml files, alongside the columns attribute to enable parse-time validation.

We should either:

  1. Resolve some tech debt around parsing ([CT-1747] Enable flagging nodes/configs for deferred validation #6511) to enable setting this config elsewhere ((dbt_project.yml, yaml file, within .sql file) or,
  2. Remove the at parse-time validation, failure will happen at runtime

Relevant discussions:

@github-actions github-actions bot changed the title Support "constraints_enabled"/"contracted" config outside model yaml file [CT-1924] Support "constraints_enabled"/"contracted" config outside model yaml file Jan 26, 2023
@MichelleArk MichelleArk changed the title [CT-1924] Support "constraints_enabled"/"contracted" config outside model yaml file [CT-1924] Support 'contracted' config outside model yaml file Jan 26, 2023
@MichelleArk MichelleArk changed the title [CT-1924] Support 'contracted' config outside model yaml file [CT-1924] Support 'contract' config outside model yaml file Feb 14, 2023
@jtcohen6
Copy link
Contributor

jtcohen6 commented Mar 2, 2023

Let's take #6511 off the critical path, and take the "lazy" approach for now, by raising the error at runtime instead of parse time.

#6986 is going to give us a neat place to raise this error at runtime:

{% macro default__get_empty_schema_sql(columns) %}
select
{% for i in columns %}
{%- set col = columns[i] -%}
cast(null as {{ col['data_type'] }}) as {{ col['name'] }}{{ ", " if not loop.last }}
{%- endfor -%}
{% endmacro %}

If col['data_type'] == "", let's raise a clear exception — rather than having it crop up as a harder-to-debug syntax error when we submit this query to the data platform:

cast(col_name as ),

Then, we should remove these validation checks:

# If we have "contract" in the config, copy to node level, for backwards
# compatibility with earlier node-only config.
if config_dict.get("contract", False):
parsed_node.contract = True
parser_name = type(self).__name__
if parser_name == "ModelParser":
original_file_path = parsed_node.original_file_path
error_message = "\n `contract=true` can only be configured within `schema.yml` files\n NOT within a model file(ex: .sql, .py) or `dbt_project.yml`."
raise ParsingError(
f"Original File Path: ({original_file_path})\nConstraints must be defined in a `yml` schema configuration file like `schema.yml`.\nOnly the SQL table materialization is supported for constraints. \n`data_type` values must be defined for all columns and NOT be null or blank.{error_message}"
)

def constraints_data_type_validator(self, patched_node):
data_type_errors = set()
for column, column_info in patched_node.columns.items():
if column_info.data_type is None:
data_type_error = {column}
data_type_errors.update(data_type_error)
data_type_errors_msg = (
f"\n Columns with `data_type` Blank/Null Errors: {data_type_errors}"
)
data_type_errors_msg_payload = f"{data_type_errors_msg if data_type_errors else None}"
return data_type_errors_msg_payload

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants