Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-1919] Create get_column_schema_from_query macro #6751

Closed
Tracked by #6747
MichelleArk opened this issue Jan 26, 2023 · 0 comments · Fixed by #6986
Closed
Tracked by #6747

[CT-1919] Create get_column_schema_from_query macro #6751

MichelleArk opened this issue Jan 26, 2023 · 0 comments · Fixed by #6986
Assignees
Labels
multi_project user docs [docs.getdbt.com] Needs better documentation

Comments

@MichelleArk
Copy link
Contributor

get_columns_in_query retrieves a list of column names from a model's sql by running an inexpensive select * from {{ sql }} where false limit 0 query. Currently, this macro is leveraged to efficiently enforce that a model's contracted names match with what its sql would actually produce.

However, we'll need to be able to retrieve column data types in addition to column names from a model's sql in order to enforce a model contract complete with data types.

Let's create a new macro, get_column_schema_from_query with a default implementation that returns a list of Column objects with column name and type. This macro will need to rely on adapter-specific implementations to data type off the connection cursor/API. Because aapter-specific connections may return data type representations that do not match up with user-provided values, we’ll need to use dbt adapters’ type translation (aliases → standard), to match up data types with user-provided values.

@jtcohen6 has taken a very thorough first stab at validating this approach in jerco/get-column-schema-from-query

@github-actions github-actions bot changed the title Create get_column_schema_from_query macro [CT-1919] Create get_column_schema_from_query macro Jan 26, 2023
@MichelleArk MichelleArk added the user docs [docs.getdbt.com] Needs better documentation label Feb 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
multi_project user docs [docs.getdbt.com] Needs better documentation
Projects
None yet
2 participants