Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: override materialization python models #8538

Merged
merged 9 commits into from
Nov 14, 2024
Merged

fix: override materialization python models #8538

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
9636aa5
fix: override materialization python models
devmessias Aug 31, 2023
7e338a1
chore: changie doc
devmessias Sep 1, 2023
55ef48c
fix: change view materialization to table in python models
devmessias Oct 2, 2023
18ed6b0
feat: create PythonNodeConfig
devmessias Oct 3, 2023
7fbaf77
fix: revert PythonNodeModel
devmessias Oct 2, 2024
6f2741f
fix: python default model should be table
devmessias Oct 2, 2024
111ef65
test(python): should test for incremental not view
devmessias Oct 2, 2024
7c94bde
chore: changie doc
devmessias Oct 2, 2024
7d743da
take out additional line
ChenyuLInx Nov 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changes/unreleased/Fixes-20241002-142638.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Fixes
body: ' override materialization python models'
time: 2024-10-02T14:26:38.332191458-03:00
custom:
Author: devmessias
Issue: "8520"
10 changes: 9 additions & 1 deletion core/dbt/parser/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,6 @@ def _create_parsetime_node(
name = block.name
if block.path.relative_path.endswith(".py"):
language = ModelLanguage.python
config.add_config_call({"materialized": "table"})
else:
# this is not ideal but we have a lot of tests to adjust if don't do it
language = ModelLanguage.sql
Expand Down Expand Up @@ -322,6 +321,15 @@ def update_parsed_node_config(
# build_config_dict takes the config_call_dict in the ContextConfig object
# and calls calculate_node_config to combine dbt_project configs and
# config calls from SQL files, plus patch configs (from schema files)
# This normalize the config for a model node due #8520; should be improved latter
if not patch_config_dict:
patch_config_dict = {}
if (
parsed_node.resource_type == NodeType.Model
and parsed_node.language == ModelLanguage.python
):
if "materialized" not in patch_config_dict:
patch_config_dict["materialized"] = "table"
config_dict = config.build_config_dict(patch_config_dict=patch_config_dict)

# Set tags on node provided in config blocks. Tags are additive, so even if
Expand Down
4 changes: 2 additions & 2 deletions tests/unit/parser/test_parser.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1004,7 +1004,7 @@ def model(dbt, session):
import pandas as pd

def model(dbt, session):
dbt.config(materialized="view")
dbt.config(materialized="incremental")
return pd.dataframe([1, 2])
"""

Expand Down Expand Up @@ -1199,7 +1199,7 @@ def test_python_model_custom_materialization(self):
self.parser.manifest.files[block.file.file_id] = block.file
self.parser.parse_file(block)
node = list(self.parser.manifest.nodes.values())[0]
self.assertEqual(node.get_materialization(), "view")
self.assertEqual(node.get_materialization(), "incremental")


class StaticModelParserTest(BaseParserTest):
Expand Down