Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to Run Only Tests for a Specific dbt Model Without Executing the Model #1242

Open
1 task
ame589 opened this issue Oct 4, 2024 · 2 comments
Open
1 task
Labels
area:config Related to configuration, like YAML files, environment variables, or executer configuration dbt:test Primarily related to dbt test command or functionality enhancement New feature or request stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed triage-needed Items need to be reviewed / assigned to milestone

Comments

@ame589
Copy link

ame589 commented Oct 4, 2024

Description

I would like to propose a feature that allows users to execute only the tests, defined at yaml level of a specific dbt model, without triggering the execution of the model itself in an DbtTaskGroup Airflow pipeline.

Use case/motivation

This could be achieved by adding a configuration that allows users to specify a model in the select parameter and set the test behavior to ONLY_TEST. Here’s an example of how this could look:

    return DbtTaskGroup(
        group_id=f"{model_name}",
        project_config=project_config,
        profile_config=profile_config,
        execution_config=execution_config,
        render_config=RenderConfig(
            select=[mod],  # Select the specific flow_name to run
            exclude=[],  # Exclude the dependency if provided
            test_behavior=TestBehavior.ONLY_TEST # Run only tests, not the model itself
        ),
        default_args={"retries": 2, "trigger_rule": "all_success"},
        operator_args={
            "install_deps": True
        }
    )

With this configuration:

  1. The select parameter will focus the dbt task on the specific model.
  2. The test_behavior=TestBehavior.ONLY_TEST setting will ensure that only the tests for the model are executed, while the actual data transformations are skipped.

This would make the testing process more flexible and tailored to specific use cases.

Related issues

No response

Are you willing to submit a PR?

  • Yes, I am willing to submit a PR!
@ame589 ame589 added enhancement New feature or request triage-needed Items need to be reviewed / assigned to milestone labels Oct 4, 2024
@dosubot dosubot bot added area:config Related to configuration, like YAML files, environment variables, or executer configuration dbt:test Primarily related to dbt test command or functionality labels Oct 4, 2024
@tatiana
Copy link
Collaborator

tatiana commented Oct 29, 2024

@ame589 This feels highly related to #959.

Would it be an option to use the DbtTestLocalOperator (

class DbtTestLocalOperator(DbtTestMixin, DbtLocalBaseOperator):
) or an equivalent directly to run tests only?

As illustrated in:

test_operator = DbtTestLocalOperator(
profile_config=real_profile_config,
project_dir=DBT_PROJ_DIR,
task_id="test",
dbt_cmd_flags=["--models", "stg_customers"],
install_deps=True,
append_env=True,
)

Copy link

github-actions bot commented Dec 4, 2024

This issue is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:config Related to configuration, like YAML files, environment variables, or executer configuration dbt:test Primarily related to dbt test command or functionality enhancement New feature or request stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed triage-needed Items need to be reviewed / assigned to milestone
Projects
None yet
Development

No branches or pull requests

2 participants