[Feature] Fail DAG Execution When dbt Nodes or Paths Don’t Exist

[pankajkoti]

Description

Currently, when dbt-core is executed with a node name or path that doesn’t exist, it prints "nothing to do" and completes successfully. While this behavior makes sense from a program perspective, it can lead to issues in Cosmos workflows when nodes or paths are misspelled or incorrectly configured.

For instance, consider a dbt DAG structured as follows:

A
 \
  C
  /
 B

If there is a typo in the select command for B, the dbt task for B does nothing, but C continues to run and succeeds. This behavior makes it difficult to identify configuration errors, especially in scenarios where node or tag names change but are not updated consistently across the workflow.

We propose a feature in Cosmos to treat dbt nodes or paths that result in "nothing to do" as a failure. This would ensure that:

1. Misspelled or missing node names/paths are caught early
2. DAG executions fail explicitly when critical dependencies are not met, preventing downstream nodes from running

Use case/motivation

This feature would help users:

• Catch errors arising from renaming or misconfiguring nodes/tags in dbt DAGs
• Improve reliability and accuracy in workflows where dbt nodes depend on specific upstream tasks. This feature would add robustness to Cosmos-managed dbt workflows by preventing silent errors that could lead to incorrect data processing or reporting

Proposed Solution

• Introduce an option in Cosmos to detect and fail tasks where dbt prints "nothing to do"
• Provide detailed error messages in the logs indicating the problematic node or path

Related issues

No response

Are you willing to submit a PR?

• Yes, I am willing to submit a PR!