[Feature] Prevent Scheduling of Multi-Parent Data Tests in Two DAGs

[Magicbeanbuyer]

Description

Currently, when a data test in dbt compares two models from different DAGs using the compare_model configuration, cosmos package schedules two tests, one for each model in each DAG. If an error occurs during the test, both tests will fail. This behavior can lead to unintended DAG failures and may not align with the user's expectations.

The behaviour can be reproduced by this repository. Note that setting should_detach_multiple_parents_tests to True does not solve this issue, when the two Models are in two different DAGs.

I would propose instead of creating a test in both DAGs, the feature would ensure that the test is only scheduled in the DAG where it is defined.

This change would improve the reliability of the data testing process and align more closely with user expectations. It would also simplify the process of managing and resolving test failures, as the responsibility would clearly lie with the owner of the DAG where the test is defined.

The implementation would likely involve changes to modify the logic around how tests with the compare_model configuration are scheduled.

Use case/motivation

No response

Related issues

No response

Are you willing to submit a PR?

• Yes, I am willing to submit a PR!

[tatiana]

Hi, @Magicbeanbuyer, this looks like a significant improvement. Thank you for creating the issue! Since you're willing to contribute, I'm adding you as an assignee. Please, let us know if you'd like any support.