Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run dbt deps when using load_mode=LoadMode.DBT_LS #445

Closed
tatiana opened this issue Aug 7, 2023 · 2 comments · Fixed by #481
Closed

Run dbt deps when using load_mode=LoadMode.DBT_LS #445

tatiana opened this issue Aug 7, 2023 · 2 comments · Fixed by #481
Labels
enhancement New feature or request good first issue Good for newcomers
Milestone
1.1.0

Comments

@tatiana
Copy link
Collaborator

tatiana commented Aug 7, 2023

As of Cosmos 1.0, when we run dbt ls as part of load_mode=LoadMode.DBT_LS, we don't pre-run dbt deps.

This would be a useful feature based on the community feedback, including this thread in the Airflow Community Slack.
https://apache-airflow.slack.com/archives/C059CC42E9W/p1691413162622129?thread_ts=1691398106.912769&cid=C059CC42E9W

The goal with this ticket is to run dbt deps before dbt ls.
@tatiana tatiana added enhancement New feature or request good first issue Good for newcomers labels Aug 7, 2023
@tatiana tatiana mentioned this issue Aug 21, 2023
Closed
@DanMawdsleyBA DanMawdsleyBA mentioned this issue Aug 21, 2023
Merged
2 tasks
@DanMawdsleyBA
Copy link
Contributor

@tatiana This is my first change here so forgive me for any mistakes. I made the changes in a forked repo tested it locally and ran ok. I also added a package.yml to the example dev project so that should prove the change works when it runs the exitisting tests. See below screenshot of logs when I ran:
image
I looked at the tests that ran a couple of failures on the integration tests although I don't think it is related to the change I made:
image
All the failures were the same. So let me know if you're happy with the changes :)

@tatiana
Copy link
Collaborator Author

tatiana commented Aug 22, 2023

Hi @DanMawdsleyBA, this is amazing. Thank you very much for your first contribution! 🎉
It is looking good, it is a great start! I'll add two comments to the PR. Please, do let me know if you need additional support and we can jump in.

tatiana added a commit that referenced this issue Aug 31, 2023
@DanMawdsleyBA @tatiana
Run dbt deps before running the dbt ls command (#481)
13087d5 
Previously, the `load_mode.DBT_LS` would not install packages declared in `packages.yml`. During parse time, Cosmos would generate a DAG without nodes.

With this change, Cosmos runs the `dbt deps` command before running the `dbt ls` command, avoiding this issue.

Closes: #445

Co-authored-by: Tatiana Al-Chueyr <tatiana.alchueyr@gmail.com>
@tatiana tatiana added this to the 1.1.0 milestone Sep 7, 2023
@tatiana tatiana mentioned this issue Nov 2, 2023
Closed
@dosubot dosubot bot mentioned this issue Nov 29, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Milestone
1.1.0
Development

Successfully merging a pull request may close this issue.

Run dbt deps before running the dbt ls command DanMawdsleyBA/astronomer-cosmos
2 participants
@tatiana @DanMawdsleyBA