Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix project dir argument when running debug (#1733) #1989

Merged

Conversation

franloza
Copy link
Contributor

@franloza franloza commented Dec 8, 2019

Fixes #1733

I did not make the class DebugTask inherited from RequiresProjectTask because it raised a dbt.exceptions.RuntimeException in get_nearest_project_dir() function if dbt_project.yml was not found. Instead, I just used the argument project_dir from arguments as first choice and set current directory as fall-back value.

The included test checks that, even if a dbt_project.yml is in the current directory, the --project-dir argument is used, leading to a "not found" error.

@cla-bot
Copy link

cla-bot bot commented Dec 8, 2019

Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA.

In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, don't hesitate to ping @drewbanin.

CLA has not been signed by users: @franloza

@drewbanin
Copy link
Contributor

Thanks for making this PR @franloza!

I gave this a quick spin locally and I noticed that the code change here doesn't have the effected specified in the PR description. Specifically, I saw that when running dbt debug --project-dir path/to/project, dbt still looked for the dbt_project.yml file in the cwd.

I think the relevant method codepath might be around here: https://github.com/fishtown-analytics/dbt/blob/cc491904bc7ed5c27a395e3a1936e57e4221354a/core/dbt/config/project.py#L391-L393

This method is called via _load_project in the DebugTask: https://github.com/fishtown-analytics/dbt/blob/cc491904bc7ed5c27a395e3a1936e57e4221354a/core/dbt/task/debug.py#L122-L133

I think we'll want to update the _load_project method to construct a Project object using args.project_dir. Let me know if you have any questions or further thoughts here. I'll kick off the tests regardless so we can see how they fare :)

(also: @cla-bot check)

@cla-bot cla-bot bot added the cla:yes label Dec 9, 2019
@cla-bot
Copy link

cla-bot bot commented Dec 9, 2019

The cla-bot has been summoned, and re-checked this pull request!

@franloza
Copy link
Contributor Author

Yes, you were completely right @drewbanin. As you mentioned, the added test only checked that we were looking outside current directory and returned the error in the first if statement:

if not os.path.exists(self.project_path):
    self.project_fail_details = FILE_NOT_FOUND
    return red('ERROR not found')

However, we were not covering the case of specifing an existent project.yml file outside current directory. I have added a test to cover this case and updated _load_project method to use Project.from_project_root instead of Project.from_current_directory. This way, we are using always --project-dir as first choice and current directory as fall-back value.

@drewbanin drewbanin changed the base branch from dev/0.15.1 to dev/barbara-gittings January 7, 2020 01:39
@drewbanin
Copy link
Contributor

Thanks for the update @franloza! I kicked off the tests here and I'm adding @beckjake to review. This LGTM when the tests pass :)

@@ -125,7 +126,7 @@ def _load_project(self):
return red('ERROR not found')

try:
self.project = Project.from_current_directory(self.cli_vars)
self.project = Project.from_project_root(self.project_dir, self.cli_vars)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this line is too long per our styleguide:

core/dbt/task/debug.py:129:80: E501 line too long (85 > 79 characters)

via: https://circleci.com/gh/fishtown-analytics/dbt/19769?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

Can you split this onto two lines?

Copy link
Contributor

@beckjake beckjake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Two tiny comments + the flake8 fix that drew noted and this will be great. Thanks for writing great tests!


@use_profile('postgres')
def test_postgres_not_found_project_dir(self):
self.use_default_project()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You shouldn't need to call use_default_project unless you do things with the project/profile as part of the test - setUp will call it (some old tests still have it, so I understand where this came from!)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you're right @beckjake . I did not realize that it is called in setUp method. I will take it into account for the next time, thanks!

splitout = self.capsys.readouterr().out.split('\n')
for line in splitout:
if line.strip().startswith('dbt_project.yml file'):
self.assertIn('ERROR invalid', line)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add a newline at the end of this file? looking at stuff in the terminal can get annoying without trailing newlines.

@franloza
Copy link
Contributor Author

franloza commented Jan 7, 2020

I have fixed style code style violations and removed self.use_default_project() from one test. Really happy to contribute, I think you all are doing a great job with tool 👍

Copy link
Contributor

@beckjake beckjake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, I've kicked off the tests for a final run. Thanks for your contribution @franloza!

@beckjake beckjake merged commit 1a72660 into dbt-labs:dev/barbara-gittings Jan 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants