Add more helpful error message for misconfiguration in profiles.yml #2627

kning · 2020-07-15T21:50:56Z

resolves #2569

Description

Adds an additional catch for if the supplied target returns no data and returns a more helpful error message.

Checklist

I have signed the CLA
I have run this code in development and it appears to resolve the stated issue
This PR includes tests, or tests are not required/relevant for this PR
I have updated the CHANGELOG.md and added information about my change to the "dbt next" section. <-- not sure if this change warrants a mention quite yet but let me know if you'd like me to add.

Further Validation

Additional screenshot of what the new terminal output looks like (tested against the example target referenced in the issue):

beckjake

Thanks for your contribution @kning!

I have a minor copy suggestion I'd like to get feedback on from Jeremy, and a suggestion on catching more bad profiles.

Can you also add an integration test? I think the appropriate place is in test/integration/049_debug_test/test_debug.py.

You can update the postgres profile to add a new entry with "none_target": None instead of an actual target definition, and then add a new test that runs debug --target none_target and make sure it says something reasonable.

Finally, please make a CHANGELOG.md update following the existing pattern, so we can give you credit in our next release! Add yourself to the contributors list for the upcoming release, and add a summary under Features to this PR and the issue. The name of this PR should be a fine summary.

beckjake · 2020-07-15T22:37:45Z

core/dbt/config/profile.py

@@ -197,6 +197,11 @@ def _get_profile_data(
                   .format(profile_name, target_name, outputs))
            raise DbtProfileError(msg, result_type='invalid_target')
        profile_data = outputs[target_name]
+
+        if profile_data is None:
+            msg = (f"{target_name} is misconfigured in your profiles.yml")


Suggested change

msg = (f"{target_name} is misconfigured in your profiles.yml")

msg = (

f"output '{target_name}' of profile '{profile_name}' is "

f"misconfigured in profiles.yml"

)

It'd be nice to point it at where the issue is a bit more clearly in large profiles, here's one way we could add the profile name in and show that we're talking about outputs: values.

@jtcohen6, do you have any other suggestions on the copy?

beckjake · 2020-07-15T22:59:27Z

core/dbt/config/profile.py

@@ -197,6 +197,11 @@ def _get_profile_data(
                   .format(profile_name, target_name, outputs))
            raise DbtProfileError(msg, result_type='invalid_target')
        profile_data = outputs[target_name]
+
+        if profile_data is None:


It'd be good to check if not isinstance(profile_data, dict) instead - if it's anything but a dict, we know it's misconfigured.

kning · 2020-07-16T15:34:32Z

thanks taking a look

kning · 2020-07-16T21:56:36Z

A question on the integration test @beckjake.

dbt debug actually outputs fine as long as there is one correctly configured target. Thus, adding a new target definition to the existing postgres profile doesn't actually trigger the test case.

I think I'd need to create a separate postgres profile with only an empty target. However, I'm not sure the best way to do this. The way the test is set up it looks like you can only have one postgres profile?

beckjake · 2020-07-16T22:01:27Z

dbt debug actually outputs fine as long as there is one correctly configured target. Thus, adding a new target definition to the existing postgres profile doesn't actually trigger the test case.

Really? I think it'd be sufficient to specify the bad target on the command line with --target. For example, in test_postgres_wronguser, we run self.run_dbt(['debug', '--target', 'wronguser']). You could do something similar with an empty target, right?

kning · 2020-07-17T15:37:35Z

ready for review now @beckjake

beckjake

This looks good to me! Thanks for your contribution to dbt, and thanks for the test!

I'll merge this once the test passes. Windows tests might fail because of a config issue, that's fine - this is not OS-specific behavior, and I have a fix for that in a different open PR.

beckjake · 2020-07-17T15:42:25Z

CHANGELOG.md

@@ -6,13 +6,15 @@
 ### Features
 - Added support for Snowflake query tags at the connection and model level ([#1030](https://github.com/fishtown-analytics/dbt/issues/1030), [#2555](https://github.com/fishtown-analytics/dbt/pull/2555/))
 - Added option to specify profile when connecting to Redshift via IAM ([#2437](https://github.com/fishtown-analytics/dbt/issues/2437), [#2581](https://github.com/fishtown-analytics/dbt/pull/2581))
+- Add more helpful error message for misconfiguration in profiles.yml ([#2627](https://github.com/fishtown-analytics/dbt/issues/2569))


Suggested change

- Add more helpful error message for misconfiguration in profiles.yml ([#2627](https://github.com/fishtown-analytics/dbt/issues/2569))

- Add more helpful error message for misconfiguration in profiles.yml ([#2569](https://github.com/fishtown-analytics/dbt/issues/2569), [#2627](https://github.com/fishtown-analytics/dbt/pull/2627))

beckjake · 2020-07-17T15:42:43Z

CHANGELOG.md


 ### Fixes
 - Adapter plugins can once again override plugins defined in core ([#2548](https://github.com/fishtown-analytics/dbt/issues/2548), [#2590](https://github.com/fishtown-analytics/dbt/pull/2590))

 Contributors:
 - [@brunomurino](https://github.com/brunomurino) ([#2437](https://github.com/fishtown-analytics/dbt/pull/2581))
 - [@DrMcTaco](https://github.com/DrMcTaco) ([#1030](https://github.com/fishtown-analytics/dbt/issues/1030)),[#2555](https://github.com/fishtown-analytics/dbt/pull/2555/))
+- [@kning](https://github.com/kning) ([#2627](https://github.com/fishtown-analytics/dbt/issues/2569))


Suggested change

- [@kning](https://github.com/kning) ([#2627](https://github.com/fishtown-analytics/dbt/issues/2569))

- [@kning](https://github.com/kning) ([#2627](https://github.com/fishtown-analytics/dbt/pull/2627))

beckjake · 2020-07-17T17:48:28Z

All looks great, this will release in 0.18.0 🎉

kning · 2020-07-17T17:53:12Z

thanks @beckjake !

Add more helpful error message for misconfiguration in profiles.yml

06a30f0

cla-bot bot added the cla:yes label Jul 15, 2020

beckjake suggested changes Jul 15, 2020

View reviewed changes

Address review

1ecd954

Add test

d7f3518

beckjake approved these changes Jul 17, 2020

View reviewed changes

Add PR link to changelog

d08a2b8

beckjake merged commit 7f0c1f8 into dbt-labs:dev/marian-anderson Jul 17, 2020

kning deleted the ISSUE-2569_bad_target branch July 17, 2020 17:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add more helpful error message for misconfiguration in profiles.yml #2627

Add more helpful error message for misconfiguration in profiles.yml #2627

kning commented Jul 15, 2020

beckjake left a comment

beckjake Jul 15, 2020

beckjake Jul 15, 2020

kning commented Jul 16, 2020

kning commented Jul 16, 2020

beckjake commented Jul 16, 2020

kning commented Jul 17, 2020

beckjake left a comment

beckjake Jul 17, 2020

beckjake Jul 17, 2020

beckjake commented Jul 17, 2020

kning commented Jul 17, 2020

-            msg = (f"{target_name} is misconfigured in your profiles.yml")
+            msg = (
+                f"output '{target_name}' of profile '{profile_name}' is "
+                f"misconfigured in profiles.yml"
+            )

	- Add more helpful error message for misconfiguration in profiles.yml ([#2627](https://github.com/fishtown-analytics/dbt/issues/2569))
	- Add more helpful error message for misconfiguration in profiles.yml ([#2569](https://github.com/fishtown-analytics/dbt/issues/2569), [#2627](https://github.com/fishtown-analytics/dbt/pull/2627))

	- [@kning](https://github.com/kning) ([#2627](https://github.com/fishtown-analytics/dbt/issues/2569))
	- [@kning](https://github.com/kning) ([#2627](https://github.com/fishtown-analytics/dbt/pull/2627))

Add more helpful error message for misconfiguration in profiles.yml #2627

Add more helpful error message for misconfiguration in profiles.yml #2627

Conversation

kning commented Jul 15, 2020

Description

Checklist

Further Validation

beckjake left a comment

Choose a reason for hiding this comment

beckjake Jul 15, 2020

Choose a reason for hiding this comment

beckjake Jul 15, 2020

Choose a reason for hiding this comment

kning commented Jul 16, 2020

kning commented Jul 16, 2020

beckjake commented Jul 16, 2020

kning commented Jul 17, 2020

beckjake left a comment

Choose a reason for hiding this comment

beckjake Jul 17, 2020

Choose a reason for hiding this comment

beckjake Jul 17, 2020

Choose a reason for hiding this comment

beckjake commented Jul 17, 2020

kning commented Jul 17, 2020