Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-2254] dbt debug for connection testing #7104

Closed
Tracked by #6356
jtcohen6 opened this issue Mar 2, 2023 · 3 comments · Fixed by #7741
Closed
Tracked by #6356

[CT-2254] dbt debug for connection testing #7104

jtcohen6 opened this issue Mar 2, 2023 · 3 comments · Fixed by #7741
Assignees
Labels
python_api Issues related to dbtRunner Python entry point Refinement Maintainer input needed

Comments

@jtcohen6
Copy link
Contributor

jtcohen6 commented Mar 2, 2023

Much older issue in the same vein: #3735

It should be possible to use dbt debug for connection testing. Currently, debug also includes checks for:

  1. Valid profiles.yml file
  2. Valid dbt_project.yml file
  3. git as an installed dependency (for deps)
  4. dbt can connect and submit a "debug" query to the selected profile

Of those checks, we really only need the last one for connection testing. Ideally, we'd be able to skip the other checks (perhaps by passing a flag?) and return success/error (programmatically & as CLI exit code) on the basis of the "debug" query only.

If we had to hack this today, it would be sorta possible by examining the structured events (DebugCmdOut) returned from the debug command, but this hardly feels like a stable interface.

Why dbt debug?

Why not just an interactive query (new preview command) with the query hard-coded to select 1 as id? Reasons to prefer:

  • leveraging an adapter method, debug_query
  • it's part of dbt-core's top-level API, and clearly the intent of debug

Questions

  • How should the profile be provided? In profiles.yml, or could it be provided as a Python data structure?
  • Could we avoid the need to have a dbt_project.yml? Just use a "dummy"/empty dbt_project.yml?

Acceptance criteria

  • Given the contents of a profile only, validate that profile via dbt debug, and return success/error
@jtcohen6 jtcohen6 added python_api Issues related to dbtRunner Python entry point Team:Execution Refinement Maintainer input needed labels Mar 2, 2023
@github-actions github-actions bot changed the title dbt debug for connection testing [CT-2254] dbt debug for connection testing Mar 2, 2023
@nssalian
Copy link
Contributor

nssalian commented Mar 2, 2023

An alternative suggestion would be to add a command
dbt test_connection that would use the selected profile. I can open a separate issue to surface this alternative way.

@VersusFacit
Copy link
Contributor

VersusFacit commented Apr 21, 2023

image

Just going by what we see here, I would say we communicate that dbt debug exists to support the validation of a dbt environment. Having a flag that skipped those validation steps would mirror the `git commit --no-verify` workflow. Basically, just validate that the connection makes sense. I worry that if we create a new command which under the hood is leveraging the same code, we are inviting unclear interfaces at the executable level

image

We already have a flag that serves as an example of what additional flag we might add to accomplish the desired behavior outlined in the issue

@VersusFacit
Copy link
Contributor

With the merge of #7741 , the workflow for this in 1.6 and on will be dbt debug --connection.

Other flag skips/features may be added. See this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
python_api Issues related to dbtRunner Python entry point Refinement Maintainer input needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants