Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BigQuery option to define a maximum query cost limit #2427

Merged

Conversation

haukeduden
Copy link
Contributor

resolves #2346

Description

This adds the option maximum_bytes_billed to BigQuery profiles. When this is set and not 0 then the corresponding query job option will be set. This is analogous to specifying bq query --maximum_bytes_billed on the commandline.

Checklist

  • I have signed the CLA
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • I have updated the CHANGELOG.md and added information about my change to the "dbt next" section.

@cla-bot
Copy link

cla-bot bot commented May 11, 2020

Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA.

In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, don't hesitate to ping @drewbanin.

CLA has not been signed by users: @haukeduden

@haukeduden
Copy link
Contributor Author

@drewbanin I signed the CLA, specifying my user name "haukeduden". I got the message "Your response has been recorded.", so I assume that it has worked. But the CLA verification still seems to fail.

@beckjake
Copy link
Contributor

@cla-bot check

@cla-bot cla-bot bot added the cla:yes label May 12, 2020
@cla-bot
Copy link

cla-bot bot commented May 12, 2020

The cla-bot has been summoned, and re-checked this pull request!

@beckjake
Copy link
Contributor

@haukeduden our cla-bot stuff requires manual intervention, unfortunately. This looks good to me, though flake8 seems upset about some things based on the failing tests. Please also add yourself to the changelog under "contributors" (just make a new section for the next release if there isn't one) so we can give you credit in the release notes!

@haukeduden
Copy link
Contributor Author

haukeduden commented May 12, 2020

@beckjake I added the entry to contributors, as you asked, and fixed a whitespace issue that flake8 complained about. Sorry for not catching this before submitting. flake8 still fails, though -- but it complains about issues in files that have not been updated (E741 ambiguous variable name 'l'in core/dbt/utils.py, core/dbt/rpc/task_handler.py and core/dbt/rpc/builtins.py).
Apparently this issue was already present in the base branch when I branched off.
It seems to me that this should be fixed in a separate PR, since it has nothing to do with the new BigQuery feature. Do you agree?

@beckjake
Copy link
Contributor

@haukeduden totally! I think if you pull in dev/octavius-catto again, drew merged a change that will suppress that error.

@haukeduden
Copy link
Contributor Author

@beckjake I pulled the latest changes and that fixed the problem 👍. The tests pass now.

Copy link
Contributor

@beckjake beckjake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thanks for contributing! I'll kick off the rest of the integration suite now

@drewbanin drewbanin merged commit df5b97c into dbt-labs:dev/octavius-catto May 13, 2020
@drewbanin
Copy link
Contributor

Merged! Thanks for your contribution @haukeduden -- this will ship in 0.17.0 :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add maximum_bytes_billed option for BigQuery
3 participants