Add tox support for testing redshift, bigquery and snowflake #320
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
emmyoop
commented
Oct 18, 2024
Comment on lines
51
to
52
| # TODO: below value - does it need to be secret or could we hard code it in tox and here? | ||
| REDSHIFT_SPECTRUM_IAM_ROLE: ${{ secrets.REDSHIFT_SPECTRUM_IAM_ROLE }} |
There was a problem hiding this comment.
Can we hardcode this? Is it actually a secret?
emmyoop
changed the title
emmyoop
force-pushed
the
er/support-tox
branch
from
f81c935
to
9fbfa89
Compare
|
To get this working I will need the following added as variables instead of secrets:
The following need to be added as secrets
|
dataders
reviewed
Oct 25, 2024
Co-authored-by: Anders <anders.swanson@dbtlabs.com>
dataders
reviewed
Nov 1, 2024
Comment on lines
+24
to
+37
| REDSHIFT_HOST: ${{ vars.REDSHIFT_HOST }} | ||
| REDSHIFT_USER: ${{ vars.REDSHIFT_USER }} | ||
| REDSHIFT_PORT: ${{ vars.REDSHIFT_PORT }} | ||
| REDSHIFT_DATABASE: ${{ vars.REDSHIFT_DATABASE }} | ||
| REDSHIFT_SCHEMA: "integration_tests_redshift_${{ github.run_number }}" | ||
| # snowflake | ||
| SNOWFLAKE_USER: ${{ vars.SNOWFLAKE_USER }} | ||
| SNOWFLAKE_WAREHOUSE: ${{ vars.SNOWFLAKE_WAREHOUSE }} | ||
| SNOWFLAKE_ROLE: ${{ vars.SNOWFLAKE_ROLE }} | ||
| SNOWFLAKE_DATABASE: ${{ vars.SNOWFLAKE_DATABASE }} | ||
| SNOWFLAKE_SCHEMA: "integration_tests_snowflake_${{ github.run_number }}" | ||
| # bigquery | ||
| BIGQUERY_PROJECT: ${{ vars.BIGQUERY_PROJECT }} | ||
| BIGQUERY_SCHEMA: "integration_tests_bigquery_${{ github.run_number }}" |
There was a problem hiding this comment.
@emmyoop is my understanding these envvars (excluding *_SCHEMA) need to be registered as variables in Github? If, so are they set at the repo-, environment-, or organization-level?
dataders
force-pushed
the
er/support-tox
branch
from
a25f8f1
to
e06d871
Compare
dataders
force-pushed
the
er/support-tox
branch
from
e06d871
to
495c454
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #316
Description & motivation
This PR make a slight update to the current testing layout to allow dbt Labs to start testing this package as part of our own
release process.
This adds support for testing with snowflake, bigquery and redshift.
To add additional adapter testing support in the future, you will just need to modify
I've left your current testing as is to allow you to decide what to do with it.
Why Do We Need These Changes
As explained in #316, by setting up tox as a standard entry-point, the Core development team at dbt Labs can include this package's integration tests (along with other popular dbt packages) as part of our continuous development & delivery of dbt.
Checklist