Mp/reduce ci daily run intervals #392
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alexander-smolyakov
added a commit
to dbt-labs/dbt-snowflake-release-test
that referenced
this pull request
Jan 17, 2023
* Comment appropriate lines in `test.env.example` (dbt-labs#367) * Bump upper bound on `cryptography` to `<39.0.0` (dbt-labs#360) Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> * Consistent capitalization for `CONTRIBUTING.md` (dbt-labs#369) * [CT-1707] snowflake local testing fix (dbt-labs#372) * testing temp fix for local testing by lowering pin for snowlfake-python-connector * updating to main * rolling back snowflake-connector-python to 2.8.1 while we look into file lock issue * add changelog entry * add setup.py to integration test filter * Revert "add setup.py to integration test filter" This reverts commit 0bd88c0. Co-authored-by: Colin <colin.rogers@dbtlabs.com> * Ct 443/strip sensitive data from logs (dbt-labs#378) * Strip potentially sensitive data from error response if present When a merge statement fails on Snowflake with a duplicate row, Snowflake will return the data from the row that failed in the format: Duplicate row detected during DML action Row Values: [12345, "col_a_value", "col_b_value", "col_c_value", "etc"...] This can potentially result in sensitive data from the warehouse being written to stdout (often captured by orchestrators) and dbt.log Resolves dbt-labs#123 / CT-443 * Add changelog generated by changie * Generalize solution to all possible row value leaks. Add test. * Get rid of now useless if gaurd. * Compile regex. Co-authored-by: Dean Verhey <deanverhey@gmail.com> Co-authored-by: Anders <anders.swanson@dbtlabs.com> Co-authored-by: Mila Page <versusfacit@users.noreply.github.com> * Add 1.3.latest to the scheduled test runs. (dbt-labs#381) Co-authored-by: Mila Page <versusfacit@users.noreply.github.com> * Have snowflake build incremental model optimized _tmp tables with temporary views. (dbt-labs#379) * Add temporary key to temporary view creation macro. * add changelog * Rearrange things to work in a more general case * Add prefix to avoid name collisions and open door to future compatibility. * Cleanup methods using Jinja's extended parameter rules * Go back to a wrapped solution * Change boolean value name to reflect its type Co-authored-by: Mila Page <versusfacit@users.noreply.github.com> * fix release.yml pypi steps (dbt-labs#385) * rename exceptions (dbt-labs#384) * rename exceptions * changelog * point back to main * Update Under the Hood-20230110-084413.yaml * Bumping version to 1.5.0a1 and generate changelog (dbt-labs#391) * Bumping version to 1.5.0a1 and generate CHANGELOG * update changelog for 1.5.0a1 Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com> Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com> * Only run at 7:00 am est (before folks get into the office) (dbt-labs#390) * Only run at 7:00 am est (before folks get into the office) * Split jobs and adjust timestamps. * Trim comments. * Update .github/workflows/release-branch-tests.yml Add 1.4.latest to testing now that it exists. Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com> Co-authored-by: Mila Page <versusfacit@users.noreply.github.com> Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com> * Mp/reduce ci daily run intervals (dbt-labs#392) * Only run at 7:00 am est (before folks get into the office) * Split jobs and adjust timestamps. * Trim comments. * Need both workflows on each job. Co-authored-by: Mila Page <versusfacit@users.noreply.github.com> * Update build-test-package.yml * Revert "Update build-test-package.yml" This reverts commit a637482. * Create test_commit_to_protected_branches.yml * Update test_commit_to_protected_branches.yml * Update test_commit_to_protected_branches.yml * Revert "Update test_commit_to_protected_branches.yml" This reverts commit 6d7c129. * Revert "Update test_commit_to_protected_branches.yml" This reverts commit cb7abab. * Revert "Create test_commit_to_protected_branches.yml" This reverts commit b0969d0. * Revert "Revert "Update build-test-package.yml"" This reverts commit 7f571c8. * Revert "Update build-test-package.yml" This reverts commit a637482. Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com> Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> Co-authored-by: Matthew McKnight <91097623+McKnight-42@users.noreply.github.com> Co-authored-by: Colin <colin.rogers@dbtlabs.com> Co-authored-by: Mila Page <67295367+VersusFacit@users.noreply.github.com> Co-authored-by: Dean Verhey <deanverhey@gmail.com> Co-authored-by: Anders <anders.swanson@dbtlabs.com> Co-authored-by: Mila Page <versusfacit@users.noreply.github.com> Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
colin-rogers-dbt
pushed a commit
that referenced
this pull request
Jan 19, 2023
* Only run at 7:00 am est (before folks get into the office) * Split jobs and adjust timestamps. * Trim comments. * Need both workflows on each job. Co-authored-by: Mila Page <versusfacit@users.noreply.github.com>
colin-rogers-dbt
added a commit
that referenced
this pull request
Jan 19, 2023
* Mp/reduce ci daily run intervals (#392) * Only run at 7:00 am est (before folks get into the office) * Split jobs and adjust timestamps. * Trim comments. * Need both workflows on each job. Co-authored-by: Mila Page <versusfacit@users.noreply.github.com> * [CT-1507] create config for incremental models to use temp tables or views to fix regression in perf (#389) * init push for pr to work on regression in temp tables vs views for dbt-snowflake * adding additional table selection logic, functionality works locally based off snowflake query history * extend condition logic to smaller more readable elif's take into account some lost table options * add changie * breaking out language selection for more readability, reorganizing conditional flow * typo fix * spacing fix * change messaging for incremental model strategy selection * add direct reference to delete+insert * changing comment formatting and structure * Convert incremental on_schema_change tests (#398) * Convert incremental on_schema_change tests * Switch to dbt-core main * [CT-1507] part 2: Expand configuration to all strategies (#403) * init push for pr to work on regression in temp tables vs views for dbt-snowflake * adding additional table selection logic, functionality works locally based off snowflake query history * extend condition logic to smaller more readable elif's take into account some lost table options * add changie * breaking out language selection for more readability, reorganizing conditional flow * typo fix * spacing fix * change messaging for incremental model strategy selection * add direct reference to delete+insert * changing comment formatting and structure * expand tmp_relation_type to be configurable for all incremental options * re organize conditionl logic, add leading condtional for exceptions * typo * seperate if statements * fix exception blocking * seperate execptions condtional into two small condtionals to prevent a always case * change conditional logic based on language * change delete+insert exception logic, and messages for both exceptions * change condition for more jinja language * remove default value assigned to tmp_relation_type, change delete+insert excpetion to only kick off if view is assigned and not unique_key not none * reword python exception * minor comment change, revert some condtional logic (need to deal with none) value * re add assignment of config * comment changes * put back low level part about append strategy * minor change to workign for excpetion * add changelog entry * fix previous changelog entry * change change log entry, remove some config from integration test * remove unnded comma (#405) Co-authored-by: Mila Page <67295367+VersusFacit@users.noreply.github.com> Co-authored-by: Mila Page <versusfacit@users.noreply.github.com> Co-authored-by: Matthew McKnight <91097623+McKnight-42@users.noreply.github.com> Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Need to tweak jobs one more time. You need main and integration on both combinations of jobs.