-
Notifications
You must be signed in to change notification settings - Fork 180
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Release 1.7 #1234
Merged
Merged
Release 1.7 #1234
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for sunny-pastelito-5ecb04 canceled.
|
pankajkoti
approved these changes
Oct 2, 2024
pankajkoti
force-pushed
the
release-1.7
branch
from
October 3, 2024 12:46
5ab7896
to
710e32a
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1234 +/- ##
=======================================
Coverage 95.73% 95.73%
=======================================
Files 67 67
Lines 3965 3965
=======================================
Hits 3796 3796
Misses 169 169 ☔ View full report in Codecov by Sentry. |
pankajastro
approved these changes
Oct 4, 2024
pankajastro
reviewed
Oct 4, 2024
Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>
Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>
dosubot
bot
added
size:M
This PR changes 30-99 lines, ignoring generated files.
area:ci
Related to CI, Github Actions, or other continuous integration tools
area:datasets
Related to the Airflow datasets feature/module
area:docs
Relating to documentation, changes, fixes, improvement
area:execution
Related to the execution environment/mode, like Docker, Kubernetes, Local, VirtualEnv, etc
labels
Oct 4, 2024
New Features * Introduction of experimental support to run dbt BQ models using Airflow deferrable operators by @pankajkoti @pankajastro @tatiana in #1224 #1230. This is a first step in this journey and we would really appreciate feedback from the community. For more information, check the documentation: https://astronomer.github.io/astronomer-cosmos/getting_started/execution-modes.html#airflow-async-experimental This work has been inspired by the talk "Airflow at Monzo: Evolving our data platform as the bank scales" by @jonathanrainer @ed-sparkes given at Airflow Summit 2023: https://airflowsummit.org/sessions/2023/airflow-at-monzo-evolving-our-data-platform-as-the-bank-scales/. * Support using ``DatasetAlias`` and fix orphaning unreferenced dataset by @tatiana in #1217 #1240 Documentation: https://astronomer.github.io/astronomer-cosmos/configuration/scheduling.html#data-aware-scheduling * Add GCP_CLOUD_RUN_JOB execution mode by @ags-de #1153 Learn more about it: https://astronomer.github.io/astronomer-cosmos/getting_started/gcp-cloud-run-job.html Enhancements * Create single virtualenv when ``DbtVirtualenvBaseOperator`` has ``virtualenv_dir=None`` and ``is_virtualenv_dir_temporary=True`` by @kesompochy in #1200 * Consistently handle build and imports in ``cosmos/__init__.py`` by @tatiana in #1215 * Add enum constants to init for direct import by @fabiomx in #1184 Bug fixes * URL encode dataset names to support multibyte characters by @t0momi219 in #1198 * Fix invalid argument (``full_refresh``) passed to DbtTestAwsEksOperator (and others) by @johnhoran in #1175 * Fix ``printer_width`` arg type in ``DbtProfileConfigVars`` by @jessicaschueler in #1191 * Fix task owner fallback by @jmaicher in #1195 Docs * Add scarf to readme and docs for website analytics by @cmarteepants in #1221 * Add ``virtualenv_dir`` param to ``ExecutionConfig`` docs by @pankajkoti in #1173 * Give credits to @LennartKloppenburg in CHANGELOG.rst by @tatiana #1174 * Refactor docs for async mode execution by @pankajkoti in #1241 Others * Remove PR branch added for testing a change in CI in #1224 by @pankajkoti in #1233 * Fix CI wrt broken coverage upload artifact @pankajkoti in #1210 * Fix CI issues - Upgrade actions/upload-artifact & actions/download-artifact to v4 and set min version for packaging by @pankajkoti in #1208 * Resolve CI failures for Apache Airflow 2.7 jobs by @pankajkoti in #1182 * CI: Update GCP manifest file path based on new secret update by @pankajkoti in #1237 * Pre-commit hook updates in #1176 #1186, #1186, #1201, #1219, #1231
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
area:ci
Related to CI, Github Actions, or other continuous integration tools
area:datasets
Related to the Airflow datasets feature/module
area:docs
Relating to documentation, changes, fixes, improvement
area:execution
Related to the execution environment/mode, like Docker, Kubernetes, Local, VirtualEnv, etc
lgtm
This PR has been approved by a maintainer
size:M
This PR changes 30-99 lines, ignoring generated files.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New Features
Support using
DatasetAlias
and fix orphaning unreferenced dataset by @tatiana in Support usingDatasetAlias
and fix orphaning unreferenced dataset #1217 Fix Dataset and DatasetAlias scheduling in Airflow 2.10 #1240Documentation: https://astronomer.github.io/astronomer-cosmos/configuration/scheduling.html#data-aware-scheduling
Add GCP_CLOUD_RUN_JOB execution mode by @ags-de Add GCP_CLOUD_RUN_JOB execution mode #1153
Learn more about it: https://astronomer.github.io/astronomer-cosmos/getting_started/gcp-cloud-run-job.html
Introduction of experimental support to run dbt BQ models using Airflow deferrable operators by @pankajkoti @pankajastro @tatiana in Introduce ground work for
ExecutionMode.AIRFLOW_ASYNC
#1224 Experimental BQ support to run dbt models withExecutionMode.AIRFLOW_ASYNC
#1230.This is the first step in the journey of running dbt resources with native Airflow, and we would appreciate feedback from the community.
For more information, check the documentation: https://astronomer.github.io/astronomer-cosmos/getting_started/execution-modes.html#airflow-async-experimental
This work has been inspired by the talk "Airflow at Monzo: Evolving our data platform as the bank scales" by
@jonathanrainer @ed-sparkes given at Airflow Summit 2023: https://airflowsummit.org/sessions/2023/airflow-at-monzo-evolving-our-data-platform-as-the-bank-scales/.
Enhancements
DbtVirtualenvBaseOperator
hasvirtualenv_dir=None
andis_virtualenv_dir_temporary=True
by @kesompochy in Create single virtualenv whenDbtVirtualenvBaseOperator
hasvirtualenv_dir=None
andis_virtualenv_dir_temporary=True
#1200cosmos/__init__.py
by @tatiana in Consistently handle build and imports in cosmos/__init__.py #1215Bug fixes
full_refresh
) passed to DbtTestAwsEksOperator (and others) by @johnhoran in Fix invalid argument (full_refresh
) passed to DbtTestAwsEksOperator (and others) #1175printer_width
arg type inDbtProfileConfigVars
by @jessicaschueler in Update type for printer_width param in class DbtProfileConfigVars to int #1191Docs
virtualenv_dir
param toExecutionConfig
docs by @pankajkoti in Add 'virtualenv_dir' param to 'Execution Config' docs #1173Others
ExecutionMode.AIRFLOW_ASYNC
#1224 by @pankajkoti in Remove PR branch added for testing a change in CI in #1224 #1233