Sync v3-0-stable with 3.0.6rc1 changes
#54808
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…etup (apache#54521) (apache#54530) Co-authored-by: Vishal <69051998+dataengineervishal@users.noreply.github.com>
(cherry picked from commit c00f1d7) Co-authored-by: Kyungjun Lee <kyungjunlee.me@gmail.com>
(cherry picked from commit ed549df) Co-authored-by: GPK <gopidesupavan@gmail.com>
…ty (apache#54536) (apache#54549) Might help to know that a triggerer has reached the maximum number of triggers it can run (cherry picked from commit 86063eb) Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>
The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added
(cherry picked from commit c107cd6)
…apache#54590) After switching to prek in apache#54258 there are still few small things thet were only discovered in our canary builds: * installation of prek hooks does not necessarily create uv cache * autoupgrade is not yet implemented in prek (cherry picked from commit 703af5e)
…4599) (apache#54601) The 0.0.28 prek has an important fix - avoids craching when you run `git commit -a` j178/prek#385 - we bump the minimum_version to 0.0.28 as well as upgrading the CI and Breeze version to 0.0.28 - also, we change the strategy of failing upgrade check in CI - we do not fail the upgrade check when there is a new upgrade of `prek` - similar like with `uv` we expect new versions to be released quickly over the `teething` period. (cherry picked from commit 3aa2448)
…pache#54608) * we do not have to check Python versions below 3.9 - that significantly speeds up the check * we run `uv sync --resolution highest` in dev/breeze when there are some changes - that will make sure that the latest version of dependencies in breeze are captured in the uv.lock file (cherry picked from commit c6c6d75) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…54477) (apache#54483) A combination of bundle path and dag.relative_fileloc more correctly determines the right location of a dag and not dag.fileloc. This PR excludes fileloc from determining the creation of a new dag version. (cherry picked from commit 9efadf8) Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>
(cherry picked from commit 5c8c135)
This upgrades prek to **just released** 0.0.29 and removes the workaround implemented in apache#54613 to add build-essential to the image that is used to build distributions - because prek 0.0.29 comes with all the necessary binary wheels/platforms that remove the need to build in in our Linux ARM image (issue j178/prek#451 has been fixed). (cherry picked from commit a4fa218)
…pache#54594) (apache#54604) Reset queued_by_job_id when rescheduling stuck tasks to ensure timeout monitoring works correctly across multiple DAG runs. (cherry picked from commit bec069c) Co-authored-by: Dheeraj Turaga <dheerajturaga@gmail.com>
(Cherry-picked from apache@5b8f631 )
…racing is disabled (apache#54591) (apache#54626) (cherry picked from commit 4550cd2) Co-authored-by: Shlomit-B <76184325+Shlomit-B@users.noreply.github.com>
We are not using black-related functionality in prek hooks any more - the only remaining hook is 'blacken-docs` - but there is no need to synchronize and upgrade black version any more for it. Upgraded to latest black version for blacken-docs manually. (cherry picked from commit 4e6fa29) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…sk is run with user impersonation (apache#54649) (apache#54672) (cherry picked from commit 3c4cdf4) closes: apache#53967 Co-authored-by: adrian-edbert <132150602+adrian-edbert@users.noreply.github.com>
(cherry picked from commit 55abbeb) Co-authored-by: Pratiksha <128999446+Prab-27@users.noreply.github.com>
) (apache#54689) (cherry picked from commit 82dbdfd) Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>
…pervisor (apache#54679) (apache#54720) Using connections stored in the Airflow Metadata for Remote logging was partially fixed in 3.0.4, but the connection was only set/put-in-scope when the logger was created before task startup. However for blob stores (i.e. S3 or WASB) that is a no-op, and we needed the connection when we try to upload too. (cherry picked from commit b340e8c) Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
…apache#54725) (cherry picked from commit bd5f86a) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…he#54686) * fix(logging): fix module loading by using `import_module()` - Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports * test(logging): add tests for logging config module path handling - Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures * fix(logging): fix module loading by using `import_module()` - Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports * test(logging): add tests for logging config module path handling - Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures * fix(logging): fix module loading by using `import_module()` - Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports * test(logging): add tests for logging config module path handling - Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures * refactor(logging): move `import_module` to the top level * test(logging): optimize test constants to reduce code duplication Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE` * test(logging): use shared helper method for logging config validation Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests. * refactor(logging_config): simplify, improve test config vars - Remove redundant `SETTINGS_FILE_NESTED_MODULE` - Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS` * refactor(logging): imporve & simplify logging testing file 1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var 2. add return typo with `_verify_basic_logging_config` 3. add unittest.mock `call` (cherry picked from commit 462ce9a) Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
…pache#54729) (cherry picked from commit ef911fa)
…low/api_fastapi/auth/managers/simple/ui in the core-ui-package-updates group across 1 directory (apache#54733)
… deps (apache#54726) (apache#54734) After j178/prek#35 has been completed and released in `prek` 0.1.1 we can bring back the functionality of flagging when our prek hooks need to be updated. This PR brings minimum version of prek to 0.1.1 as well as upgrades all other important versions up, upgrades breeze uv.lock and brings back the steps in upgrade check to check for auto-upgradeable hooks. (cherry picked from commit f187852)
kaxil
requested review from
XD-DENG,
bbovenzi,
bugraoz93,
ephraimbuddy,
o-nikolas,
rawwar,
ryanahamilton and
shubhamraj-git
as code owners
jedcunningham
approved these changes
Aug 22, 2025
Contributor
|
We also have #54648 |
eladkal
approved these changes
Aug 22, 2025
amoghrajesh
approved these changes
Aug 22, 2025
Contributor
amoghrajesh
left a comment
amoghrajesh
left a comment
There was a problem hiding this comment.
LGTM +1
Can we pull in the fix for map_index_template: #54648?
ashb
reviewed
Aug 22, 2025
potiuk
approved these changes
Aug 22, 2025
Member
Author
…ng an error (apache#54769) (apache#54780) * Ensure that Connection extra can get masked without causing an error Fixes apache#54768 This was caused by pydantic#9541 and improper testing on my part. Sorry folks. Thsi happens because `Iterable` is too open-ended a type Yes, this should absolutely have unit tests to go with the fix, but a fix is better than nothing, and I'm about to leave on a camping holiday. * fixup! Ensure that Connection extra can get masked without causing an error * fixup! fixup! Ensure that Connection extra can get masked without causing an error * fixup! fixup! fixup! Ensure that Connection extra can get masked without causing an error --------- (cherry picked from commit 5aec867) Co-authored-by: Ash Berlin-Taylor <ash@apache.org> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…url (apache#54218) (apache#54760) (cherry picked from commit 0204a89) Co-authored-by: kandharvishnu <46064835+kandharvishnu@users.noreply.github.com> Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> Co-authored-by: Pierre Jeambrun <pierrejbrun@gmail.com>
…4793) (apache#54802) We've had this for a while now, time to make it official. I've also reworded the docs section a bit as well. (cherry picked from commit fdef01c) Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>
…ing to default data (apache#54495) (apache#54639) (cherry picked from commit 723297f) Co-authored-by: vikrantkumar-max <vikrant.kumar@zemosolabs.com>
…es (apache#54799) (apache#54806) (cherry picked from commit 2b10f1a) Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
"Prek" didn't exist then :) (cherry picked from commit 3c68c50)
(cherry picked from commit 600716f)
hussein-awala
approved these changes
Aug 22, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
12 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR Syncs
v3-0-stablewithv3-0-testto release 3.0.6.Release notes and version bumps added in
Changes between Task SDK 1.0.5 and 1.0.6
KRB5CCNAMEenv) when a task is run with user impersonation (#54649) #54672)