Status of testing of Apache Airflow 3.0.2rc2

[kaxil]

Body

We are kindly requesting that contributors to Apache Airflow RC 3.0.2rc1 help test the RC.

Please let us know by commenting if the issue is addressed in the latest RC.

• Bring back support for local dagbag based list/list-import-errors (#49380): @dheerajturaga
  Linked issues:
  • Dags list/list-import-errors cli commands broken and don't track local DagBag (#49330)
• Rename operation IDs for task instance endpoints to include map index… (#49608): @abhishekbhakat
  Linked issues:
  • Unique operationId in openapi (#49607)
• Fix airflow tasks clear command (#49631): @vatsrahul1001
• Add "disappearing DAGs" section on FAQ doc (#49987): @amoghrajesh @rtrindvg
• Use latest bundle version when clearing / re-running dag (#50040): @dstandish @uranusjr
• Implement slice on LazyXComSequence (#50117): @uranusjr
• Add Apache Airflow setup instructions for Apple Silicon (#50179): @praneeth552
• Always Show Trendcount in Dag Overview (#50183): @dominic-lcw
• [v3-0-test] Remove experimental ARM support from container images (#50261) (#50263): @potiuk @github-actions[bot]
• Fix next asset schedule and dag card UX (#50271): @bbovenzi
• Add basic json check to variable value (#50277): @bbovenzi
• [v3-0-test] Fix publish docs runs-on config (#50280) (#50283): @gopidesupavan
• [v3-0-test] Fixing indent while setting finalize tests action (#50289) (#50291): @github-actions[bot] @amoghrajesh
• [v3-0-test] Updating operator extra links doc for AF3 (#50197) (#50293): @github-actions[bot] @amoghrajesh
• [v3-0-test] Bump minor and patch package json dependencies (#50282) (#50298): @bbovenzi @github-actions[bot]
• Port dag.test to Task SDK (#50300): @kaxil
• [v3-0-test] Add a link to the Airflow logo in Nav (#50304) (#50307): @github-actions[bot] @scalewithlee
• [v3-0-test] Bump Pyspark to even higher version (#50308) (#50311): @potiuk @github-actions[bot]
• EmptyOperator raises InvalidURL for non-existing logs (#50325): @zachliu
• [v3-0-test] Split release image into per-python independent matrix of workflows (#50320) (#50331): @potiuk @github-actions[bot]
• [v3-0-test] Move map index column (#50302) (#50347): @bbovenzi @github-actions[bot]
• [v3-0-test] Raise exception if downgrade can't proceed due to no ab_user table (#50343) (#50351): @ephraimbuddy @github-actions[bot]
• Docs: Add Airflow Version in Page Title (#50358): @kaxil
• [v3-0-test] Fixes DagProcessor stats log | process durations now use time.monotonic() (#50316) (#50370): @ayush3singh @github-actions[bot]
• Replace DagBag from global app state to with FastAPI dependency (#50372): @kaxil
• [v3-0-test] Fix callbacks docs (#50377) (#50381): @github-actions[bot] @kaxil
• [v3-0-test] Adding backcompat shim for BaseNotifier (#50340) (#50384): @github-actions[bot] @amoghrajesh
• CLI: Exclude example dags when a bundle is passed (#50401): @kaxil
• Fix typo in fundamentals.rst (#50402): @AzisK
• [v3-0-test] Add user-defined macros warning to Plugins doc page (#50357) (#50407): @zach-overflow @github-actions[bot]
• UI: Implement navigation on bar click (#50416): @dominic-lcw @RoyLee1224
  Linked issues:
  • Detecting long-running tasks (#50184)
• Fix bug with in-process request handling for dag.test (#50419): @kaxil
• Support running airflow dags test from local files (#50420): @kaxil
• Ensure singleton use of DagBag across all API requests (#50447): @kaxil
• Fixes border overlaps in events page (#50453): @shubhamraj-git @aritra24
• [v3-0-test] Limit click to < 8.2.0 for breeze (#50452) (#50460): @potiuk
• Fix timezone selection and dashboard layout (#50463): @guan404ming
• [v3-0-test] Install sphinx-airflow-theme package from GitHub release … (#50465): @potiuk
• [v3-0-test] Fix link to container images in README (#50484) (#50486): @potiuk @github-actions[bot]
• [v3-0-test] Actually, that should be linked to contributing guide (#50488) (#50489): @potiuk @github-actions[bot]
• [v3-0-test] Fix index to sort by last parsing duration (#50388) (#50493): @tirkarthi @github-actions[bot]
• [v3-0-test] Don't auto activate assets in activation tests (#50478) (#50494): @github-actions[bot] @uranusjr
• [v3-0-test] Better structure of extras documentation (#50495) (#50508): @potiuk @github-actions[bot]
• [v3-0-test] Add bundle root to sys.path in dag processor (#50385) (#50509): @github-actions[bot] @uranusjr
• More diagnostics and more logical name for cleaning up the runner (#50540): @potiuk
• [v3-0-test] Move access_denied_message webserver config to fab (#50208) (#50544): @sunank200 @pierrejeambrun
• Fix main - Generate FAB assets (#50546): @vincbeck
• Clean-up /mnt disk in case it is not empty (#50549): @potiuk
• [v3-0-test] Fix memory leak in dag-processor (#50558) (#50559): @github-actions[bot] @kaxil
• Another place where we double clean the /mnt drive (#50565): @potiuk
• Apply dag_id filter when updating backfill details in dag_run (#50577): @vatsrahul1001
• [v3-0-test] Fix default_args application in operator partial (#50525) (#50589): @github-actions[bot] @uranusjr
• [v3-0-test] Fix patch_task_instance endpoint (#50550) (#50591): @pierrejeambrun
• [v3-0-test] Do not flood worker logs with secrets backend loading logs (#50581) (#50596): @github-actions[bot] @amoghrajesh
• [v3-0-test] Cadwyn migration for backcompat of /run endpoint (#50528) (#50602): @github-actions[bot] @amoghrajesh
• [v3-0-test] feat: Add owner_links field to DAGDetailsResponse (#50557) (#50604): @choo121600 @pierrejeambrun
• [v3-0-test] Move webserver expose_hostname config to fab (#50269) (#50605): @pierrejeambrun
• Backport 3e03435 v3 0 test (#50608): @potiuk
• [v3-0-test] Pre-release distributions documentation links use `staged… (#50609): @potiuk
• [v3-0-test] Add --owerwrite when publishing documentation to S3 (#50543) (#50611): @potiuk
• [v3-0-test] Move preparing breeze/cleaning before cloning airflow-sit… (#50612): @potiuk
• [v3-0-test] Remove special tests from finalize's dependency (#49972) (#50613): @potiuk
• [v3-0-test] Skip provider tests when tests are skipped in general (#4… (#50614): @potiuk
• [v3-0-test] Fix basic-tests duplication in dependends in ci.amd (#50615): @potiuk
• [v3-0-test] Move prepare-breeze to the begining of job in publish-docs-to-s3 (#50606) (#50616): @potiuk @github-actions[bot]
• [v3-0-test] Fix permission check on the ui config endpoint (#50564) (#50619): @jedcunningham @github-actions[bot]
• [v3-0-test] Upgrade to latest versions of installers and pre-commit (#50630) (#50633): @potiuk @github-actions[bot]
• [v3-0-test] Update upgrading_to_airflow3.rst (#50632) (#50635): @github-actions[bot] @parksunah
• [v3-0-test] Update default sort for connections and dags (#50600) (#50652): @guan404ming @github-actions[bot]
• [v3-0-test] Fix Pydantic ForwardRef error by reordering discriminated union definitions (#50688) (#50691): @github-actions[bot] @kaxil
• [v3-0-test] Fix XCom deserialization for mapped tasks with custom backend (#50687) (#50697): @github-actions[bot] @amoghrajesh
• Fix Airflow Python client generation for 3.0.0 (#50704): @kaxil
• [v3-0-test] Use GitHub API to download constraint files and avoid rate limiting (#50716) (#50721): @github-actions[bot] @amoghrajesh
• [v3-0-test] Fixing fab assets generation (#50664) (#50723): @potiuk @vincbeck
• [v3-0-test] Better description why excluding distributions for PyPI constraints (#50680) (#50724): @potiuk @github-actions[bot]
• [v3-0-test] Use Github API to fetch K8S schemas to fix rate limiting (#50729) (#50731): @github-actions[bot] @amoghrajesh
• Increase max zoom (#50772): @bbovenzi
• Add back invalid inlet and outlet check before running tasks (#50773): @Lee-W
• [v3-0-test] Fix SQLite Lock error in TestTaskInstancesLog setup (#50763) (#50782): @jason810496 @github-actions[bot]
• Add dates for Limited Maintenance & EOL for Airflow 2.x (#50794): @kaxil
• [v3-0-test] Update chart dependencies to fix CI on main (#50748) (#50797): @potiuk @amoghrajesh
• [v3-0-test] Fix selective checks failing on breeze dependabot upgrade… (#50798): @potiuk
• Fix a docstring for airflow.sdk.execution_time.superviseor.supervise (#50805): @XD-DENG
• [v3-0-test] Support macros defined via plugins in Airflow 3 (#50642) (#50808): @amoghrajesh
• [v3-0-test] Correctly render dag tags when there are MAX_TAGS + 1 tags (#50669) (#50809): @amoghrajesh
• [v3-0-test] make dag_run conf nullable in Details page (#50719) (#50817): @ayush3singh @github-actions[bot]
• [v3-0-test] Remove doc warning for macros defined via plugins (#50810) (#50819): @github-actions[bot] @amoghrajesh
• Port task.test to Task SDK (#50827): @kaxil
• Update docs with Airflow 2.11.0 info (#50832): @kaxil @vatsrahul1001
• [v3-0-test] Fix 404 log error for non-latest task tries in multi-host worker environments (#50833): @jason810496 @oboki
  Linked issues:
  • fix: resolve 404 log error for non-latest task tries in multi-host worker environments (#50175)
• [v3-0-test] Fetch only the most recent dagrun value used in header. (#50767) (#50835): @tirkarthi @github-actions[bot]
• [v3-0-test] Generated yaml files should not have licence added (#50830) (#50836): @potiuk @github-actions[bot]
• [v3-0-test] Remove unecessary breaking flag in config command (#50781) (#50842): @pierrejeambrun
• [v3-0-test] Move webserver config to fab provider (#50774) (#50848): @pierrejeambrun
• [v3-0-test] Fix dropdowns in accordion and error text colors (#50845) (#50856): @bbovenzi @github-actions[bot]
• [v3-0-test] Update useTableURLState hook for sticky table sort (#50720) (#50858): @guan404ming @github-actions[bot]
• Further prune release notes (#50860): @kaxil
• [v3-0-test] API handle slashes in variable keys (#50841) (#50874): @pierrejeambrun @github-actions[bot]
• [v3-0-test] Fix reading number of dags from direct system providers path (#50843) (#50877): @potiuk @github-actions[bot]
  Linked issues:
  • Add back missing [sources] link in generated documentation's includes (#49978)
• [v3-0-test] Fix fab assets (#50878): @pierrejeambrun
• [v3-0-test] Do not run FAB pre-commits in release branches (#50875) (#50886): @potiuk
• [v3-0-test] Deserialize should work while retrieving variables with secrets backend (#50880) (#50889): @github-actions[bot] @amoghrajesh
• [v3-0-test] Move secret_key config to api section (#50839) (#50894): @pierrejeambrun
• [v3-0-test] Always create serdag in dagmaker fixture (#50359) (#50900): @dstandish @github-actions[bot]
• [v3-0-test] Stabilize FAB asset compilation (#50829) (#50901): @potiuk
• [v3-0-test] Bugfix: Logical date isn't populated in Context vars: (#50898) (#50903): @github-actions[bot] @kaxil
• [v3-0-test] Update config endpoint to use the get interface (#50902) (#50910): @pierrejeambrun
• [v3-0-test] Make test_dagbag test work locally, not only in Breeze (#50916) (#50924): @potiuk @github-actions[bot]
• [v3-0-test] Mask variable values in task logs only if the variable key is sensitive (#50775) (#50927): @github-actions[bot] @amoghrajesh
• [v3-0-test] Move enable_swagger_ui config to api (#50896) (#50946): @pierrejeambrun
• [v3-0-test] Update Airflow 3 migration guide with step about custom operators (#50871) (#50948): @eladkal @github-actions[bot]
• [v3-0-test] Make is_stdout work when sys.stdout does not have a fd (#50328) (#50949): @github-actions[bot] @quentin-sommer
• [v3-0-test] fix(task_instances): handle upstream_mapped_index when xcom access is needed (#50641) (#50950): @Lee-W @github-actions[bot]
• [v3-0-test] Move macros to task SDK execution_time module (#50940) (#50953): @github-actions[bot] @amoghrajesh
• [v3-0-test] Do not use outdated schedule_interval in tutorial dags (#50947) (#50954): @github-actions[bot] @amoghrajesh
• [v3-0-test] Isolate celery tests to separate container (#50952) (#50959): @potiuk
• [v3-0-test] Fix wrong import for PythonOperator in tutorial dag (#50962) (#50967): @github-actions[bot] @amoghrajesh
• [v3-0-test] Always show backfill button but enable it only on dags with schedule. (#50969) (#50970): @tirkarthi @github-actions[bot]
• Fixing failing upgrade checks on main (#50936) (#50974): @amoghrajesh
• [v3-0-test] Increase timeouts for unit tests (#50973) (#50979): @potiuk
• [v3-0-test] Port ti.run to Task SDK execution path (#50141) (#50980): @kaxil
• [v3-0-test] Fetch only the most recent dagrun value for list display. (#50834) (#50988): @tirkarthi @github-actions[bot]
• [v3-0-test] Bump uv to 0.7.7 (#50985) (#50989): @github-actions[bot] @kaxil
• Notify scheduled run failures on slack for v3-0-test branch (#50996): @amoghrajesh
• [v3-0-test] Mask secrets when retrieving variables from secrets backend (#50895) (#50998): @github-actions[bot] @amoghrajesh
• [v3-0-test] Docs assets.rst: use AssetAlias for alias in Metadata example (#50768) (#51000): @QBidolet @github-actions[bot]
• [v3-0-test] Add backcompat shim and deprecation warning for EmailOperator (#51004) (#51008): @github-actions[bot] @amoghrajesh
• [v3-0-test] Run image verification in isolated environments (#51029) (#51030): @potiuk @github-actions[bot]
• [v3-0-test] Fix CI status badge (#51032) (#51033): @potiuk @github-actions[bot]
• [v3-0-test] Bugfix/fix latest pypi version check (#51039) (#51044): @jscheffl
• [v3-0-test] Bump UV to 0.7.8 (#51040) (#51046): @github-actions[bot] @jscheffl
• [v3-0-test] Prevent CPU spike in task supervisor when heartbeat timeout exceeded (#51023) (#51047): @github-actions[bot] @kaxil
• [v3-0-test] Bugfix Breeze BetterChoice with newer click versions (#51051) (#51053): @github-actions[bot] @jscheffl
• [v3-0-test] Update BulkVariableService to support JSON serialization (#51057) (#51111): @guan404ming @github-actions[bot]
• [v3-0-test] Fix bulk action annotation (#50852) (#51116): @guan404ming @pierrejeambrun
• [v3-0-test] Exclude libcst 1.8.0 (no wheels <3.12) (#51120) (#51126): @github-actions[bot] @amoghrajesh
• Revert "[v3-0-test] Fix 404 log error for non-latest task tries in multi-host worker environments" (#51145): @pierrejeambrun
• [v3-0-test] Fix config templates type (#50792) (#51147): @guan404ming @pierrejeambrun
• [v3-0-test] Link to latest dagrun in Dag header. (#51119) (#51148): @tirkarthi @github-actions[bot]
• [v3-0-test] Fix get dags query to not have join explosion (#50984) (#51172): @dstandish @github-actions[bot]
• Fix execution API server URL handling for relative paths in KE (#51183): @kaxil
• [v3-0-test] Execution api versioning docs (#51186): @kaxil @amoghrajesh
• [v3-0-test] Handle SIGSEGV signals during DAG file imports (#51171) (#51189): @ephraimbuddy @github-actions[bot]
• [v3-0-test] Mark test_mapped_task_upstream_removed_with_all_failed_.. as flaky (#51193) (#51198): @potiuk @amoghrajesh @github-actions[bot]
  Linked issues:
  • Fix flaky test: test_mapped_task_upstream_removed_with_all_failed_trigger_rules (#50474)
• [v3-0-test] Fix deferred task resumption in dag.test() (#51182) (#51199): @github-actions[bot] @kaxil
• [v3-0-test] Update TaskLogContent to support virtualized rendering (#50746) (#51202): @guan404ming @jason810496 @github-actions[bot]
• [v3-0-test] Refresh Dag details page on new run (#51173) (#51204): @shubhamraj-git @github-actions[bot]
• [v3-0-test] Bring back "standard" example dags to the airflow-core package (#51192) (#51220): @potiuk @github-actions[bot]
• [v3-0-test] Make test_scheduler_job work locally after moving examples (#50917) (#51221): @potiuk @github-actions[bot]
• [v3-0-test] Significantly speed up Pytest bootstrapping on MacOS in Breeze (#51223) (#51234): @potiuk @github-actions[bot]
• [v3-0-test] Faster note on grid endpoint (#51247) (#51251): @jedcunningham @github-actions[bot]
• [v3-0-test] Move example_dags in standard provider to example_dags in… (#51275): @potiuk
  Linked issues:
  • Bring back "standard" example dags to the airflow-core package (#51192)
• [v3-0-test] Backport gh docs publish breeze workflows (#51286): @gopidesupavan
• [v3-0-test] Fix building PROD images with uv (#51277) (#51289): @potiuk @github-actions[bot]
• [v3-0-test] Log fallback to None when no XCom value is found (#51285) (#51291): @github-actions[bot] @amoghrajesh
• Fix doubel quote mtime in airflow-core/docs/faq.rst (#51293): @gopidesupavan
• [v3-0-test] Fix widget error for core hooks (#51168) (#51300): @pierrejeambrun @github-actions[bot]
• [v3-0-test] Bump structlog min to 25.4.0 and handle tests accordingly (#51306) (#51313): @potiuk @kaxil @amoghrajesh @github-actions[bot]
• [v3-0-test] Remove PytestReturnNotNoneWarning (#51327) (#51332): @github-actions[bot] @gopidesupavan
• [v3-0-test] Add bundle path to sys.path in task runner (#51318) (#51341): @jedcunningham @github-actions[bot] @amoghrajesh
• [v3-0-test] Restrict typing-extensions 4.14.0 to fix CI temporarily (#51342): @amoghrajesh
• [v3-0-test] Separate configurations for colorized and json logs in Task SDK / Celery Executor (#51082) (#51344): @highlyavailable @github-actions[bot]
• [v3-0-test] Ensure that both public and ui dags endpoints map to DagService (#51226) (#51345): @dstandish @pierrejeambrun
• [v3-0-test] Remove filtering by last dag run state in patch dags endpoint (#51176) (#51347): @dstandish @pierrejeambrun
• [v3-0-test] Add link to cadwyn issue created (#51354) (#51355): @potiuk @github-actions[bot]
• [v3-0-test] Fix OpenAPI schema for get_log API (#50547) (#51357): @pierrejeambrun @jason810496
• [v3-0-test] Allow test migration to pytest 8.4.0 (#51349) (#51359): @potiuk @amoghrajesh

Thanks to all who contributed to the release (probably not a complete list!):
@sunank200 @zachliu @praneeth552 @amoghrajesh @ayush3singh @AzisK @QBidolet @zach-overflow @highlyavailable @jason810496 @Lee-W @kaxil @vatsrahul1001 @aritra24 @uranusjr @oboki @potiuk @vincbeck @dominic-lcw @guan404ming @tirkarthi @jscheffl @rtrindvg @scalewithlee @jedcunningham @ephraimbuddy @quentin-sommer @XD-DENG @shubhamraj-git @dheerajturaga @parksunah @abhishekbhakat @bbovenzi @RoyLee1224 @pierrejeambrun @dstandish @choo121600 @gopidesupavan @eladkal

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[jscheffl]

All good, mainly CI fixes that back-ported. But still a lot of fixes into this release. Wow.

[onprema]

Are there plans to push the 3.0.2rc1 version to docker hub?

[jason810496]

Checked:

• [v3-0-test] Fix 404 log error for non-latest task tries in multi-host worker environments #50833
• Revert "[v3-0-test] Fix 404 log error for non-latest task tries in multi-host worker environments" #51145
• [v3-0-test] Fix SQLite Lock error in TestTaskInstancesLog setup (#50763) #50782
• [v3-0-test] Fix OpenAPI schema for get_log API (#50547) #51357

Note: #50833 was reverted by #51145.

[kaxil]

Are there plans to push the 3.0.2rc1 version to docker hub?

They are available, check https://hub.docker.com/r/apache/airflow/tags/?page=1&name=3.0.2

[praneeth552]

Hi @kaxil and team,

I’ve tested the changes from Add Apache Airflow setup instructions for Apple Silicon (#50179) on Airflow 3.0.2rc1 with Python 3.11.13 on my Apple Silicon Mac.

Running airflow standalone worked flawlessly, and I was able to log in with the default credentials.

The setup instructions work well on Apple Silicon.

Thanks for the great work!
— Praneeth

[Lee-W]

tested #50773, #50950

I think we might have missed #51295, not sure whether it's expected, but the original case seems to be well handled by #50950. The only other case that might not be handled yet is this rare mysql case https://github.com/apache/airflow/pull/51295/files#diff-680cf6d70e96761db3869142642ba1df32e5c2c5c8d1b9ab65d0200dca4718daL471

also I saw the following dag import error when testing

Timestamp: 2025-06-05, 18:05:02

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/.../.venv/lib/python3.13/site-packages/airflow/example_dags/standard/example_external_task_parent_deferrable.py", line 60, in <module>
    from tests_common.test_utils.watcher import watcher
ModuleNotFoundError: No module named 'tests_common'

[onprema]

Tested #50304 -- No issues ✅

[guan404ming]

Tested all my changes and LGTM

• Fix timezone selection and dashboard layout #50463
• [v3-0-test] Update default sort for connections and dags (#50600) #50652
• [v3-0-test] Update useTableURLState hook for sticky table sort (#50720) #50858
• [v3-0-test] Update BulkVariableService to support JSON serialization (#51057) #51111
• [v3-0-test] Fix bulk action annotation (#50852) #51116
• [v3-0-test] Fix config templates type (#50792) #51147
• [v3-0-test] Update TaskLogContent to support virtualized rendering (#50746) #51202

[aritra24]

Validated my change

[Lee-W]

I found one critical bug that will break most asset-related features. example_assets.py won't work correctly now and here is the fixing PR #51464

[kaxil]

Changes from 3.0.2rc2

• Remove logical_date check when validate inlets and outlets #51464 : @Lee-W
• feat(task_instances): guard ti update state and set task to fail if exception encountered #51295 : @Lee-W

[dheerajturaga]

@kaxil, Im trying to install 3.0.2rc2 with pip install --pre apache-airflow==3.0.2rc2 but it fails on libcst wheel

  Building wheel for libcst (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for libcst (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [443 lines of output]
      /tmp/pip-build-env-ft4bazon/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py:289: UserWarning: Unknown distribution option: 'test_suite'
        warnings.warn(msg)
      WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
      Traceback (most recent call last):
        File "/tmp/pip-build-env-ft4bazon/overlay/lib/python3.11/site-packages/setuptools_scm/_integration/pyproject_reading.py", line 36, in read_pyproject
          section = defn.get("tool", {})[tool_name]
                    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^