Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile.ci
Original file line number Diff line number Diff line change
Expand Up @@ -840,7 +840,7 @@ chmod 1777 /tmp

AIRFLOW_SOURCES=$(cd "${IN_CONTAINER_DIR}/../.." || exit 1; pwd)

PYTHON_MAJOR_MINOR_VERSION=${PYTHON_MAJOR_MINOR_VERSION:=3.9}
PYTHON_MAJOR_MINOR_VERSION=${PYTHON_MAJOR_MINOR_VERSION:=3.10}

export AIRFLOW_HOME=${AIRFLOW_HOME:=${HOME}}

Expand Down
1 change: 0 additions & 1 deletion chart/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ classifiers = [
"Intended Audience :: Developers",
"Intended Audience :: System Administrators",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
Expand Down
3 changes: 1 addition & 2 deletions clients/python/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ classifiers = [
"Intended Audience :: Developers",
"Intended Audience :: System Administrators",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
Expand Down Expand Up @@ -73,7 +72,7 @@ run-coverage = "pytest test"
run = "run-coverage --no-cov"

[[tool.hatch.envs.test.matrix]]
python = ["3.9", "3.10", "3.11"]
python = ["3.10", "3.11"]

[tool.hatch.version]
path = "./version.txt"
Expand Down
2 changes: 1 addition & 1 deletion clients/python/test_python_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
#
# PEP 723 compliant inline script metadata (not yet widely supported)
# /// script
# requires-python = ">=3.9"
# requires-python = ">=3.10"
# dependencies = [
# "apache-airflow-client",
# "rich",
Expand Down
6 changes: 3 additions & 3 deletions constraints/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,12 @@ This allows you to iterate on dependencies without having to run `--upgrade-to-n
Typical workflow in this case is:

* download and copy the constraint file to the folder (for example via
[The GitHub Raw Link](https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.9.txt))
[The GitHub Raw Link](https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.10.txt))
* modify the constraint file in "constraints" folder
* build the image using this command

```bash
breeze ci-image build --python 3.9 --airflow-constraints-location constraints/constraints-3.9.txt
breeze ci-image build --python 3.10 --airflow-constraints-location constraints/constraints-3.10.txt
```

You can continue iterating and updating the constraint file (and rebuilding the image)
Expand All @@ -46,7 +46,7 @@ pip freeze | sort | \
grep -v "apache_airflow" | \
grep -v "apache-airflow==" | \
grep -v "@" | \
grep -v "/opt/airflow" > /opt/airflow/constraints/constraints-3.9.txt
grep -v "/opt/airflow" > /opt/airflow/constraints/constraints-3.10.txt
```

If you are working with others on updating the dependencies, you can also commit the constraint
Expand Down
24 changes: 12 additions & 12 deletions contributing-docs/03_contributors_quick_start.rst
Original file line number Diff line number Diff line change
Expand Up @@ -255,8 +255,8 @@ To avoid burden on our CI infrastructure and to save time, Pre-commit hooks can
We have recently started to recommend ``uv`` for our local development.

.. note::
Remember to have global python set to Python >= 3.9 - Python 3.8 is end-of-life already and we've
started to use Python 3.9+ features in Airflow and accompanying scripts.
Remember to have global python set to Python >= 3.10 - Python 3.10 is end-of-life already and we've
started to use Python 3.10+ features in Airflow and accompanying scripts.

Installing pre-commit is best done with ``uv`` (recommended) or ``pipx``.

Expand Down Expand Up @@ -433,7 +433,7 @@ see in CI in your local environment.

.. code-block:: bash

breeze --python 3.9 --backend postgres
breeze --python 3.10 --backend postgres

.. note::
If you encounter an error like "docker.credentials.errors.InitializationError:
Expand Down Expand Up @@ -490,7 +490,7 @@ Using Breeze
------------

1. Starting the Breeze environment using ``breeze start-airflow`` starts the Breeze environment with last configuration run(
In this case Python version and backend are picked up from last execution ``breeze --python 3.9 --backend postgres``)
In this case Python version and backend are picked up from last execution ``breeze --python 3.10 --backend postgres``)
It also automatically starts the API server (FastAPI api and UI), triggerer, dag processor and scheduler. It drops you in tmux with triggerer to the right, and
Scheduler, API server (FastAPI api and UI), DAG processor from left to right at the bottom. Use ``[Ctrl + B] and Arrow keys`` to navigate.

Expand All @@ -501,9 +501,9 @@ Using Breeze
Use CI image.

Branch name: main
Docker image: ghcr.io/apache/airflow/main/ci/python3.9:latest
Docker image: ghcr.io/apache/airflow/main/ci/python3.10:latest
Airflow source version: 2.4.0.dev0
Python version: 3.9
Python version: 3.10
Backend: mysql 5.7

* Port forwarding:
Expand Down Expand Up @@ -540,7 +540,7 @@ Using Breeze

.. code-block:: bash

breeze --python 3.9 --backend postgres
breeze --python 3.10 --backend postgres

2. Open tmux

Expand Down Expand Up @@ -614,7 +614,7 @@ If ``breeze`` was started with ``breeze start-airflow``, this command will stop
root@f3619b74c59a:/opt/airflow# stop_airflow
breeze down

If ``breeze`` was started with ``breeze --python 3.9 --backend postgres`` (or similar):
If ``breeze`` was started with ``breeze --python 3.10 --backend postgres`` (or similar):

.. code-block:: bash

Expand Down Expand Up @@ -674,7 +674,7 @@ All Tests are inside ./tests directory.

root@63528318c8b1:/opt/airflow# pytest tests/utils/test_dates.py
============================================================= test session starts ==============================================================
platform linux -- Python 3.9.20, pytest-8.3.3, pluggy-1.5.0 -- /usr/local/bin/python
platform linux -- Python 3.10.20, pytest-8.3.3, pluggy-1.5.0 -- /usr/local/bin/python
cachedir: .pytest_cache
rootdir: /opt/airflow
configfile: pyproject.toml
Expand All @@ -694,20 +694,20 @@ All Tests are inside ./tests directory.

.. code-block:: bash

breeze --backend postgres --postgres-version 15 --python 3.9 --db-reset testing tests --test-type All
breeze --backend postgres --postgres-version 15 --python 3.10 --db-reset testing tests --test-type All

- Running specific type of test

.. code-block:: bash

breeze --backend postgres --postgres-version 15 --python 3.9 --db-reset testing tests --test-type Core
breeze --backend postgres --postgres-version 15 --python 3.10 --db-reset testing tests --test-type Core


- Running Integration test for specific test type

.. code-block:: bash

breeze --backend postgres --postgres-version 15 --python 3.9 --db-reset testing tests --test-type All --integration mongo
breeze --backend postgres --postgres-version 15 --python 3.10 --db-reset testing tests --test-type All --integration mongo

- For more information on Testing visit |09_testing.rst|

Expand Down
2 changes: 1 addition & 1 deletion contributing-docs/05_pull_requests.rst
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ these guidelines:
you can push your code to PR and see results of the tests in the CI.

- You can use any supported python version to run the tests, but the best is to check
if it works for the oldest supported version (Python 3.9 currently). In rare cases
if it works for the oldest supported version (Python 3.10 currently). In rare cases
tests might fail with the oldest version when you use features that are available in newer Python
versions. For that purpose we have ``airflow.compat`` package where we keep back-ported
useful features from newer versions.
Expand Down
14 changes: 7 additions & 7 deletions contributing-docs/07_local_virtualenv.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Required Software Packages
Use system-level package managers like yum, apt-get for Linux, or
Homebrew for macOS to install required software packages:

* Python (One of: 3.9, 3.10, 3.11, 3.12)
* Python (One of: 3.10, 3.11, 3.12)
* MySQL 5.7+
* libxml
* helm (only for helm chart tests)
Expand Down Expand Up @@ -100,11 +100,11 @@ Installing Python versions

This step can be skipped - ``uv`` will automatically install the Python version you need when you create a virtualenv.

You can install Python versions using ``uv python install`` command. For example, to install Python 3.9.7, you can run:
You can install Python versions using ``uv python install`` command. For example, to install Python 3.10.7, you can run:

.. code:: bash

uv python install 3.9.7
uv python install 3.10.7

This is optional step - ``uv`` will automatically install the Python version you need when you create a virtualenv.

Expand All @@ -124,7 +124,7 @@ with a specific Python version by running:

.. code:: bash

uv venv --python 3.9.7
uv venv --python 3.10.7

You can also create a venv with a different venv directory name by running:

Expand Down Expand Up @@ -275,12 +275,12 @@ to avoid "works-for-me" syndrome, where you use different version of dependencie
that are used in main, CI tests and by other contributors.

There are different constraint files for different python versions. For example this command will install
all basic devel requirements and requirements of google provider as last successfully tested for Python 3.9:
all basic devel requirements and requirements of google provider as last successfully tested for Python 3.10:

.. code:: bash

uv pip install -e ".[devel,google]" \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-source-providers-3.9.txt"
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-source-providers-3.10.txt"


In the future we will utilise ``uv.lock`` to manage dependencies and constraints, but for the moment we do not
Expand All @@ -305,7 +305,7 @@ and install to latest supported ones by pure Airflow core.
.. code:: bash

uv pip install -e ".[devel]" \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.9.txt"
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.10.txt"

These are examples of the development options available with the local virtualenv in your IDE:

Expand Down
2 changes: 1 addition & 1 deletion contributing-docs/08_static_code_checks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ use. So, you can be sure your modifications will also work for CI if they pass
pre-commit hooks.

We have integrated the fantastic `pre-commit <https://pre-commit.com>`__ framework
in our development workflow. To install and use it, you need at least Python 3.9 locally.
in our development workflow. To install and use it, you need at least Python 3.10 locally.

Installing pre-commit hooks
---------------------------
Expand Down
8 changes: 4 additions & 4 deletions contributing-docs/testing/docker_compose_tests.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Running complete test with breeze:

.. code-block:: bash

breeze prod-image build --python 3.9
breeze prod-image build --python 3.10
breeze testing docker-compose-tests

In case the test fails, it will dump the logs from the running containers to the console and it
Expand All @@ -65,8 +65,8 @@ to see the output of the test as it happens (it can be also set via
The test can be also run manually with ``pytest docker_tests/test_docker_compose_quick_start.py``
command, provided that you have a local Airflow venv with ``dev`` extra set and the
``DOCKER_IMAGE`` environment variable is set to the image you want to test. The variable defaults
to ``ghcr.io/apache/airflow/main/prod/python3.9:latest`` which is built by default
when you run ``breeze prod-image build --python 3.9``. also the switches ``--skip-docker-compose-deletion``
to ``ghcr.io/apache/airflow/main/prod/python3.10:latest`` which is built by default
when you run ``breeze prod-image build --python 3.10``. also the switches ``--skip-docker-compose-deletion``
and ``--wait-for-containers-timeout`` can only be passed via environment variables.

If you want to debug the deployment using ``docker compose`` commands after ``SKIP_DOCKER_COMPOSE_DELETION``
Expand All @@ -87,7 +87,7 @@ the prod image build command above.

.. code-block:: bash

export AIRFLOW_IMAGE_NAME=ghcr.io/apache/airflow/main/prod/python3.9:latest
export AIRFLOW_IMAGE_NAME=ghcr.io/apache/airflow/main/prod/python3.10:latest

and follow the instructions in the
`Running Airflow in Docker <https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html>`_
Expand Down
Loading