Dockerfile 3.0.0 does not work with uv enabled #50134
Description
Apache Airflow version
3.0.0
If "Other Airflow 2 version" selected, which one?
No response
What happened?
There is an error about pyproject.toml not being found when AIRFLOW_USE_UV is set to true.
This does not happen on the 2.10.5 Dockerfile version.
0.080 Using 'uv' to install Airflow
0.080
0.090
0.090 Downloading constraints from https://raw.githubusercontent.com/apache/airflow/constraints-3.0.0/constraints-3.12.txt to /home/airflow/constraints.txt
0.090
0.468 PATH=/home/airflow/.local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/mssql-tools/bin
0.810 Installed pip: pip 25.0.1 from /home/airflow/.local/lib/python3.12/site-packages/pip (python 3.12): /home/airflow/.local/bin/pip
0.810 Using 'uv' to install Airflow
0.813 Installed uv: uv 0.6.13: /home/airflow/.local/bin/uv
0.813 MYSQL client installation is disabled. Extra 'mysql' installations were therefore omitted.
0.813
0.813 Installing all packages with constraints. Installation method: apache-airflow
0.813
0.814 + uv pip install --group=dev 'apache-airflow[async,docker,kubernetes,celery,redis,ftp,google,google_auth,http,microsoft.azure,sftp,ssh,statsd,openlineage,postgres,odbc]==3.0.0' --constraint /home/airflow/constraints.txt
0.817 error: File not found: `pyproject.toml`
0.817
0.817 Likely pyproject.toml has new dependencies conflicting with constraints.
0.817
0.817 Falling back to no-constraints installation.
0.817
0.817 + set +x
0.817 + uv pip install --group=dev --upgrade 'apache-airflow[async,docker,kubernetes,celery,redis,ftp,google,google_auth,http,microsoft.azure,sftp,ssh,statsd,openlineage,postgres,odbc]==3.0.0'
0.820 error: File not found: `pyproject.toml`
What you think should happen instead?
I have used UV since July 25th 2024 to build custom Airflow images and it has worked fine.
If I change the MRE (below) to use a previous version of the Dockerfile like 2.10.5 then it runs fine.
curl -L "https://raw.githubusercontent.com/apache/airflow/2.10.5/Dockerfile" -o Dockerfile
Disabling uv also works fine (I would prefer not to, but I understand uv is an experimental feature and meant for development purposes).
How to reproduce
export ARCHITECTURE="arm64"
export AIRFLOW_VERSION="3.0.0"
export DEBIAN_VERSION="bookworm"
export PYTHON_VERSION="3.12"
export DOCKER_BUILDKIT=1
CONSTRAINTS_VERSION="constraints-${AIRFLOW_VERSION}"
TARGET_PLATFORM="linux/${ARCHITECTURE}"
echo "Building Airflow ${AIRFLOW_VERSION} with Python ${PYTHON_VERSION} and Debian ${DEBIAN_VERSION}"
curl -L "https://raw.githubusercontent.com/apache/airflow/${AIRFLOW_VERSION}/Dockerfile" -o Dockerfile
docker build \
--file Dockerfile \
--build-arg TARGET_PLATFORM="${TARGET_PLATFORM}" \
--build-arg PYTHON_BASE_IMAGE="python:${PYTHON_VERSION}-slim-${DEBIAN_VERSION}" \
--build-arg AIRFLOW_USE_UV="true" \
--build-arg AIRFLOW_VERSION="${AIRFLOW_VERSION}" \
--build-arg AIRFLOW_EXTRAS="async,docker,kubernetes,celery,redis,ftp,google,google_auth,http,microsoft.azure,sftp,ssh,statsd,openlineage,postgres,odbc" \
--build-arg AIRFLOW_CONSTRAINTS_REFERENCE="${CONSTRAINTS_VERSION}" \
--build-arg INSTALL_MYSQL_CLIENT="false" \
--build-arg INSTALL_MSSQL_CLIENT="false" \
--build-arg INSTALL_POSTGRES_CLIENT="true" \
--build-arg UPGRADE_TO_NEWER_DEPENDENCIES="true" \
--build-arg ADDITIONAL_DEV_APT_DEPS="unixodbc-dev g++ gnupg2 lsb-release" \
--build-arg ADDITIONAL_RUNTIME_APT_DEPS="git graphviz procps vim odbc-postgresql python3-pybind11 python3-pyodbc libgeos-dev" \
--tag "airflow-example:${AIRFLOW_VERSION}-base" \
.
Operating System
Sonoma 14.7 (23H124)
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
Currently I am just building the image (locally for testing and through Gitlab CI/CD pipelines). The official Helm chart would then point to the custom image I build.
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct