Skip to content

Conversation

@amoghrajesh
Copy link
Contributor

Docker build is failing with error:

  #57 23.38   Preparing metadata (pyproject.toml): finished with status 'done'
  #57 23.38 INFO: pip is looking at multiple versions of apache-airflow[aiobotocore,amazon,async,celery,cncf-kubernetes,common-io,common-messaging,docker,elasticsearch,fab,ftp,git,google,google-auth,graphviz,grpc,hashicorp,http,ldap,microsoft-azure,mysql,odbc,openlineage,pandas,postgres,redis,sendgrid,sftp,slack,snowflake,ssh,statsd,uv] to determine which version is compatible with other requirements. This could take a while.
  #57 23.39 ERROR: Cannot install apache-airflow-providers-openlineage 2.2.0 (from /docker-context-files/apache_airflow_providers_openlineage-2.2.0-py3-none-any.whl) and apache-airflow[aiobotocore,amazon,async,celery,cncf-kubernetes,common-io,common-messaging,docker,elasticsearch,fab,ftp,git,google,google-auth,graphviz,grpc,hashicorp,http,ldap,microsoft-azure,mysql,odbc,openlineage,pandas,postgres,redis,sendgrid,sftp,slack,snowflake,ssh,statsd,uv]==3.1.0 because these package versions have conflicting dependencies.
  #57 23.39 
  #57 23.39 The conflict is caused by:
  #57 23.39     The user requested apache-airflow-providers-openlineage 2.2.0 (from /docker-context-files/apache_airflow_providers_openlineage-2.2.0-py3-none-any.whl)
  #57 23.39     apache-airflow[aiobotocore,amazon,async,celery,cncf-kubernetes,common-io,common-messaging,docker,elasticsearch,fab,ftp,git,google,google-auth,graphviz,grpc,hashicorp,http,ldap,microsoft-azure,mysql,odbc,openlineage,pandas,postgres,redis,sendgrid,sftp,slack,snowflake,ssh,statsd,uv] 3.1.0 depends on apache-airflow-providers-openlineage>=2.3.0; extra == "openlineage"
  #57 23.39 
  #57 23.39 To fix this you could try to:
  #57 23.39 1. loosen the range of package versions you've specified
  #57 23.39 2. remove package versions to allow pip to attempt to solve the dependency conflict
  #57 23.39 
  #57 23.39 ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
  #57 ERROR: process "/bin/bash -o pipefail -o errexit -o nounset -o nolog -c if [[ ${INSTALL_DISTRIBUTIONS_FROM_CONTEXT} == \"true\" ]]; then         bash /scripts/docker/install_from_docker_context_files.sh;     fi;     if ! airflow version 2>/dev/null >/dev/null; then         bash /scripts/docker/install_airflow_when_building_images.sh;     fi;     if [[ -n \"${ADDITIONAL_PYTHON_DEPS}\" ]]; then         bash /scripts/docker/install_additional_dependencies.sh;     fi;     find \"${AIRFLOW_USER_HOME_DIR}/.local/\" -name '*.pyc' -print0 | xargs -0 rm -f || true ;     find \"${AIRFLOW_USER_HOME_DIR}/.local/\" -type d -name '__pycache__' -print0 | xargs -0 rm -rf || true ;     find \"${AIRFLOW_USER_HOME_DIR}/.local\" -executable ! -type l -print0 | xargs --null chmod g+x;     find \"${AIRFLOW_USER_HOME_DIR}/.local\" ! -type l -print0 | xargs --null chmod g+rw" did not complete successfully: exit code: 1
  ------
   > [airflow-build-image 15/16] RUN --mount=type=cache,id=prod-amd64-9,target=/tmp/.cache/,uid=50000     if [[ true == "true" ]]; then         bash /scripts/docker/install_from_docker_context_files.sh;     fi;     if ! airflow version 2>/dev/null >/dev/null; then         bash /scripts/docker/install_airflow_when_building_images.sh;     fi;     if [[ -n "" ]]; then         bash /scripts/docker/install_additional_dependencies.sh;     fi;     find "/home/airflow/.local/" -name '*.pyc' -print0 | xargs -0 rm -f || true ;     find "/home/airflow/.local/" -type d -name '__pycache__' -print0 | xargs -0 rm -rf || true ;     find "/home/airflow/.local" -executable ! -type l -print0 | xargs --null chmod g+x;     find "/home/airflow/.local" ! -type l -print0 | xargs --null chmod g+rw:
  23.39 
  23.39 The conflict is caused by:
  23.39     The user requested apache-airflow-providers-openlineage 2.2.0 (from /docker-context-files/apache_airflow_providers_openlineage-2.2.0-py3-none-any.whl)
  23.39     apache-airflow[aiobotocore,amazon,async,celery,cncf-kubernetes,common-io,common-messaging,docker,elasticsearch,fab,ftp,git,google,google-auth,graphviz,grpc,hashicorp,http,ldap,microsoft-azure,mysql,odbc,openlineage,pandas,postgres,redis,sendgrid,sftp,slack,snowflake,ssh,statsd,uv] 3.1.0 depends on apache-airflow-providers-openlineage>=2.3.0; extra == "openlineage"

Added in #49237 but probably not for all providers.

Example: https://github.com/apache/airflow/actions/runs/14852598445/job/41699559748?pr=50224


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@amoghrajesh amoghrajesh added the full tests needed We need to run full set of tests for this PR to merge label May 6, 2025
@amoghrajesh
Copy link
Contributor Author

Running with full tests too

@amoghrajesh amoghrajesh closed this May 6, 2025
@amoghrajesh amoghrajesh reopened this May 6, 2025
@amoghrajesh amoghrajesh changed the title Adding upper bound to openlineage in optional dependencies Adding upper bound to openlineage in optional dependencies for all providers May 6, 2025
@amoghrajesh
Copy link
Contributor Author

This doesn't fix the issue but #50230 does. Closing in favour of that

@amoghrajesh amoghrajesh closed this May 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants