Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The constraints.txt that causes the error when logout #40922

Closed
1 of 2 tasks
phi-friday opened this issue Jul 22, 2024 · 6 comments
Closed
1 of 2 tasks

The constraints.txt that causes the error when logout #40922

phi-friday opened this issue Jul 22, 2024 · 6 comments
Labels
area:core area:dependencies Issues related to dependencies problems area:webserver Webserver related Issues kind:bug This is a clearly a bug provider:fab

Comments

@phi-friday
Copy link
Contributor

Apache Airflow version

2.9.3

If "Other Airflow 2 version" selected, which one?

No response

What happened?

in https://raw.githubusercontent.com/apache/airflow/constraints-2.9.3/constraints-3.12.txt,
apache-airflow-providers-fab==1.2.1

but according to #40470 and #40784, <=2.9.3 needs apache-airflow-providers-fab<1.2.0
스크린샷 2024-07-22 오전 10 04 31

What you think should happen instead?

No response

How to reproduce

in dockerfile

RUN curl -Lf https://raw.githubusercontent.com/apache/airflow/constraints-2.9.3/constraints-3.12.txt -o constraint.txt && \
    uv pip compile pyproject.toml -o requirements.txt --constraint constraint.txt && \
    uv pip sync requirements.txt --constraint constraint.txt --no-cache

Operating System

PRETTY_NAME="Ubuntu 22.04.3 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.3 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy

Versions of Apache Airflow Providers

➜  airflow pip freeze | grep apache-airflow-providers
apache-airflow-providers-celery==3.7.2
apache-airflow-providers-common-io==1.3.2
apache-airflow-providers-common-sql==1.14.2
apache-airflow-providers-docker==3.12.2
apache-airflow-providers-fab==1.2.1
apache-airflow-providers-ftp==3.10.0
apache-airflow-providers-http==4.12.0
apache-airflow-providers-imap==3.6.1
apache-airflow-providers-jdbc==4.3.1
apache-airflow-providers-odbc==4.6.2
apache-airflow-providers-postgres==5.11.2
apache-airflow-providers-redis==3.7.1
apache-airflow-providers-smtp==1.7.1
apache-airflow-providers-sqlite==3.8.1

Deployment

Docker-Compose

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@phi-friday phi-friday added area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels Jul 22, 2024
@phi-friday phi-friday changed the title The constrains.txt that causes the error The constrains.txt that causes the error when logout Jul 22, 2024
@dosubot dosubot bot added the area:dependencies Issues related to dependencies problems label Jul 22, 2024
@phi-friday
Copy link
Contributor Author

docker image apache/airflow:slim-2.9.3-python3.12 also has apache-airflow-providers-fab==1.2.1

❯ docker run -it --rm apache/airflow:slim-2.9.3-python3.12 bash
Unable to find image 'apache/airflow:slim-2.9.3-python3.12' locally
slim-2.9.3-python3.12: Pulling from apache/airflow
ea235d1ccf77: Already exists 
26f13e32d8cd: Already exists 
eee30644a5dc: Already exists 
b9c7fa2607b2: Already exists 
a722882a2fdc: Already exists 
af8854376e74: Already exists 
1941d27b236c: Already exists 
a77ea609467b: Already exists 
22480f27be1d: Already exists 
bc5deb28b981: Already exists 
25aaf209ef27: Already exists 
4ec7874b8c66: Already exists 
c8ef768c3a3b: Already exists 
9016b3531598: Already exists 
268c66ad56e0: Already exists 
b20e5315c25d: Already exists 
f41cc2f8e1af: Already exists 
2c08b196ed73: Already exists 
c06d2457e55c: Already exists 
b85c1ee3156a: Already exists 
ac76d474c5e5: Already exists 
4f4fb700ef54: Already exists 
Digest: sha256:e41cd2aabd88e5692ca6fe4e95bd3098aed444e1d3cd42bd2ffa751927019539
Status: Downloaded newer image for apache/airflow:slim-2.9.3-python3.12

airflow@f4bb16409bdb:/opt/airflow$ pip list | grep fab
apache-airflow-providers-fab             1.2.1

@shahar1
Copy link
Contributor

shahar1 commented Jul 22, 2024

@potiuk if I understood correctly - according to #40784 the min. Airflow version for supporting the latest FAB provider will be 2.10, yet the constraints file for 2.9.3 refers to the latest. How should the fix be applied in thus case?

@potiuk
Copy link
Member

potiuk commented Jul 22, 2024

It will have to wait for the next FAB provider release and then we might be able to bump it in the constraints.

@shahar1 shahar1 changed the title The constrains.txt that causes the error when logout The constraints.txt that causes the error when logout Jul 22, 2024
@shahar1 shahar1 added provider:fab area:webserver Webserver related Issues and removed needs-triage label for new issues that we didn't triage yet labels Jul 22, 2024
@potiuk
Copy link
Member

potiuk commented Aug 1, 2024

Should be solved now with the FAB provider released. Constraints are refreshed, images are rebuild.

Changelog is being added to the dockerfile/image changelog #41205 -> will be updated with next Airflow release as a track record.

Here is the image provider set now:

docker run -it apache/airflow:2.9.3 providers list

package_name                             | description                                                                                  | version
=========================================+==============================================================================================+========
apache-airflow-providers-amazon          | Amazon integration (including Amazon Web Services (AWS) https://aws.amazon.com/)             | 8.25.0
apache-airflow-providers-celery          | Celery https://docs.celeryq.dev/en/stable/                                                   | 3.7.2
apache-airflow-providers-cncf-kubernetes | Kubernetes https://kubernetes.io/                                                            | 8.3.3
apache-airflow-providers-common-io       | Common IO Provider                                                                           | 1.3.2
apache-airflow-providers-common-sql      | Common SQL Provider https://en.wikipedia.org/wiki/SQL                                        | 1.14.2
apache-airflow-providers-docker          | Docker https://www.docker.com/                                                               | 3.12.2
apache-airflow-providers-elasticsearch   | Elasticsearch https://www.elastic.co/elasticsearch                                           | 5.4.1
apache-airflow-providers-fab             | Flask App Builder https://flask-appbuilder.readthedocs.io/                                   | 1.2.2
apache-airflow-providers-ftp             | File Transfer Protocol (FTP) https://tools.ietf.org/html/rfc114                              | 3.10.0
apache-airflow-providers-google          | Google services including:                                                                   | 10.21.0
                                         |                                                                                              |
                                         |   - Google Ads https://ads.google.com/                                                       |
                                         |   - Google Cloud (GCP) https://cloud.google.com/                                             |
                                         |   - Google Firebase https://firebase.google.com/                                             |
                                         |   - Google LevelDB https://github.com/google/leveldb/                                        |
                                         |   - Google Marketing Platform https://marketingplatform.google.com/                          |
                                         |   - Google Workspace https://workspace.google.com/ (formerly Google Suite)                   |
apache-airflow-providers-grpc            | gRPC https://grpc.io/                                                                        | 3.5.2
apache-airflow-providers-hashicorp       | Hashicorp including Hashicorp Vault https://www.vaultproject.io/                             | 3.7.1
apache-airflow-providers-http            | Hypertext Transfer Protocol (HTTP) https://www.w3.org/Protocols/                             | 4.12.0
apache-airflow-providers-imap            | Internet Message Access Protocol (IMAP) https://tools.ietf.org/html/rfc3501                  | 3.6.1
apache-airflow-providers-microsoft-azure | Microsoft Azure https://azure.microsoft.com/                                                 | 10.2.0
apache-airflow-providers-mysql           | MySQL https://www.mysql.com/                                                                 | 5.6.2
apache-airflow-providers-odbc            | ODBC https://github.com/mkleehammer/pyodbc/wiki                                              | 4.6.2
apache-airflow-providers-openlineage     | OpenLineage https://openlineage.io/                                                          | 1.9.1
apache-airflow-providers-postgres        | PostgreSQL https://www.postgresql.org/                                                       | 5.11.2
apache-airflow-providers-redis           | Redis https://redis.io/                                                                      | 3.7.1
apache-airflow-providers-sendgrid        | Sendgrid https://sendgrid.com/                                                               | 3.5.1
apache-airflow-providers-sftp            | SSH File Transfer Protocol (SFTP) https://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/ | 4.10.2
apache-airflow-providers-slack           | Slack https://slack.com/ services integration including:                                     | 8.7.1
                                         |                                                                                              |
                                         |   - Slack API https://api.slack.com/                                                         |
                                         |   - Slack Incoming Webhook https://api.slack.com/messaging/webhooks                          |
apache-airflow-providers-smtp            | Simple Mail Transfer Protocol (SMTP) https://tools.ietf.org/html/rfc5321                     | 1.7.1
apache-airflow-providers-snowflake       | Snowflake https://www.snowflake.com/                                                         | 5.6.0
apache-airflow-providers-sqlite          | SQLite https://www.sqlite.org/                                                               | 3.8.1
apache-airflow-providers-ssh             | Secure Shell (SSH) https://tools.ietf.org/html/rfc4251                                       | 3.11.2

Also confirmed with breeze start-airflow --use-airflow 2.9.3 that the constraints point to the new FAB provider and logout works.

@potiuk potiuk closed this as completed Aug 1, 2024
@phi-friday
Copy link
Contributor Author

phi-friday commented Aug 2, 2024

@potiuk
I see that apache/airflow:2.9.3 is fixed, but it looks like apache/airflow:slim-2.9.3-python3.12 is still 1.2.1.
Is an update to the slim image also planned?

(in docker hub, 2.9.3 last pushed 2 hours ago,
slim-2.9.3 last pushed 17 days ago.)

❯ docker run -it --rm apache/airflow:slim-2.9.3-python3.12 providers list

package_name                        | description                                                 | version
====================================+=============================================================+========
apache-airflow-providers-common-io  | Common IO Provider                                          | 1.3.2
apache-airflow-providers-common-sql | Common SQL Provider https://en.wikipedia.org/wiki/SQL       | 1.14.2
apache-airflow-providers-fab        | Flask App Builder https://flask-appbuilder.readthedocs.io/  | 1.2.1
apache-airflow-providers-ftp        | File Transfer Protocol (FTP)                                | 3.10.0
                                    | https://tools.ietf.org/html/rfc114                          |
apache-airflow-providers-http       | Hypertext Transfer Protocol (HTTP)                          | 4.12.0
                                    | https://www.w3.org/Protocols/                               |
apache-airflow-providers-imap       | Internet Message Access Protocol (IMAP)                     | 3.6.1
                                    | https://tools.ietf.org/html/rfc3501                         |
apache-airflow-providers-smtp       | Simple Mail Transfer Protocol (SMTP)                        | 1.7.1
                                    | https://tools.ietf.org/html/rfc5321                         |
apache-airflow-providers-sqlite     | SQLite https://www.sqlite.org/                              | 3.8.1

The constraints.txt has been fixed, so it's not an issue for me, but I ask just in case.

@potiuk
Copy link
Member

potiuk commented Aug 2, 2024

Oh yes. Very good point. We do it very rarely ( see https://github.com/apache/airflow/pull/41205/files). And the slim part failed (I have not noticed it). Thanks for double-checking and notifying @phi-friday !

All slim images are rebuilt and uploaded.

docker run -it  apache/airflow:slim-2.9.3 providers list

package_name                        | description                                                                 | version
====================================+=============================================================================+========
apache-airflow-providers-common-io  | Common IO Provider                                                          | 1.3.2
apache-airflow-providers-common-sql | Common SQL Provider https://en.wikipedia.org/wiki/SQL                       | 1.14.2
apache-airflow-providers-fab        | Flask App Builder https://flask-appbuilder.readthedocs.io/                  | 1.2.2
apache-airflow-providers-ftp        | File Transfer Protocol (FTP) https://tools.ietf.org/html/rfc114             | 3.10.0
apache-airflow-providers-http       | Hypertext Transfer Protocol (HTTP) https://www.w3.org/Protocols/            | 4.12.0
apache-airflow-providers-imap       | Internet Message Access Protocol (IMAP) https://tools.ietf.org/html/rfc3501 | 3.6.1
apache-airflow-providers-smtp       | Simple Mail Transfer Protocol (SMTP) https://tools.ietf.org/html/rfc5321    | 1.7.1
apache-airflow-providers-sqlite     | SQLite https://www.sqlite.org/                                              | 3.8.1
Screenshot 2024-08-02 at 10 24 06

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:core area:dependencies Issues related to dependencies problems area:webserver Webserver related Issues kind:bug This is a clearly a bug provider:fab
Projects
None yet
Development

No branches or pull requests

3 participants