Update UI authentication to handle JWT token in backend

[vincbeck]

Resolves #55143.

For better simplicity and better security this PR updates the Airflow UI authentication. With this change the front-end no longer needs to handle the JWT token, it makes API calls to Airflow API (public and UI APIs) with no authentication. A fallback mechanism is added to the back-end so that, if no authentication is provided as part of a request, it fetches the JWT token from the cookies (if it exists). This cookie is saved by auth managers like today, the only change is now this cookie is httpOnly, so Javascript cannot read it.

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[pierrejeambrun]

LGTM, thanks

[github-actions]

Backport failed to create: v3-1-test. View the failure log Run details

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker b8af111 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

[vincbeck]

Backport: #56677

[IDo4axD]

Invalid parameter: redirect_uri

I’ve been working with Airflow and Keycloak, and ran into the same issue: Airflow sends an incorrect redirect_uri using http instead of https, which is insecure. I want it to use https.
https://github.com/apache/airflow/blob/main/providers/keycloak/src/airflow/providers/keycloak/auth_manager/routes/login.py

I’ve tried setting base_url in the config, but it has no effect — the redirect still uses http during the login flow. My Airflow web UI is behind NGINX, which is properly configured to forward all necessary headers and redirect to https.

From what I understand, this seems to be a widespread issue, and I’d like to know when it will be fixed.

Also, in the newer Airflow versions, I can no longer use the old OIDC config with Flask AppBuilder to temporarily bypass the issue — it's no longer compatible.

[vincbeck]

Invalid parameter: redirect_uri

I’ve been working with Airflow and Keycloak, and ran into the same issue: Airflow sends an incorrect redirect_uri using http instead of https, which is insecure. I want it to use https. https://github.com/apache/airflow/blob/main/providers/keycloak/src/airflow/providers/keycloak/auth_manager/routes/login.py

I’ve tried setting base_url in the config, but it has no effect — the redirect still uses http during the login flow. My Airflow web UI is behind NGINX, which is properly configured to forward all necessary headers and redirect to https.

From what I understand, this seems to be a widespread issue, and I’d like to know when it will be fixed.

Interesting. I think the issue comes from the fact the api server is behind a proxy. When Airflow does request.url_for("login_callback"), it assumes it uses http instead of https. https://github.com/Kludex/uvicorn/blob/main/uvicorn/middleware/proxy_headers.py looks the perfect candidate to fix that. Is that okay to add that middleware @pierrejeambrun or we are trying to limit the number of middlewares as much as we can?

Also, in the newer Airflow versions, I can no longer use the old OIDC config with Flask AppBuilder to temporarily bypass the issue — it's no longer compatible.

OIDC authentication has been removed from Flask-appbuilder 5, so yes, this option is longer possible in Fab auth manager because it now uses Flask-appbuilder 5.

[pierrejeambrun]

We've been trying to avoid setting any responsibility in Airflow for handling https, and leave that up to the proxy to do SSL termination.

Similarly can this be fixed at the NGINX level instead? @Romanchyk

[IDo4axD]

@pierrejeambrun
It example of my NGINX configuration: I have a default redirect that forces all HTTP traffic to HTTPS.

According to the documentation you shared, I’ve already configured all the required headers X-Forwarded-Proto.

I also tried setting the following environment variable, but it didn’t help:

• name: AIRFLOW__API__UVICORN_CMD_ARGS
  value: "--host 0.0.0.0 --port 8080 --proxy-headers"

Next, I’ll try overriding the headers explicitly using serverSnippet, because I’m using a VirtualServer (Kubernetes NGINX custom resource).

    location / {
        proxy_connect_timeout 200s;
        proxy_read_timeout 200s;
        proxy_send_timeout 200s;
        client_max_body_size 0;
        proxy_buffer_size 8k;
        proxy_ssl_verify off;

        proxy_buffering on;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_pass_request_headers on;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $host;
        proxy_pass https://airflow-test-api-server;
        proxy_next_upstream error timeout;
        proxy_next_upstream_timeout 0s;
        proxy_next_upstream_tries 0;
    }

[IDo4axD]

@pierrejeambrun
It can’t be fixed via the proxy, because the client is already coming over HTTPS. The redirect is being performed by Airflow itself. Most likely, the issue needs to be fixed in the AuthManager, as it doesn’t know the correct scheme (HTTP/HTTPS) to use for redirecting the request.

[IDo4axD]

You need to take a more serious approach to setting up Airflow through Keycloak. There’s no documentation, and a lot of errors appear due to the default configurations. There are many issues overall. It would be preferable to use OIDC in newer versions since it’s more stable, or to properly configure and debug the integration with the new manager.

[pierrejeambrun]

You need to rewrite the location header at the proxy level to make sure it starts with https.

You can take a look at proxy_redirect that should solve your problem.

You need to take a more serious approach to setting up Airflow through Keycloak. There’s no documentation, and a lot of errors appear due to the default configurations. There are many issues overall. It would be preferable to use OIDC in newer versions since it’s more stable, or to properly configure and debug the integration with the new manager.

This is all done and maintained mostly by people working on Airflow pro bono and on their free time, the best way to see improvement is to contribute yourself. If you feel like something is not right or need some improvement (documentation, code etc..) feel free to open a PR. That is also the best way to contribute back for the software you get for free. This is especially true for providers that are community maintained, if nobody keeps maintaining them, we might suspend and even remove them at some point.

[IDo4axD]

proxy_redirect won’t help, because this comes from the client — I’m already connecting via HTTPS.
I’ve tried specifying it.
I’ve now enabled http valid_uri on the Keycloak side.
Now I’m facing the need to configure permissions.
According to this documentation:
https://airflow.apache.org/docs/apache-airflow-providers-keycloak/stable/auth-manager/manage/permissions.html

It’s not clear whether I can do this through webserver-config.
I’d prefer not to configure it manually.
I also see there’s a way to define it via a command when launching the service. What’s the best approach here?

Thank you for your help — I didn’t realize the project was community-driven!

[vincbeck]

You need to take a more serious approach to setting up Airflow through Keycloak. There’s no documentation, and a lot of errors appear due to the default configurations. There are many issues overall. It would be preferable to use OIDC in newer versions since it’s more stable, or to properly configure and debug the integration with the new manager.

The best way to improve it is to contribute yourself. Handling all different user cases, user architecture, user configuration is clearly impossible. So if you run into an issue and feel the documentation is not good enough, you are the perfect candidate to make it better. Nobody will do it for you. So please do :) I'll be very happy to review the PR

[vincbeck]

I also see there’s a way to define it via a command when launching the service. What’s the best approach here?

You can run the commands described in the documentation to automatically configure the permissions in Keycloak for you. If you can, run it form the api-server since you have already configured access to Keycloak there

[saaifali]

Hi, I have been facing this issue with my airflow 3 setup behind my enterprise reverse proxy. I see that this will go out with the 3.1.1 release and there is a tag already created for it in the github repo.
When can we expect this to be released?
P.S. Thanks for resolving this issue. I am hoping 3.1.1 will sort out my problems and I can go ahead and deploy this in my enterprise systems.

I would love to contribute to the project as we go ahead!

[pierrejeambrun]

This PR will be part of 3.1.1

@saaifali 3.1.1 is being released as we speak, the final version should be out in the next few days, you can follow along on the airflow dev mailing list is you need more information.