Keycloak provider infinitely redirects after login

[Rafnel]

Apache Airflow Provider(s)

keycloak

Versions of Apache Airflow Providers

apache-airflow-providers-keycloak==0.1.0

Apache Airflow version

3.1.0

Operating System

Ubuntu 24.04.1 LTS

Deployment

Virtualenv installation

Deployment details

Running Airflow in a plain Python virtual environment.

What happened

I recently upgraded my Airflow instance from 2.9 -> 3 and unfortunately Airflow 3 breaks existing custom integrations with Keycloak (#51362), so I decided to try out the new official Keycloak provider per the documentation here: https://airflow.apache.org/docs/apache-airflow-providers-keycloak/stable/auth-manager/setup/config.html .

I installed the provider and updated my airflow.cfg like so as directed by the documentation:

[core]
...
auth_manager = airflow.providers.keycloak.auth_manager.keycloak_auth_manager.KeycloakAuthManager
...
[keycloak_auth_manager]
client_id = MY_KEYCLOAK_AIRFLOW_CLIENT_ID
client_secret = MY_KEYCLOAK_AIRFLOW_CLIENT_SECRET
realm = MY_CLIENT_REALM
server_url = https://my.keycloak.instance.com

Additionally, my Keycloak user has the "Admin" role assigned to it from the MY_KEYCLOAK_AIRFLOW_CLIENT_ID.

I then logged out of Keycloak to start fresh and restarted my Airflow api-server and accessed my Airflow UI.

Upon accessing my Airflow UI, it redirects to the Keycloak login form and I logged into my user account. All good so far.

Next it redirects back to the Airflow instance. Immediately it starts infinitely redirecting in a loop with these logs (replaced UUIDs/other identifying info):

...
INFO:     MY_IP - "GET / HTTP/1.1" 200 OK
INFO:     MY_IP - "GET /ui/config HTTP/1.1" 401 Unauthorized
INFO:     MY_IP - "GET /api/v2/auth/login?next=http%3A%2F%2Fmy.airflow.instance%3A9010%2F HTTP/1.1" 307 Temporary Redirect
INFO:     MY_IP - "GET /auth/login?next=http://my.airflow.instance:9010/ HTTP/1.1" 307 Temporary Redirect
INFO:     MY_IP - "GET /auth/login_callback?state=&session_state=UUID&iss=https%3A%2F%2Fmy.keycloak.instance.com%2Frealms%2FMY_CLIENT_REALM&code=UUID_2.UUID.UUID_3 HTTP/1.1" 303 See Other
INFO:     MY_IP - "GET / HTTP/1.1" 200 OK
INFO:     MY_IP - "GET /ui/config HTTP/1.1" 401 Unauthorized
INFO:     MY_IP - "GET /api/v2/auth/login?next=http%3A%2F%2Fmy.airflow.instance%3A9010%2F HTTP/1.1" 307 Temporary Redirect
INFO:     MY_IP - "GET /auth/login?next=http://my.airflow.instance:9010/ HTTP/1.1" 307 Temporary Redirect
INFO:     MY_IP - "GET /auth/login_callback?state=&session_state=UUID&iss=https%3A%2F%2Fmy.keycloak.instance.com%2Frealms%2FMY_CLIENT_REALM&code=UUID_4.UUID.UUID_3 HTTP/1.1" 303 See Other
...

In the meantime I am going back to using my custom FAB <-> Keycloak auth implementation but it only works for the UI. My custom API auth integration is still broken by Airflow 3 unfortunately, so when I use the API I have to use basic auth with a synthetic Airflow user created via the CLI.

What you think should happen instead

After logging into my Keycloak account with the proper "Admin" role, Airflow should take me to my instance's home page instead of redirecting infinitely.

How to reproduce

1. Run Airflow 3.1.0 inside a virtual environment (with Python 3.10)
2. Install the apache-airflow-providers-keycloak==0.1.0 package
3. Set up your airflow.cfg to point to your Keycloak instance and the proper client id/secret
4. Run Airflow, access the web UI of the instance, get redirected to Keycloak, log into your user, get redirected back to Airflow, and expect infinite redirects to start taking place.

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Rafnel]

I am willing to take a look at the source code and potentially submit a fix (if I can find the bug :) )if the team believes that this is indeed a bug and not a configuration issue on my side.

[eladkal]

if the team believes that this is indeed a bug and not a configuration issue on my side.

cc @vincbeck WDYT?

[vincbeck]

Do you think the reason could be #55143? If you run your environment behind a proxy that has some strict policy regarding cookie, this could be the reason

[Rafnel]

I am running my environment inside a corporate network but I don't know many networking details outside that. My Keycloak and Airflow instances are running inside the same corporate network, both self-hosted.

[vincbeck]

I am running my environment inside a corporate network

That is probably the issue. We are currently working on it and we are close to have it fixed (I think). Let me get back to you once this is fixed.

[Rafnel]

Thanks so much!

[tjedynak]

Hi, @vincbeck look like i have similar issue, do you have any info when fix will be released? Is it any workaround to use native keycloak provider?

[WiktorKarczewski]

same issue here, @vincbeck we are really counting on you!! :)