Set up JWT token authentication in Fast APIs

[vincbeck]

Goal

Set-up authentication using JWT token in UI API. The UI API should be protected hy authentication using JWT token. This PR creates the logic behind JWT token authentication.

What is done?

• Authentication logic using JWT token

What has been done but is not in this PR?

• Authentication done for the 2 endpoints in airflow/api_fastapi/views/public/dags.py. I am aware this is the public API and not the UI API but this is just for testing.
• Manual testing using these 2 endpoints

The reason why I did not include these changes in the PR is we need first the UI to pass down the token to the API. Otherwise all APIs will fail. I'll create follow-up PRs once the new UI handle the token.

What is not done?

The actual JWT auth-backend. This can be done in a separate PR once this one is merged so for the sake of keeping a PR as small as possible, I'll do it in a separate PR

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[vincbeck]

I think we want to deprecate and then remove the session auth backend?

[pierrejeambrun]

Really nice 👍 This is what I had in mind I think it's worth continuing efforts on that. Thanks Vincent for this work :)

(There's a small formatting indent space thing for generated files, but that's not important at this point)

[pierrejeambrun]

UI API use JWT token authentication only (no way to configure it from the user)?
Public API uses the auth backends specify in auth_backends? We want to add JWT token as possible auth backend (separate PR)

I would say yes. The front end uses JWT Token for accessing both Public and Private API. (because the front-end will call both APIs, so basically JWT should always be in the auth.backends at least for the UI to work, otherwise it can be modified but the front-end using JWT won't work.

Yes public API can specify additional backends if needed by the user. (But the front-end is always using the JWT one). This will be for custom front-end or API calls made by other clients that desieres an other authentication schema.

To summarize my idea:

• Public API takes 'auth_backends` iterates on it and check if one of them has auth (similar to what we have for the legacy one)
• Private/Public API only needs to support JWT auth backend (but making a difference between public and private might actually be more work, doing the same as what we have for the public one might be easier, but is not required)
• Front end only uses JWT security schema to communicate with the backend for both public and UI API
• Other clients of the API uses any configured auth schema to authenticate to the public API.

If that's too much work, we can at first only support JWT Oauth for the new API (both public and UI). And add more backends in 3.x. I don't know how critical it is to have that in the initial airflow 3 release.

[vincbeck]

@ashb @potiuk Question for you. Since we want to create a new major version of FAB provider that would only be compatible with Airflow 3, I do not need to work on the compat tests failures right? They are failing because I introduced some breaking changes in the auth manager interface so the FAB auth manager is no longer compatible with the auth manager interface from Airflow 2.X

[vincbeck]

Ready for review @pierrejeambrun

[pierrejeambrun]

Looking good, thanks

[vincbeck]

If that's too much work, we can at first only support JWT Oauth for the new API (both public and UI). And add more backends in 3.x. I don't know how critical it is to have that in the initial airflow 3 release.

I have been trying and experimenting to support auth backends in the public API in FastApi and I am still unsure it is possible to support them in FastApi. session, basic_auth and kerberos are heavily dependent on Flask and require a Flask application created in order to work. So I am wondering if JWT should not be the only way to authenticate to UI and public API.

If we go that direction we would have to create an API to create a token. Here is an example of flow:

• As a user, I want to use the public API, to do that I need a JWT token
• I call an api like POST /public/token with authentication information as part of the body (e.g. username and password)
• If the authentication succeeds, the API returns the JWT token
• With this token I can call the public API

[pierrejeambrun]

Yes, we can use a flask server as our airflow authentication server. FastAPI or any other service that want to auth on airflow can use it.

It will also be useful at first to still have a flask server somewhere for old plugins backward compatibility.