Skip to content

Conversation

@vincbeck
Copy link
Contributor

@vincbeck vincbeck commented Mar 12, 2025

Set the default auth manager as simple auth manager. This will help us to remove Flask dependencies from airflow-core.

Breeze changes

This PR also adds a parameter --auth-manager to breeze start-airflow to make it easier to switch between auth managers.

Example:

breeze start-airflow --auth-manager FabAuthManager

By default it uses SimpleAuthManager. I also predefine two users admin and viewer for SimpleAuthManager ready to be used out of the box.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@jedcunningham
Copy link
Member

We probably need to do breeze at the same time. I'd be in favor of turning on all_admins by default there. But otherwise, we'd at least need to provision the admin user with a known password vs generating it.

@vincbeck
Copy link
Contributor Author

We probably need to do breeze at the same time. I'd be in favor of turning on all_admins by default there. But otherwise, we'd at least need to provision the admin user with a known password vs generating it.

Turning on the option all_admins in breeze seems reasonable to me

@vincbeck
Copy link
Contributor Author

Any other opinion @potiuk, @ashb ?

@potiuk
Copy link
Member

potiuk commented Mar 13, 2025

Any other opinion @potiuk, @ashb ?

Yes. But It would be great to have a simple way to set "normal" / FAB configuraiton for Breeze. If only to test it.

@vincbeck
Copy link
Contributor Author

Any other opinion @potiuk, @ashb ?

Yes. But It would be great to have a simple way to set "normal" / FAB configuraiton for Breeze. If only to test it.

You mean to add a --auth-manager parameter to breeze start-airflow?

@potiuk
Copy link
Member

potiuk commented Mar 13, 2025

yep. and maybe a way to have simple auth manager somewhat configurable (hard-coded configuraiton that can be used with different users ?)

@vincbeck
Copy link
Contributor Author

yep. and maybe a way to have simple auth manager somewhat configurable (hard-coded configuraiton that can be used with different users ?)

I could create by default 2 users:

  • admin
  • viewer

Or even more?

I can also generate friendly passwords to avoid having to look at generated passwords (only for breeze of course)

@vincbeck vincbeck force-pushed the vincbeck/simple_am_default branch from 8612094 to 42cb7bc Compare March 13, 2025 15:06
@vincbeck
Copy link
Contributor Author

vincbeck commented Mar 13, 2025

I added the --auth-manager parameter to breeze start-airflow. I'll create predefined users in SimpleAuthManager. To do that, I need to create the file /files/simple_auth_manager_passwords.json.generated in Docker. Any hint where it is best to do that @potiuk ?

@potiuk
Copy link
Member

potiuk commented Mar 13, 2025

I added the --auth-manager parameter to breeze start-airflow. I'll create predefined users in SimpleAuthManager. To do that, I need to create the file /files/simple_auth_manager_passwords.json.generated in Docker. Any hint where it is best to do that @potiuk ?

Maybe better to create it elsewhere (/somewhere in root) dynamically in entrypoint_ci.sh when certain flag is passed? Then it will not be persistent

@potiuk
Copy link
Member

potiuk commented Mar 13, 2025

Or we can keep it in a repo and simply pass an env variable to point to it?

@potiuk
Copy link
Member

potiuk commented Mar 13, 2025

The latter likely better

@vincbeck
Copy link
Contributor Author

I like that too :) I'll go with that

@vincbeck
Copy link
Contributor Author

Oh but we already have a config simple_auth_manager_passwords_file. We can use that

@vincbeck vincbeck force-pushed the vincbeck/simple_am_default branch from 42cb7bc to 8f19d60 Compare March 13, 2025 16:57
@vincbeck vincbeck marked this pull request as ready for review March 13, 2025 16:57
@vincbeck vincbeck force-pushed the vincbeck/simple_am_default branch 8 times, most recently from 27e0567 to f1cbe32 Compare March 14, 2025 16:26
@vincbeck vincbeck marked this pull request as draft March 14, 2025 21:59
@vincbeck vincbeck force-pushed the vincbeck/simple_am_default branch from f3d430e to 57c87f3 Compare March 17, 2025 13:14
@vincbeck vincbeck marked this pull request as ready for review March 17, 2025 14:17
@vincbeck
Copy link
Contributor Author

All tests are passing 🟢

Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall. Just one question will that also impact airflow standalone ? It looks like it by the config update but I'm not sure about that. (At first I thought this was only for development purpose in breeze)

@vincbeck
Copy link
Contributor Author

LGTM overall. Just one question will that also impact airflow standalone ? It looks like it by the config update but I'm not sure about that. (At first I thought this was only for development purpose in breeze)

It will. We set simple auth manager as default auth manager in airflow config so airflow standalone (unless it overrides the auth manager but I do not think so?) and any other installation of Airflow will have simple auth manager as default.

I agree this is a bit weird cause the simple auth manager is only for development purposes so that means forcing users to switch to another auth manager for production purposes. But we also agreed on that plan to make the simple auth manager by default in Airflow so that we no longer have FAB dependency in Airflow by default. Also, since we might want to deprecate/remove/discourage users of using FAB auth manager in Airflow 3.X as some point, we cannot make it the default one. We would need to wait for Airflow 4.0 to change that, we do not want that. So having for now the simple auth manager as default and telling users to use another one for production purposes is probably the most future safe.

@vincbeck vincbeck force-pushed the vincbeck/simple_am_default branch from 57c87f3 to 75e2bdf Compare March 19, 2025 13:56
Copy link
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One NIT about redirecting all to /dev/null on user creation. ... Even if we need it in case create user does not work and we want to silence it in case of SimpleAuth Manager we probably should do it differently, maybe simply handle user creation silently and print out "No user creation in Simple Auth Manager" and do it in sdout and treat it as success?

@vincbeck
Copy link
Contributor Author

One NIT about redirecting all to /dev/null on user creation. ... Even if we need it in case create user does not work and we want to silence it in case of SimpleAuth Manager we probably should do it differently, maybe simply handle user creation silently and print out "No user creation in Simple Auth Manager" and do it in sdout and treat it as success?

I agree with the concern, I did not like it either when I was working on it but I could not find a better solution. The problem is we are using airflow users in different place and this CI command is provided by FAB auth manager. So when simple auth manager is used, this command fails.

maybe simply handle user creation silently and print out "No user creation in Simple Auth Manager" and do it in sdout and treat it as success?

I dont think we should do that because it means that every auth manager then should also have this CLI implemented because this command is used in the CI. Unless we agree that only FAB and simple auth managers are used in the CI and then they should have this command implemented? I am okay with that

@potiuk
Copy link
Member

potiuk commented Mar 19, 2025

I agree with the concern, I did not like it either when I was working on it but I could not find a better solution. The problem is we are using airflow users in different place and this CI command is provided by FAB auth manager. So when simple auth manager is used, this command fails.

Let's check if Simple Auth Manager is configured when the command start, print a message that we are skipping it and exit(0).

@potiuk
Copy link
Member

potiuk commented Mar 19, 2025

We can do it in CI in the scripts that run that command - not in the command itself :

if [[ ! <SIMPLE AUTH MANAGER CONFIGURED>> ]]; then
   echo "Skipping  user creation as Simple auth manager is used"
else
  create user ....
fi

Yes. I know it's bash, but it's not too complex :) and it's done in a number of cases alreaady that we do stuff conditionally in the scripts.

(you need to add the right condition of course)

@vincbeck
Copy link
Contributor Author

We can do it in CI in the scripts that run that command - not in the command itself :

if [[ ! <SIMPLE AUTH MANAGER CONFIGURED>> ]]; then
   echo "Skipping  user creation as Simple auth manager is used"
else
  create user ....
fi

Yes. I know it's bash, but it's not too complex :) and it's done in a number of cases alreaady that we do stuff conditionally in the scripts.

(you need to add the right condition of course)

I like that. Let me try

@vincbeck vincbeck force-pushed the vincbeck/simple_am_default branch from 75e2bdf to 09af0b0 Compare March 19, 2025 15:11
@vincbeck
Copy link
Contributor Author

I did that. Let's see if tests are passing

@potiuk
Copy link
Member

potiuk commented Mar 19, 2025

I did that. Let's see if tests are passing

Perfect!

Copy link
Contributor

@bugraoz93 bugraoz93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Great reload and clear logic

@vincbeck
Copy link
Contributor Author

I did that. Let's see if tests are passing

Perfect!

Should I count it as an approval @potiuk ? :)

Copy link
Collaborator

@aritra24 aritra24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits

@vincbeck vincbeck force-pushed the vincbeck/simple_am_default branch from 09af0b0 to bf5169e Compare March 19, 2025 18:41
@vincbeck vincbeck merged commit 8242ba5 into apache:main Mar 20, 2025
89 checks passed
@vincbeck vincbeck deleted the vincbeck/simple_am_default branch March 20, 2025 13:15
agupta01 pushed a commit to agupta01/airflow that referenced this pull request Mar 21, 2025
shubham-pyc pushed a commit to shubham-pyc/airflow that referenced this pull request Mar 22, 2025
nailo2c pushed a commit to nailo2c/airflow that referenced this pull request Apr 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants