Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support native JupyterHub OAuthenticator in 2i2c-managed hubs #706

Merged
merged 22 commits into from
Sep 29, 2021

Conversation

sgibson91
Copy link
Member

@sgibson91 sgibson91 commented Sep 23, 2021

Summary

This PR is implementing support for the native JupyterHub OAuthenticator to authenticate against GitHub orgs/teams in our hubs. A config matching the thought process laid out here is currently deployed on the Pangeo staging hub and works! 🎉

fixes #625

What has changed?

  • Schema config for config/hubs/ now allows auth0 block to have additional properties (such as enabled: false) and makes the auth0.connection field not required so we don't have to provide it when using JupyterHub OAuthenticator
  • Updates in deployer/hub.py:
    • Explicitly set the JupyterHub.authenticator_class as part of the generated config
    • Check for a partner secrets/config/hubs/*.cluster.yaml file. If one exists, read it in and parse the config to the helm upgrade command. Merging of config happens at the helm level, instead of merging dicts in Python.
  • Updated auth-management.md docs to add process for enabling JupyterHub OAuthenticator for GitHub orgs and teams.
  • Changes in basehub/values.yaml:
    • Where we were explicitly setting the JupyterHub.authenticator_class here before, it is now being set in the deployer as part of the generated config (see above)
  • Added a schema for secrets/config/hubs/*.cluster.yaml files
    • Only the name and config parameters are allowed under the hubs key. Any other config should live in the main config file under config/hubs.

@sgibson91 sgibson91 self-assigned this Sep 23, 2021
@sgibson91 sgibson91 added 🏷️ authentication Enhancement An improvement to something or creating something new. labels Sep 23, 2021
Copy link
Member

@yuvipanda yuvipanda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

\o/ AWESOME. I love the general approach here, +1 to the simplicity of it :)

deployer/hub.py Outdated Show resolved Hide resolved
docs/howto/configure/auth-management.md Outdated Show resolved Hide resolved
docs/howto/configure/auth-management.md Outdated Show resolved Hide resolved
secrets/config/hubs/schema.yaml Show resolved Hide resolved
docs/howto/configure/auth-management.md Outdated Show resolved Hide resolved
Copy link
Member

@choldgraf choldgraf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some quick thoughts from me - in general this looks good! I like the simplicity of it, it is surprisingly not as much effort as I thought it would be :-)

I won't "approve" as I don't understand the implementation details as well

docs/howto/configure/auth-management.md Outdated Show resolved Hide resolved
docs/howto/configure/auth-management.md Outdated Show resolved Hide resolved
docs/howto/configure/auth-management.md Outdated Show resolved Hide resolved
docs/howto/configure/auth-management.md Outdated Show resolved Hide resolved
docs/howto/configure/auth-management.md Show resolved Hide resolved
sgibson91 and others added 3 commits September 27, 2021 11:08
Co-authored-by: Chris Holdgraf <choldgraf@gmail.com>
Co-authored-by: Chris Holdgraf <choldgraf@gmail.com>
Co-authored-by: Chris Holdgraf <choldgraf@gmail.com>
deployer/hub.py Outdated Show resolved Hide resolved
@sgibson91 sgibson91 marked this pull request as ready for review September 27, 2021 12:34
@@ -317,8 +314,6 @@ jupyterhub:
resp['name'] = resp['name'].split('|')[-1]
return resp

c.JupyterHub.authenticator_class = CustomOAuthenticator

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not get this removal... don't you need to keep this in auth0 enabled clusters?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it might be that this line

https://github.com/sgibson91/pilot-hubs/blob/fe4ab0101b6900997b9c1c0fc90e62b1b57e1720/deployer/hub.py#L351

should be the following instead

generated_config['jupyterhub']['hub']['config']['JupyterHub']['authenticator_class'] = 'CustomOAuthenticator'

I wasn't sure and was a bit confused, hence why I asked for my work to be checked in this discussion thread #706 (comment)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure the alternative you wrote in the previous comment will work in your pangeo (non-auth0) case.
What about doing all the stuff under 06-custom-authenticator conditional if auth0 is configured? As it is done here with the scratch_bucket?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it might be that this line

https://github.com/sgibson91/pilot-hubs/blob/fe4ab0101b6900997b9c1c0fc90e62b1b57e1720/deployer/hub.py#L351

should be the following instead

generated_config['jupyterhub']['hub']['config']['JupyterHub']['authenticator_class'] = 'CustomOAuthenticator'

@sgibson91, I believe so too!

At some point, I did this chart about how different configs are put together. I'm not sure if it's super accurate now, but for me, at least it provides a visual starting point, though I'm still a bit confused about them (hence, my forever-taking PR with the staff admin users 🙄 )

One thing that's not accurate in that chart (I think), is that although values.yaml template is the first one that gets loaded, , the extra_config in values.yaml is actually the last one that get loaded by the hub. At least that's what I understand from the hub logs first few lines:

Loading extra config: 01-working-dir
Loading extra config: 02-prometheus
Loading extra config: 03-no-setuid
Loading extra config: 04-custom-theme
Loading extra config: 05-custom-admin
Loading extra config: 06-custom-authenticator
Loading extra config: 07-cloud-storage-bucket

I am not sure the alternative you wrote in the previous comment will work in your pangeo (non-auth0) case.

@damianavila, it shouldn't be set for non-auth0 cases.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @GeorgianaElena! I will update that line and see what people think. Also very nice chart! 😉


@damianavila for clarity's sake, we now need to dynamically set the value of JupyterHub.authenticator_class based on which authentication method we are using, hence why these values are being removed from the chart. So when using auth0, authenticator_class will be set in the deployer in this block:

https://github.com/sgibson91/pilot-hubs/blob/fe4ab0101b6900997b9c1c0fc90e62b1b57e1720/deployer/hub.py#L336-L352

And when using the native GitHubOAuthenticator, we set the class in the *.cluster.yaml hub config, as demonstrated in the accompanying docs, e.g. Step 3 here: https://github.com/sgibson91/pilot-hubs/blob/jupyterhub-oauth-github/docs/howto/configure/auth-management.md#native-jupyterhub-oauthenticator-for-github-orgs-and-teams


What about doing all the stuff under 06-custom-authenticator conditional if auth0 is configured? As it is done here with the scratch_bucket?

I am not convinced that method will work. If I understand the code correctly, the highest level config key that get_config reads in is everything under jupyterhub and the auth0.enabled key is much higher in the hierarchy of keys, adjacent to hubs, so I'm not sure the z2jh module can even see it?

Image to clarify what I mean by the hierarchy:

config

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the authenticator_class for auth0 hubs in b1d6aac

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, thanks @GeorgianaElena and @sgibson91 for the discussion and the clarifications!
I now understand why I did not catch it... I totally missed this line. This is why I proposed the alternative "conditional approach" (without any testing, of course 😉 ).
In fact, I was expecting that auth0 "enabled" property to have some representation in the schema (I guess that is another discussion to have...), and since I did not find it there, and because I did not check some lines above 🤦 , I assumed there was not a conditional approach.
I concur with the last commit you pushed @sgibson91.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I'm not sure the z2jh module can even see it?

Yep, you would need to have somehow an auth under custom as well for being able to catch it... but no need to test that now 😜 .

Copy link
Contributor

@damianavila damianavila left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this one is ready!
Thanks, @sgibson91 for the work you have done here and to bear with my comments/questions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement An improvement to something or creating something new.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support OAuthenticator for our hubs
5 participants