[ENH] - Make JupyterHub use groups and roles from Keycloak #2308
Comments
|
Thanks for the extra details!
From a quick glance it looks that is only set on |
Sounds reasonable to me. |
|
Looking at the codebase, I see that the authenticator class is set in nebari here: I also see that there is a # Authenticate users with Native Authenticator
c.JupyterHub.authenticator_class = "nativeauthenticator.NativeAuthenticator"
# Allow anyone to sign-up without approval
c.NativeAuthenticator.open_signup = TrueIs this for testing only, or does it take precedence over the one from nebari? |
Yes, only for testing. That's an example |
|
Ok, populating groups is rather easy with the latest (not yet released) GenericOAuthenticator = {
+ manage_groups = true
client_id = module.jupyterhub-openid-client.config.client_id
client_secret = module.jupyterhub-openid-client.config.client_secret
oauth_callback_url = "https://${var.external-url}/hub/oauth_callback"
authorize_url = module.jupyterhub-openid-client.config.authentication_url
token_url = module.jupyterhub-openid-client.config.token_url
userdata_url = module.jupyterhub-openid-client.config.userinfo_url
login_service = "Keycloak"
- username_key = "preferred_username"
+ username_claim = "preferred_username"
claim_groups_key = "roles"
allowed_groups = ["jupyterhub_admin", "jupyterhub_developer"]
admin_groups = ["jupyterhub_admin"]
- tls_verify = false
+ validate_server_cert = false
}(we should probably toggle Roles are a bit more tricky and will require actually overriding the |
|
Here are details on how the API responses look like with That results in: Then for [
{
"admin": true,
"groups": [
"grafana_developer",
"query-users",
"manage-identity-providers",
"manage-clients",
"manage-account",
"manage-realm",
"view-profile",
"argo-admin",
"dask_gateway_developer",
"grafana_admin",
"view-identity-providers",
"jupyterhub_admin",
"view-realm",
"view-authorization",
"jupyterhub_developer",
"view-clients",
"query-groups",
"conda_store_developer",
"view-events",
"query-realms",
"impersonation",
"realm-admin",
"create-client",
"conda_store_superadmin",
"argo-viewer",
"argo-developer",
"manage-events",
"grafana_viewer",
"manage-users",
"dask_gateway_admin",
"manage-account-links",
"manage-authorization",
"query-clients",
"view-users",
"conda_store_admin"
],
"pending": null,
"auth_state": null,
"kind": "user",
"server": "/user/mike/",
"roles": [
"user",
"admin"
],
"name": "mike"
}
]And for [
{
"properties": {},
"roles": [],
"name": "grafana_developer",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "query-users",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "manage-identity-providers",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "manage-clients",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "manage-account",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "manage-realm",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "view-profile",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "argo-admin",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "dask_gateway_developer",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "grafana_admin",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "view-identity-providers",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "jupyterhub_admin",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "view-realm",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "view-authorization",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "jupyterhub_developer",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "view-clients",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "query-groups",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "conda_store_developer",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "view-events",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "query-realms",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "impersonation",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "realm-admin",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "create-client",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "conda_store_superadmin",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "argo-viewer",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "argo-developer",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "manage-events",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "grafana_viewer",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "manage-users",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "dask_gateway_admin",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "manage-account-links",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "manage-authorization",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "query-clients",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "view-users",
"kind": "group",
"users": [
"mike"
]
},
{
"properties": {},
"roles": [],
"name": "conda_store_admin",
"kind": "group",
"users": [
"mike"
]
}
] |
Well, it looks like targeting the very outdated version we have, while possible, may not be worth it because the divergence in codebase is significant as accummulated over two years since it was not updated. |
|
Currently JupyterHub roles have to be defined at configuration time. There is an issue proposing to allow roles to be configured at runtime: There is a (stale?) PR adding a REST API for runtime role creation: But possibly more handy would be implementing @aktech can we pre-define a set of roles and only use Keycloak to get the user-role association (for the predefined roles) or do we need to be able to get arbitrary roles from Keycloak? If we need arbitrary roles the way forward is to fetch the roles from Keycloak at JupyterHub configuration (or contribute upstream, e.g. the I infer that fetching from Keycloak at JupyterHub config time should is feasible as Keycloak starts up before JupyterHub gets setup: Lines 35 to 38 in a06fcc5 |
agreed, makes sense.
I believe that'll do for now as long as its dynamic, as in roles association show up realtime if there are any changes to the roles association in the keycloak, it doesn't require jupyterhub to restart to show up in the api.
Not urgent from app sharing point of view, we can definitely target that later.
If a groups is deleted in keycloak, is that reflected in the JupyterHub immidiately? |
No. Currently the user needs to logout and login back for it to be reflected. However, we can set:
It might be possible to configure keycloak to send a REST API request to JupyterHub to trigger the refresh. There is an endpoint for removing a user from a group and for removing a group altogether, but there are no corresponding endpoints for roles (but there is a draft PR for it). |
I think this is reasonable for our use case, the alternatives are not feasible. |
|
Making a call to JupyterHub API, on: {
"roles": [
"admin",
"user"
],
"last_activity": "2024-04-03T13:47:46.510679Z",
"server": null,
"pending": null,
"admin": true,
"groups": [],
"created": "2024-03-14T17:06:47.354116Z",
"name": "akumar@quansight.com",
"kind": "user",
"auth_state": {
"access_token": "<SANITIZED>",
"refresh_token": "<SANITIZED>",
"oauth_user": {
"sub": "<SANITIZED>",
"email_verified": false,
"roles": [
"jupyterhub_admin",
"jupyterhub_developer",
"dask_gateway_developer",
"grafana_viewer",
"argo-viewer",
"conda_store_developer",
"manage-account",
"manage-account-links",
"view-profile"
],
"name": "Amit Kumar",
"groups": [
"/analyst"
],
"jupyterlab_profiles": [
"Small Instance"
],
"preferred_username": "akumar@quansight.com",
"given_name": "Amit ",
"family_name": "Kumar",
"email": "akumar@quansight.com"
},
"scope": [
"profile",
"email"
]
},
"servers": {}
}I see the keycloak roles and groups are present in:
I found this while investigating how dask_gateway permissions work, If the structure of the response is similar for any other authenticator besides keycloak (which needs investigation), then we might just be fine, using the groups and roles from |
Well this might not work out of the box, as for everything we need to be able to map them to jupyterhub roles/groups too. Like for example: If an admin creates a role on keycloak that says a user has the ability to share a server, then that needs to be added in jupyterhub to actually have the permissions, equivalent to: c.JupyterHub.load_roles = [
{
"name": "user",
"scopes": ["self", "shares!user", "read:users:name", "read:groups:name"],
},
]This also means roles are not just a string, it could be an object. Which can be defined in keycloak as: name as the name of role in keycloak and scopes as role attributes. |
So in OAuth this gets selected using |
Right, so we will need to pass the role attributes from Keycloak via oauth so that they are accessible in |
Yep, we may have some pre-defined roles but mostly we want to import from keycloak, this gives the most flexibility in terms of customisation, as different deployments (at different orgs) might need different set of permissions (roles) for different set of users/groups. |
|
The PR implementing managed roles in JupyterHub was merged today and will be included in JupyterHub 5.0. |
|
Awesome, that's great news! Is this one: jupyterhub/jupyterhub#3858 getting closed completely? I see it referenced in your PR. |
|
I think it may stay open as it lists a number of other ideas like managing roles via REST API, or allowing users to grant roles (I think less needed now given that we have share codes). |
|
Ah, I see. After your PR, are we able to dynamically update roles (like sync from keycloak), without restarting hub? |
|
Yes. |
viniciusdc
added
type: enhancement 💅🏼
github-project-automation
bot
moved this from New 🚦
to Done 💪🏾
in 🪴 Nebari Project Management
Feature description
Until now we haven't been using JupyterHub groups and roles much. We have Keycloak as the identity provider and we plan to use groups and roles more in keycloak for permissions overhaul, see following issues
The main motivation for this is to be able to fetch groups and roles from the JupyterHub API in jhub-apps to be able to decide permissions, since jhub-apps is not supposed to be tied to Nebari, hence would be great to be able to fetch roles and groups from JupyterHub API in jhub-apps.
Relevant links
I reckon, we might have to make changes to our Authenticator to make this happen.
nebari/src/_nebari/stages/kubernetes_services/template/modules/kubernetes/services/dask-gateway/files/gateway_config.py
Line 63 in 5319447
Definition of done:
https://jupyterhub.readthedocs.io/en/stable/reference/rest-api.html#/default/get_groups
Currently, this is what I get on the fetching groups from JupyterHub:
{ "last_activity": "2024-03-11T15:59:01.194646Z", "server": null, "groups": [], "created": "2022-12-07T16:43:29.002132Z", "auth_state": null, "name": "akumar@quansight.com", "kind": "user", "pending": null, "admin": true, "roles": [ "user", "admin" ], "servers": {} "session_id": null, "scopes": ["truncated"] }You can see the groups are empty and roles are also not the ones from keycloak.
Value and/or benefit
This will help us implement app sharing and permissioning seamlessly with keycloak.
Anything else?
No response
The text was updated successfully, but these errors were encountered: