diff --git a/idp-registration.md b/idp-registration.md new file mode 100644 index 0000000..c40162e --- /dev/null +++ b/idp-registration.md @@ -0,0 +1,330 @@ +# Identity Provider Registration + +## Introduction + +This guide details the process for bootstrapping different layers of the ESGf identity management infrastructure. Specifically, the registration of client applications (*Relying Parties*) with identity management services (*Identity Providers*). We are using [OpenID Connect](https://openid.net/connect/) on top of the OAuth 2 protocol to facilitate the sharing of identities between applications. + +[Keycloak](https://www.keycloak.org/documentation), an open source and highly customisable identity management server, can act as both *Relying Party* and an *Identity Provider*. This is the technology we are using for the main components of the identity management system, the **IDP Proxy** and the organisation-specific **Local IDP** servers. + +The diagram below shows the various components in this architecture. The connections between components are our use-cases: + + + +## Actors + +- **Relying Party (RP):** + An OAuth 2.0 Client application that requires user authentication and claims from an Identity Provider. Relying Parties must register themselves with an Identity Provider to allow this interaction. +- **Identity Provider (IDP)** + This is a trusted source of authority on user identity which communicates with a Relying Party using the OAuth 2.0 protocol. +- **Keycloak:** + Keycloak is an identity management server written in Java. Its documentation can be found [here](https://www.keycloak.org/documentation). +- **Local IDP:** + This is a Keycloak server installed on a node at an ESGF site and acting as an Identity Provider. It would be appropriately branded and provide login and registration for users at that site. +- **IDP Proxy:** + This is a Keycloak server acting as the central identity management service for ESGF. While it facilitates login for ESGF users, it does not allow registration. Instead acting as proxy to multiple Local IDPs (hence "IDP Proxy") which provide it with user identities. This gives it access to all users in the federation, allowing it to be the Identity Provider for all ESGF applications. +- **Commercial IDP:** + An Identity Provider not managed by ESGF but which could be hooked up to the IDP Proxy to provide additional login options for "homeless" users. e.g. GitHub, Google Plus, etc. +- **ESGF Application:** + An application, typically a web application (e.g. COG), which may utilise user account information provided by the OpenID Connect protocol. This would be facilitated by acting as Relying Party to the IDP Proxy using some kind of OpenID Connect authentication library. +- **IDP Administrator:** + A person with access to the administration backend of an Identity Provider. For example, somebody with access to the admin console of a Keycloak instance. This may also be the owner or an administrator of a GitHub organisation. +- **RP Administrator:** + A person with access to the administration backend of a Relying Party. This could be the manager of a web application or the administrator of a Keycloak server that is behaving as a Relying Party. +- **Realm:** + In Keycloak, a realm manages a set of users, credentials, roles, and groups. All interactions with a Keycloak server happen within the context of a realm. +- **Client:** + In Keycloak, clients are entities that can request Keycloak to authenticate a user. All OAuth 2.0 flows happen via a client configured on an Identity Provider's server. +- **Keycloak API:** + The Keycloak Admin REST API provides endpoints to manipulate a Keycloak realm. We are using it to automate the process of client registration. +- **Keycloak API User:** + This is any Keycloak user with authority to use the Keycloak Admin REST API for a given realm. +- **Bearer Token:** + This is a token included in an HTTP Authorization header that authorizes access to a particular resource or set of resources without the sender needing to prove their identity. + +## Use cases + +This document will detail the procedure for the following use-cases: + +1. Registering the ESGF IDP Proxy with an organisation’s Local IDP + + **Relying Party**: Local IDP (Keycloak) + **Identity Provider**: IDP Proxy (Keycloak) + +2. Registering the ESGF IDP proxy with a Commercial IDP + + **Relying Party**: IDP Proxy (Keycloak) + **Identity Provider**: Commercial IDP (e.g. GitHub) + +3. Registering an ESGF application with the ESGF IDP Proxy + + **Relying Party**: An ESGF application (e.g. COG) + **Identity Provider**: IDP Proxy (Keycloak) + +## Setting up access to a Keycloak server’s REST API + +*This section assumes you have access to a **Keycloak** server with an accessible **realm*** + +Before being able to use the API for a Keycloak installation, some setup must be performed by an administrator of that server. + +1. The Keycloak administrator should create a dedicated *API user*. This account should have permission to access the *Keycloak API* for `` on the server. + +2. Access to this *API user* should be granted to the person who wishes to use the API to, for example, register a *Relying Party* (e.g. an *RP administrator*). + + - Keycloak allows you to send a password reset email for a user via the admin interface. This makes it easy to grant accounts to other people. + +3. With the credentials of this *API user*, we can now a request an API access token (*Bearer Token*): + + *Request* + ```http + Endpoint: https://keycloak.example.com/auth/realms//protocol/openid-connect/token + Method: POST + Content-Type: application/x-www-form-urlencoded + + Data: + client_id = admin-cli + grant_type = password + username = + password = + ``` + + *JSON Response* + ```json + { + "access_token": "", + "expires_in": 60, + "refresh_expires_in": 1800, + "refresh_token": "", + "token_type": "bearer", + "not-before-policy": 0, + "session_state": "", + "scope": "profile email" + } + ``` + +4. Save the `` from the response. This is our *Bearer Token*. It can be used until the expiry time (60 seconds, in this case) has passed. + +The *Bearer Token* can now be used to access the Keycloak admin API to register clients or configure IDPs. + +## Case 1: Registering the IDP Proxy with an organisation’s Local IDP + +In this use case, the **Local IDP** is acting as an *Identity Provider* and the **IDP Proxy** is a *Relying Party*. + +The first step is to establish access to the *Keycloak API*: + +1. Fetch a *Bearer Token* from the *Local IDP* for the relevant *Realm*. (see [Setting up access to a Keycloak IDP’s REST API](#setting-up-access-to-a-keycloak-servers-rest-api)) + +Now the *Keycloak API* for the *Local IDP*'s `` can be queried: + +2. Register a new *Client*: + + *Request* + ```http + Endpoint: https://localidp.example.com/auth/realms//clients + Authorization: Bearer + Method: POST + Content-Type: application/json + + JSON: + { + "clientId": "" + } + ``` + + The response will be empty but a new *Client* will have been created on `` + +3. Retrieve the `` using the ``: + + *Request* + ```http + Endpoint: https://localidp.example.com/auth/realms//clients + Authorization: Bearer + Method: GET + + Query: + clientId = + ``` + + *JSON Response* + ```json + [ + { + "id": "", + "clientId": "", + ... + } + ] + ``` + +4. Using ``, retrieve the ``: + + *Request* + ```http + Endpoint: https://localidp.example.com/auth/realms//clients//client-secret + Authorization: Bearer + Method: GET + ``` + + *JSON Response* + ```json + { + "type": "secret", + "value": "" + } + ``` + +Now that the *Client* details have been retrieved, the last stage is to set up the new IDP in Keycloak based on the acquired credentials. This will require querying the *IDP Proxy's* *Keycloak API*. + +5. Fetch a *Bearer Token* from the *IDP Proxy* for the relevant *Realm*. (see [Setting up access to a Keycloak IDP’s REST API](#setting-up-access-to-a-keycloak-servers-rest-api)) + +6. Create a new Keycloak Identity Provider on `` with the `` and ``: + + *Request* + ```http + Endpoint: https://centralidp.example.com/auth/realms//identity-provider/instances + Authorization: Bearer + Method: POST + Content-Type: application/json + + JSON: + { + "alias": "", + "config": + { + "authorizationUrl": "https://localidp.example.com/auth/realms//protocol/openid-connect/auth", + "tokenUrl": "https://localidp.example.com/auth/realms//protocol/openid-connect/token", + "clientAuthentication": "jwt", + "clientId": "", + "clientSecret": "" + } + } + ``` + + The response will be empty but a new *Identity Provider* will have been created on `` + +## Case 2: Registering the ESGF IDP proxy with a Commercial IDP + +In this use case, a **Commercial IDP** is acting as an *Identity Provider* and the **IDP Proxy** is a *Relying Party*. + +The first part of this process, the registration of our client, will differ depending on which provider you are using. Though, most will provide instructions. + +In this example, we are using GitHub as the *Identity Provider*: + +1. On your GitHub organisation's settings page, under "Developer settings", click the "OAuth Apps" button. This will let you register a new OAuth application, which is what we want. + + + +2. The application must be configured with a name, a homepage URL and an authorization callback URL. + + + + The only important thing here is the callback URL, since this will be where users are redirected to once they have authenticated with their GitHub credentials. In our case, the authorization callback URL will be: + + `https://centralidp.example.com/auth/realms//broker/github/endpoint` + +3. Selecting the application, you should be able to see its ID and secret: + + + +Now that the *Client* has been set up and we have the ID and secret, the last stage is to configure a new IDP in Keycloak with these client details. This will require querying the *IDP Proxy's* *Keycloak API*: + +4. Fetch a *Bearer Token* from the *IDP Proxy* for the relevant *Realm*. (see ***Setting up access to a Keycloak IDP’s REST API***) + +5. Create a new Keycloak Identity Provider on `` with the `` and ``: + + *Request* + ```http + Endpoint: https://centralidp.example.com/auth/realms//identity-provider/instances + Authorization: Bearer + Method: POST + Content-Type: application/json + + JSON: + { + "alias": "", + "config": + { + "authorizationUrl": "https://localidp.example.com/auth/realms//protocol/openid-connect/auth", + "tokenUrl": "https://localidp.example.com/auth/realms//protocol/openid-connect/token", + "clientAuthentication": "jwt", + "clientId": "", + "clientSecret": "" + } + } + ``` + + The response will be empty but a new *Identity Provider* will have been created on `` + +## Case 3: Registering an ESGF application with the ESGF IDP proxy + +In this use case, the **IDP Proxy** is acting as an *Identity Provider* and an **ESGF Application** is acting as a *Relying Party*. + +This case is similar to registering the IDP Proxy with a Local IDP, but in this case we are registering with the IDP Proxy. + +The first step is to establish access to the *Keycloak API*: + +1. Fetch a *Bearer Token* from the **IDP Proxy** for the relevant *Realm*. (see ***Setting up access to a Keycloak IDP’s REST API***) + +Now the *Keycloak API* for the **IDP Proxy**'s `` can be queried: + +2. Register a new *Client*: + + *Request* + ```http + Endpoint: https://centralidp.example.com/auth/realms//clients + Authorization: Bearer + Method: POST + Content-Type: application/json + + JSON: + { + "clientId": "" + } + ``` + + The response will be empty but a new *Client* will have been created on `` + +3. Retrieve the `` using the ``: + + *Request* + ```http + Endpoint: https://centralidp.example.com/auth/realms//clients + Authorization: Bearer + Method: GET + + Query: + clientId = + ``` + + *JSON Response* + ```json + [ + { + "id": "", + "clientId": "", + ... + } + ] + ``` + +4. Using ``, retrieve the ``: + + *Request* + ```http + Endpoint: https://centralidp.example.com/auth/realms//clients//client-secret + Authorization: Bearer + Method: GET + ``` + + *JSON Response* + ```json + { + "type": "secret", + "value": "" + } + ``` + +This completes the registration of the ESGF Application's client. Next, the application must be configured to use the **IDP Proxy** as an *Identity Provider*. This configuration will look different depending on what kind of OIDC support you are using. + +As an example, we will assume that we are configuring a Django web application using the `mozilla-django-oidc` authentication backen. Since this configuration requires direct access to the web application's deployment, it can obviously only be done by someone with privileges access to the deployment (i.e. the *RP Administrator*). + +The steps for this are detailed in the [mozilla-django-oidc documentation](https://mozilla-django-oidc.readthedocs.io/en/stable/installation.html). diff --git a/idp_user_accounts.md b/idp_user_accounts.md new file mode 100644 index 0000000..38c731c --- /dev/null +++ b/idp_user_accounts.md @@ -0,0 +1,103 @@ +# IDP Proxy User Sign In and Registration + +The ESGF IDP Proxy functions as a general identity provider for all ESGF sites. Though it does not directly provide password-based login or account registation, it does act as a gateway to other identity providers that users can sign up to and login with. + +This document will outline the user management facilities provided by Keycloak, as well as some of the common use-cases that we are supporting. + +### Actors + +- **Keycloak:** + Keycloak is an identity management server written in Java. Its documentation can be found [here](https://www.keycloak.org/documentation). +- **Relying Party (RP):** + An OAuth 2.0 Client application that requires user authentication and claims from an Identity Provider. Relying Parties must register themselves with an Identity Provider to allow this interaction. +- **Identity Provider (IDP)** + This is a trusted source of authority on user identity which communicates with a Relying Party using the OAuth 2.0 protocol. +- **IDP Proxy:** + This is a Keycloak server acting as the central identity management service for ESGF. While it facilitates login for ESGF users, it does not allow registration. Instead acting as proxy to multiple Local IDPs (hence "IDP Proxy") which provide it with user identities. This gives it access to all users in the federation, allowing it to be the Identity Provider for all ESGF applications. +- **ESGF site-specific IDP:** + This is a Keycloak server installed on a node at an ESGF site and acting as an Identity Provider. It would be appropriately branded and provide login and registration for users at that site. +- **Third-party IDP:** + An Identity Provider not managed by ESGF but which could be hooked up to the IDP Proxy to provide additional login options for "homeless" users. e.g. GitHub, Google Plus, etc. +- **External IDP:** + Any IDP being relied on to provide an identity. e.g. A third-party or ESGF site-specific IDP. + +## Keycloak Functionality Overview + +Except where specified, the information in this section applies to both the ESGF IDP Proxy and site-specific ESGF IDP Keycloak servers. + +### Themes + +All Keycloak view templates can be edited or overridden with a custom theme. This allows for full customisation of the look-and-feel of Keycloak, as well as the content of each page. + +The IDP Proxy uses an ESGF styled theme to override much of the content of the default Keycloak views. For instance, it does not allow user's to login with a username and password. Instead, they authenticate with another ESGF IDP or with a third-party IDP. Site-specific ESGF IDPs retain the standard Keycloak functionality but alter the appearance with CSS. + +### Registration + +This is the process by which a visitor to the Keycloak server can enter their personal details to generate an account in a Keycloak server's database. Registration, by default, prompts for the following information: + +- First name (required) +- Last name (required) +- Email (required, unique) +- Username (required, unique) - Only if “Email as username” is not enabled. +- Password (required) + +The registration form contents can be modified in the relevant theme template. Additional information about the user can be requested this way by adding custom fields mapped to *user attributes* (explained in the next section). + +User registration is disabled on the IDP Proxy. Instead, a user record is created in the database when a user authenticates via an external IDP. The fields for this user are populated from their OIDC profile. + +### User Accounts + +Account information is stored as a row in the Keycloak database’s user table. + + - Each row in the user table of the database is automatically assigned a randomly generated unique ID, a realm ID, a timestamp, etc. + - Passwords are stored in a separate table. + - If an IDP is used to authenticate, a new user account is created. A many-to-many linking table is used to associate users with IDPs. + - If the “Email as username” realm option is enabled, the user’s username field will match their email address. + - When a new user authenticates with an external IDP, a row will be created in the user database from the `preferred_username`, `email`, `given_name` and `family_name` attributes of the OIDC ID token / UserInfo response. + +*User attributes* can be used to associate additional information with an account. + + - Attributes are arbitrary key/value pairs stored in their own table in the database and associated with users by ID. + - Attribute values can be provided to a relying party by adding them to a scope. + +Keycloak has a profile page, which users can access to change their account details. This also applies to federated users at the IDP Proxy, who would be able to alter their account details to be different to those of their home IDP. + +## Case 1: New User Registration and Login + +When a new user arrives at the IDP Proxy, they will be presented with a selection of IDPs. They will have two options: + +1. Sign in with a third party IDP such as GitHub, Google Plus, etc. + + - Selecting one of these providers will send them to the third party site's login page where they will be able to login with an existing account or create a new a account to login with. The external IDP will then communicate their account information to the IDP Proxy using OIDC, completing their registration and login at the IDP Proxy. + +2. Sign in with one of the available site-specific ESGF IDPs such as LLNL, CEDA, etc. + + - The user will be redirected to the login page of their chosen site's customised Keycloak IDP, where they will be able to register a new account (as detailed in the above section) and login. Their information will then be communicated with the IDP Proxy using OIDC, completing their registration and login at the IDP Proxy. + +Once an approved IDP has been used to register and authenticate at the IDP Proxy, the user's authenticated status and saved details can be passed on to a relying ESGF web application or service to complete their login. + +## Case 2: Legacy User Password Reset + +Users that had an account at an ESGF institution before the migration to Keycloak should be familiar with the available site-specific IDPs shown at the IDP Proxy login screen. + +However, once they are directed to their chosen IDP, they will not be able to login using their old password. This is because the ESGF user database migration did not transfer ESGF user passwords. Therefore, legacy users will have to reset their password at their organisation's Keycloak IDP before they can login. + +After they have reset their password and logged in, their old details will be saved to the IDP Proxy and authentication will be complete. + +## Case 3: Login with multiple IDPs + +There will be situations where users who have authenticated with one IDP in the past attempt to authenticate with a second IDP. Since the IDP Proxy creates a user record in its database when somebody first logs in, there must be a way to determine that a new user with the same email address or username as an existing user is the same person as that user. + +Fortunately, this is a fairly ordinary scenario for sites with external IDP providers, and Keycloak's solution should is quite simple: If either the `username` or the `email address` of a newly authenticated user matches that of user already in the database, the IDP Proxy will give you two options: + +1. Link the new user account with the one in the database. + + - This will redirect you back to the login page, where it expects you to log in to the proxy with the account that has the conflicting details (which may be the username or the email address, but only the one prompted for matters). Once logged in, that account will be associated with your new IDP, in addition to any others it might have been linked to. User details of the account are unchanged. + + - As implied by the above, in the scenario where the details of a user on two IDPs are different (e.g. a user with different usernames per organisation but the same email address, or vice versa) the proxy will only store the relevant details of the first account to authenticate. + +2. Change your email/username on the proxy + + - The proxy will let you change the details of your “new” proxy account before it is created. In other words, you will be able to change the username or email address to something that doesn’t conflict with an account already present on the proxy. + + - If the proxy is using the “Email as username” realm option, then the above flow will ignore “preferred_username” and simply use the user’s email address instead. With this enabled, account linking functions the same as described above, but without needing to compare usernames. diff --git a/media/images/github-oauth-application-setup.png b/media/images/github-oauth-application-setup.png new file mode 100644 index 0000000..9faf17c Binary files /dev/null and b/media/images/github-oauth-application-setup.png differ diff --git a/media/images/github-oauth-client-details.png b/media/images/github-oauth-client-details.png new file mode 100644 index 0000000..35f88b3 Binary files /dev/null and b/media/images/github-oauth-client-details.png differ diff --git a/media/images/github-oauth-register-an-application.png b/media/images/github-oauth-register-an-application.png new file mode 100644 index 0000000..74b1141 Binary files /dev/null and b/media/images/github-oauth-register-an-application.png differ diff --git a/media/images/idp-proxy-architecture.png b/media/images/idp-proxy-architecture.png new file mode 100644 index 0000000..62e1850 Binary files /dev/null and b/media/images/idp-proxy-architecture.png differ