Support separate auth tokens for Onboarding API and MCPs

[andreaskienle]

Current State and Problem

• When a user accesses an MCP, we currently use the same access token for retrieving MCP details as we do for requests to the Onboarding API, assuming both use the same token.
  • ✅ This is true in the DEV environment.
  • ❌ However, in Staging and Canary, MCPs have a different client ID, which means they require a separate access token.
• This results in a 401 authentication error (was reported by @reshnm). The UI then assumes a new authentication flow is required and redirects the user.

Proposal

We need to manage separate tokens:

• One for the Onboarding API (currently stored in the onboarding cookie)
• ✨ One (or more) for the MCP(s)

Currently, the frontend checks ⁠/api/auth/me to determine if the user is authenticated with the Onboarding API. This is done by verifying if there is an access token in the cookie (fastify.get("/auth/me")). If the token is missing, the login screen is displayed.

✨ Before accessing MCP resources, we should also check if the user is authenticated with the MCP API. This could be handled similarly by checking for an MCP-specific access token. If it is missing, the frontend should initiate the login process (or perhaps we can trigger the login automatically? 🤔).

The IdP details for the Onboarding API (such as endpoint and client ID) are provided via environment variables.

✨ To get the IdP details for accessing an MCP, we could retrieve the kubeconfig. The frontend already provides MCP details such as name, project, and workspace (via HTTP x- header attributes), along with the Onboarding API access token (via cookie), so we can use this information.

✨ We can then initiate the OIDC flow to get an access token for the MCP. (This step could be skipped if the IdP and OIDC attributes are identical, as is currently the case in the DEV environment 🤔)

Cookies vs Session

• Currently, our BFF is stateless, and the Onboarding API token is stored in the ⁠onboarding cookie.
• We could store each MCP’s token in its own cookie (or rather, store the combination of IdP and settings such as client ID 🤔). But dynamically registering cookies can be challenging (currently, the ⁠onboarding cookie is registered statically in ⁠secure-session.js).
• Alternatively, we could store exactly two cookies: one for the Onboarding API and one for the MCP. However, this may cause issues when switching between MCPs or working with multiple MCPs in parallel.

→ It may be best to switch to sessions and store tokens in the backend.

(Switching to sessions has the additional benefit of allowing us to refresh access tokens in the BFF in the background, avoiding any possible race conditions.)

How this relates to Multiple IdPs

#143 disables the scenario of multiple IdPs. The approached outlined here, might work also in this multi-IdP scenario. 🤔 Needs further clarification

To Implement

• Switch from cookies to sessions
• Support different auth tokens for MCPs
• Support for live environments (Redis?)

[enrico-kaack-comp]

Outline for a cookie-based, stateless BFF solution:
every IDP gets its own cookie, encrypted by the BFF. This way we dont have a problem with the 4kb size limit.
We use the Path option in the cookie to only send the cookies to the backend that are required for the current auth flow (otherwise with would add more and more cookies to every request --> increase request size).
The frontend tracks the expiration times of its tokens and the refresh flow needs to be triggered by a frontend request against a certain bff endpoint. This way, the frontend can also queue requests if the refresh flow is not yet finished etc..
The bff does not perform automatic token renewal but just response with 401 unauthorized if the token are invalid.

Advantage:

• stateless backend --> less data to secure, less data that can be leaked
• http only cookie cant be read by browser extension/malicious javascript. Only on compromised OS level the values could be extraced but would still be encrypted by server key
• there is no single point where all tokens are stored (compared to storing them in BFF)

Disadvantage:

• probably no out of the box support from plugins in fastify --> more effort

[enrico-kaack-comp]

Detailed outline of session based approach (with encryption extension):

Session storage on bff: Browser only stores a session cookie, the access/refresh token are stored on backend. We can easily store as many tokens as we need.

• less data stored on user device, smaller request size
• no limits on amount of cookies the browser support

Encryption Extension:
To NOT store all users tokens with a common encryption key, we introduce a per-user encryption:
When we receive the tokens from the oidc flow, we generate a random encryption key, store the tokens encrypted and set the encryption key as a cookie (server side encrypted) to the user.
That way, we can only decrypt the user tokens during request time.

• better security of user tokens: we cant misuse them and if our system is compromised we don’t compromise users tokens (comparable to storing hashed and salted passwords instead of cleartext)

This also enables us to do a token refresh in the backend when a user request comes in and the token is expired.

We do not need to backup the servers (in-memory) state as the user can simply login again if the server needs to restart.
When we scale horizontally, we can use normal scaleable state handling solutions like sticky sessions or a redis.

We decided (@ValentinGerlach @n3rdc4ptn @andreaskienle @enrico-kaack-comp ) to progress with this solution to fix this issue and to support custom idps for mcps in the future.