Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OIDC: add resource url-parameter for self-hosted ADFS #19318

Closed
eumikhailov opened this issue Dec 5, 2023 · 25 comments · Fixed by #19566
Closed

OIDC: add resource url-parameter for self-hosted ADFS #19318

eumikhailov opened this issue Dec 5, 2023 · 25 comments · Fixed by #19566

Comments

@eumikhailov
Copy link
Contributor

Hi there,

I encountered a problem when setting up OIDC auth provider over ADFS 2019, same as described here.

Nomad version

v1.6.x

Issue

There is no way to set up resource url-parameter for for self-hosted ADFS as OIDC provider.

Proposals for implementation

It can be done via the optional parameter OIDCResource in the Nomad OIDC config and optional config method WithResource in cap module.

@tgross
Copy link
Member

tgross commented Dec 5, 2023

Hi @eumikhailov! Just to clarify acronyms here, self-hosted ADFS is "OIDC for Microsoft Active Directory", right? Any chance you have a pointer to Microsoft documentation on this parameter? It looks like this is a non-standard part of the Auth Request, according to the specs linked from the cap code you pointed to. See https://openid.net/specs/openid-connect-core-1_0.html#AuthRequest.

A quick survey suggests that some OIDC libraries provide a way to add arbitrary query parameters. I imagine if there are other providers with non-standard parameters, we would want to add a withResource rather than a withQueryParam. If necessary, we'd need to land that upstream in cap before it could appear in Nomad.

@tgross tgross self-assigned this Dec 5, 2023
@tgross tgross changed the title There is no way to set up resource url-parameter for for self-hosted ADFS as OIDC provider OIDC: add resource url-parameter for self-hosted ADFS Dec 5, 2023
@eumikhailov
Copy link
Contributor Author

HI @tgross!

Yes, self-hosted ADFS is OIDC for Microsoft Active Directory.
Some docs about is there:

According MS docs resource param is optional.

If no resource param is specified, AccessToken will have invalid aud 😔 This problem is well known, good repros are there and there.

On Nomad frontend /v1/acl/oidc/complete-auth error is

failed to retrieve the user info claims: Provider.UserInfo: provider UserInfo request failed: 401 Unauthorized: 

@tgross
Copy link
Member

tgross commented Dec 5, 2023

From the docs you linked:

Note – If using MSAL client library, then resource parameter isn't sent. Instead the resource url is sent as a part of the scope parameter: scope = [resource url]//[scope values e.g., openid]
If resource isn't passed here or in scope, AD FS uses a default resource urn:microsoft:userinfo. userinfo resource policies such as MFA, Issuance or authorization policy, can't be customized.

You should be able to add an OIDCScope to cover this in the auth method.

@eumikhailov
Copy link
Contributor Author

Hi @tgross!

Thank You for feedback!

We have found problem with our version of ADFS - https://serverfault.com/a/876784.
Now we will try to update it to 2019+ and will try scope again.

You can close this issue.

@tgross
Copy link
Member

tgross commented Dec 6, 2023

Thanks!

@tgross tgross closed this as completed Dec 6, 2023
@eumikhailov
Copy link
Contributor Author

Hi @tgross!

New version of Microsoft ADFS have same problem - access_token has client_id in aud and this value cannot be used for userinfo.

We got 401 error from ADFS:

Received invalid UserInfo request. Audience <cleint_id>' in the access token is not same as the identifier of the UserInfo relying party trust 'urn:microsoft:userinfo'

And 500 error from Nomad /v1/acl/oidc/complete-auth:

failed to retrieve the user info claims: Provider.UserInfo: provider UserInfo request failed: 401 Unauthorized: 

Can You explain me please why UserInfo is required or is there any option to disable UserInfo loading?

if userTokenSource := oidcToken.StaticTokenSource(); userTokenSource != nil {

@eumikhailov
Copy link
Contributor Author

If we use scope = urn:microsoft:userinfo we cannot set up custom claims in token such as roles, groups etc.

So I think option for disable UserInfo loading is required for MS ADFS as OIDC provider.

@tgross
Copy link
Member

tgross commented Dec 6, 2023

Did you add an OIDCScope field to cover the resource parameter, as I mentioned previously?

@eumikhailov
Copy link
Contributor Author

@tgross yes, I did and aud claim was changed from urn:microsoft:userinfo to resource which not applicable for UserProfile in ADFS 😔

@eumikhailov
Copy link
Contributor Author

And if scope is not set we cannot set up custom claims in token such as roles, groups etc.

@tgross
Copy link
Member

tgross commented Dec 6, 2023

@eumikhailov can you share your auth config please? That'll help us narrow down what needs to be done here.

@eumikhailov
Copy link
Contributor Author

eumikhailov commented Dec 6, 2023

@tgross yes, of course.

  1. Config with scope option.
{
  "OIDCDiscoveryURL": "https://<ADFS_FQDN>/adfs",
  "OIDCClientID": "https://<NOMAD_FQDN>",
  "OIDCScope" : ["https://<NOMAD_FQDN>"],
  "BoundAudiences": ["https://<NOMAD_FQDN>"],
  "AllowedRedirectURIs": [
    "https://<NOMAD_FQDN>:4646/ui/settings/tokens"
  ],
  "ListClaimMappings": {
    "roles": "roles"
  }
}

In access_token we have claim aud = https://<NOMAD_FQDN>, in id_token we have claim roles = [ .. ] and claim aud = https://<NOMAD_FQDN>.
This acces_token is not acceptable for https://<ADFS_FQDN>/adfs/userinfo due to 401 error

Audience 'https://<NOMAD_FQDN>' in the access token is not same as the identifier of the UserInfo relying party trust 'urn:microsoft:userinfo'
  1. Config without scope option.
{
  "OIDCDiscoveryURL": "https://<ADFS_FQDN>/adfs",
  "OIDCClientID": "https://<NOMAD_FQDN>",
  "OIDCScope" : [ ],
  "BoundAudiences": ["https://<NOMAD_FQDN>"],
  "AllowedRedirectURIs": [
    "https://<NOMAD_FQDN>:4646/ui/settings/tokens"
  ],
  "ListClaimMappings": {
    "roles": "roles"
  }
}

In access_token we will have aud = urn:microsoft:userinfo, in id_token we have not claim roles (custom claims is not configurable for urn:microsoft:userinfo) and claim aud = https://<NOMAD_FQDN>.
UserInfo works fone, but roles is empty, we cannot find binding-rule.

@eumikhailov
Copy link
Contributor Author

@tgross Hi!

I think there is only one way to fix this - disable UserInfo loading by ODIC option.
https://stackoverflow.com/a/45762227
image

DuendeArchive/IdentityServer4#812 (comment)
image

Should we change issue title to feature request OIDCDisableLoadUserInfo optional flag in OIDC config?

@tgross
Copy link
Member

tgross commented Dec 7, 2023

@eumikhailov let me review this and get back to you.

@eumikhailov
Copy link
Contributor Author

Hi @tgross!
Are any updates here?

@tgross
Copy link
Member

tgross commented Jan 2, 2024

@eumikhailov ok I've had a look through this and although disabling the UserInfo fetch entirely as in your PR might work (and seems like not a bad option to have), isn't the underlying problem that you've got the wrong audience set?

Received invalid UserInfo request. Audience <cleint_id>' in the access token is not same as the identifier of the UserInfo relying party trust 'urn:microsoft:userinfo'

@eumikhailov
Copy link
Contributor Author

eumikhailov commented Jan 2, 2024

@tgross yes, disabling the UserInfo fetch will solve this issue: no fetch, no issue with wrong aud :)

I've tried both settings and there is no way to set up urn:microsoft:userinfo audience and custom claims with roles in Microsoft OIDC provider:

If resource isn't passed here or in scope, AD FS uses a default resource urn:microsoft:userinfo. userinfo resource policies such as MFA, Issuance or authorization policy, can't be customized.

By the way Microsoft OIDC provider returns only sub claim in UserInfo endpoint
And https://learn.microsoft.com/en-us/windows-server/identity/ad-fs/overview/ad-fs-faq#i-m-trying-to-get-more-claims-on-the-userinfo-endpoint--but-it-s-only-returning-subject--how-can-i-get-more-claims-

@tgross
Copy link
Member

tgross commented Jan 2, 2024

I've tried both settings

In both the examples you linked to, your audience was "BoundAudiences": ["https://<NOMAD_FQDN>"],

@eumikhailov
Copy link
Contributor Author

eumikhailov commented Jan 2, 2024

Yes, Nomad verifies BoundAudiences by id_token which always is https://<NOMAD_FQDN> in my cases (Application ID in Microsoft OIDC provider wizard).
All differences are in access_token audience which Nomad uses for UserInfo fetching.

@tgross
Copy link
Member

tgross commented Jan 4, 2024

Just a heads up that I've got someone configuring a ADFS install for me so that I can do some validation at my end here.

@tgross
Copy link
Member

tgross commented Jan 8, 2024

@eumikhailov I'm still having some trouble getting my ADFS to issue me correct claims, but I'm not hitting the UserInfo error you're hitting either. Instead I'm getting a response for UserInfo that only includes sub. I did run into two other bugs in the process: #19668 and #19667.

Does the Client Permissions tab of your Web API Properties include the checkbox for the profile, openid, and allatclaims? I'm pretty sure you also need these in your OIDC scope list as well.

@eumikhailov
Copy link
Contributor Author

@tgross

UserInfo that only includes sub

It's by designed in Microsoft OIDC/ADFS - https://learn.microsoft.com/en-us/windows-server/identity/ad-fs/overview/ad-fs-faq#i-m-trying-to-get-more-claims-on-the-userinfo-endpoint--but-it-s-only-returning-subject--how-can-i-get-more-claims-
In my case Nomad does not get valid response from UserInfo, please verify OIDCScopes setting in Nomad OIDC config, It should be like https://nomad.local/openid.

Does the Client Permissions tab of your Web API Properties include the checkbox for the profile, openid, and allatclaims?

I've used following - allatclaims, email, openid, profile, there is my configs:

  1. ADFS Settings
    a. Create new ApplicationGroup WebAPI + Native Application
    b. Setup ClientId, Redirects and scopes (I'm using nomad.local identity)
    c. Config in YAML format (You can use same config with same IDs)

    ApplicationGroupIdentifier: "nomad.local"
    Name: "nomad.local"
    Description: "Nomad ApplicationGroup"
      
    NativeClientApplications:
        - Name: "nomad.local - Native application"
          ClientId: "de2658c8-cfbc-4d65-9b5e-bc3e85d1552d"
          Redirects:
              - http://localhost:4646/ui/settings/tokens
      
    WebApiApplications:
        - Name: "nomad - Web API"
          Uri:
              - https://nomad.local/
          ACP: Permit every one
          Permissions:
              - Application: "nomad.local - Native application"
                Scopes: [allatclaims, email, openid, profile]
          TransformRules: >
              Name => Token: SID
              Body =>
                c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/primarysid"]
                => issue(claim = c);
  2. Nomad Settings
    a. Setup microsoft-oidc.json config (OIDCClientSecret is not required for ADFS Native Application)

    {
        "OIDCDiscoveryURL": "https://<ADFS_FQDN>/adfs",
        "OIDCClientID": "de2658c8-cfbc-4d65-9b5e-bc3e85d1552d",
        "OIDCDisableUserInfo": true,
        "OIDCScopes": [
            "https://nomad.local/openid",
            "openid"
        ],
        "BoundAudiences": [
            "de2658c8-cfbc-4d65-9b5e-bc3e85d1552d",
            "https://nomad.local"
        ],
        "AllowedRedirectURIs": [
            "http://localhost:4646/ui/settings/tokens"
        ],
        "ClaimMappings": {
            "upn": "upn"
        },
        "ListClaimMappings": {
            "roles": "roles"
        }
    }

    b. Apply config in Nomad

    nomad acl auth-method create -type=OIDC -name=microsoft-oidc -max-token-ttl=60m -token-locality=global -config=@microsoft-oidc.json

@tgross
Copy link
Member

tgross commented Jan 9, 2024

Ok, thanks so much for your patience with me on this @eumikhailov. With a lot of help from my colleague @Sokren, I've reproduced the bug and verified that your patch fixes it. I'll leave a couple minor comments there but I think we'll be able to wrap this up quickly now.


Here's how I reproduced, which I'm documenting here so we can maybe come back and write up a ADFS tutorial in the future.

In ADFS:

  • Go to Application Groups and add a new Application Group.
  • Give our application a name (ex "Nomad")
  • Select Server Application
  • Configure the client ID to any value we want (nomad.local in this example).
  • Add the redirect URLS:
  • Create a shared secret and make sure you've copied it down.
  • Finish the wizard.
  • Double-click on the Application Group.
  • Select Add Application and add a Web API
  • For identifier, use the client from before (ex. nomad.local here)
  • Select Permit Everyone
  • Select allatclaims, email, openid, and profile
  • Finish the wizard.
  • Double-click on the Web API you just created.
  • Select Send LDAP Attributes as Claims
  • Name the claim rule (ex. "LDAP Group for Nomad")
  • Select Active Directory for the attribute store
  • For the LDAP Attribute, select Token-Groups - Unqualified Names
  • For the Outgoing Claim Type, select Group

Whew! Now on to Nomad. First, create a policy file:

// Grants read access to the namespace “default”.
namespace "default" {
  policy = "read"
}

// Grants read access to Nomad nodes.
node {
  policy = "read"
}

Create the policy and a role that uses it:

$ nomad acl policy apply engineering-read policy.hcl
$ nomad acl role create -name=engineering-read -policy=engineering-read

Create the auth method configuration file. The OIDCDiscoveryURL is set to the ADFS endpoint of my local ADFS VM, which has the appropriate cert set up for its domain. I've also configured a /etc/hosts value for that domain so that my host can reach the VM on the correct IP address.

Note the OIDCDisableUserInfo field has been added here (we get the 401 you reported if it's missing).

{
  "OIDCDiscoveryURL": "https://adfs.example.com/adfs",
  "OIDCClientID": "nomad.local",
  "OIDCClientSecret": "RvFWUU2HHgl_REDACTED",
  "BoundAudiences": ["nomad.local"],
  "OIDCDisableUserInfo": true,
  "OIDCScopes": ["openid", "profile", "allatclaims"],
  "AllowedRedirectURIs": [
    "http://localhost:4649/oidc/callback",
    "http://localhost:4646/ui/settings/tokens"
  ],
  "ListClaimMappings": {
    "group": "roles"
  }
}

Create the binding rule that binds to this auth method.

$ nomad acl binding-rule create \
    -auth-method=ADFS \
    -bind-type=role \
    -bind-name=engineering-read \
    -selector='engineering in list.roles'

And finally, login and see that we've successfully bound to the correct role!

$ nomad login -method=ADFS -oidc-callback-addr=localhost:4649
Successfully logged in via OIDC and ADFS

Accessor ID  = d9bfdf5d-f978-87e1-b477-4388a02b44b1
Secret ID    = 1ec9ff28-3841-cc44-7ea4-69d3ac6f544f
Name         = OIDC-ADFS
Type         = client
Global       = true
Create Time  = 2024-01-09 16:15:16.571920835 +0000 UTC
Expiry Time  = 2024-01-09 16:20:16.571920835 +0000 UTC
Create Index = 27
Modify Index = 27
Policies     = []

Roles
ID                                    Name
9db84e67-9855-2814-4f50-080270ae4759  engineering-read

tgross pushed a commit that referenced this issue Jan 9, 2024
…ovider (#19566)

Add new optional `OIDCDisableUserInfo` setting for OIDC auth provider which
disables a request to the identity provider to get OIDC UserInfo.

This option is helpful when your identity provider doesn't send any additional
claims from the UserInfo endpoint, such as Microsoft AD FS OIDC Provider:

> The AD FS UserInfo endpoint always returns the subject claim as specified in the
> OpenID standards. AD FS doesn't support additional claims requested via the
> UserInfo endpoint

Fixes #19318
@tgross tgross added this to the 1.7.x milestone Jan 9, 2024
@tgross
Copy link
Member

tgross commented Jan 9, 2024

#19566 has been merged and will ship in the next regular version of Nomad 1.7.x (with backports to 1.6.x and 1.5.x)

Copy link

github-actions bot commented Jan 2, 2025

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 2, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Development

Successfully merging a pull request may close this issue.

2 participants