OIDC: add `resource` url-parameter for self-hosted ADFS #19318

eumikhailov · 2023-12-05T18:19:03Z

Hi there,

I encountered a problem when setting up OIDC auth provider over ADFS 2019, same as described here.

Nomad version

v1.6.x

Issue

There is no way to set up resource url-parameter for for self-hosted ADFS as OIDC provider.

Proposals for implementation

It can be done via the optional parameter OIDCResource in the Nomad OIDC config and optional config method WithResource in cap module.

The text was updated successfully, but these errors were encountered:

tgross · 2023-12-05T19:06:41Z

Hi @eumikhailov! Just to clarify acronyms here, self-hosted ADFS is "OIDC for Microsoft Active Directory", right? Any chance you have a pointer to Microsoft documentation on this parameter? It looks like this is a non-standard part of the Auth Request, according to the specs linked from the cap code you pointed to. See https://openid.net/specs/openid-connect-core-1_0.html#AuthRequest.

A quick survey suggests that some OIDC libraries provide a way to add arbitrary query parameters. I imagine if there are other providers with non-standard parameters, we would want to add a withResource rather than a withQueryParam. If necessary, we'd need to land that upstream in cap before it could appear in Nomad.

eumikhailov · 2023-12-05T19:22:29Z

HI @tgross!

Yes, self-hosted ADFS is OIDC for Microsoft Active Directory.
Some docs about is there:

According MS docs resource param is optional.

If no resource param is specified, AccessToken will have invalid aud 😔 This problem is well known, good repros are there and there.

On Nomad frontend /v1/acl/oidc/complete-auth error is

failed to retrieve the user info claims: Provider.UserInfo: provider UserInfo request failed: 401 Unauthorized:

tgross · 2023-12-05T20:26:16Z

From the docs you linked:

Note – If using MSAL client library, then resource parameter isn't sent. Instead the resource url is sent as a part of the scope parameter: scope = [resource url]//[scope values e.g., openid]
If resource isn't passed here or in scope, AD FS uses a default resource urn:microsoft:userinfo. userinfo resource policies such as MFA, Issuance or authorization policy, can't be customized.

You should be able to add an OIDCScope to cover this in the auth method.

eumikhailov · 2023-12-06T13:38:24Z

Hi @tgross!

Thank You for feedback!

We have found problem with our version of ADFS - https://serverfault.com/a/876784.
Now we will try to update it to 2019+ and will try scope again.

You can close this issue.

tgross · 2023-12-06T14:20:32Z

Thanks!

eumikhailov · 2023-12-06T21:40:20Z

Hi @tgross!

New version of Microsoft ADFS have same problem - access_token has client_id in aud and this value cannot be used for userinfo.

We got 401 error from ADFS:

Received invalid UserInfo request. Audience <cleint_id>' in the access token is not same as the identifier of the UserInfo relying party trust 'urn:microsoft:userinfo'

And 500 error from Nomad /v1/acl/oidc/complete-auth:

failed to retrieve the user info claims: Provider.UserInfo: provider UserInfo request failed: 401 Unauthorized:

Can You explain me please why UserInfo is required or is there any option to disable UserInfo loading?

nomad/nomad/acl_endpoint.go

Line 2740 in 87e7bf4

if userTokenSource := oidcToken.StaticTokenSource(); userTokenSource != nil {

eumikhailov · 2023-12-06T21:44:08Z

If we use scope = urn:microsoft:userinfo we cannot set up custom claims in token such as roles, groups etc.

So I think option for disable UserInfo loading is required for MS ADFS as OIDC provider.

tgross · 2023-12-06T21:44:18Z

Did you add an OIDCScope field to cover the resource parameter, as I mentioned previously?

eumikhailov · 2023-12-06T21:54:58Z

@tgross yes, I did and aud claim was changed from urn:microsoft:userinfo to resource which not applicable for UserProfile in ADFS 😔

eumikhailov · 2023-12-06T21:56:55Z

And if scope is not set we cannot set up custom claims in token such as roles, groups etc.

tgross · 2023-12-06T22:01:04Z

@eumikhailov can you share your auth config please? That'll help us narrow down what needs to be done here.

eumikhailov · 2023-12-06T22:27:47Z

@tgross yes, of course.

Config with scope option.

{
  "OIDCDiscoveryURL": "https://<ADFS_FQDN>/adfs",
  "OIDCClientID": "https://<NOMAD_FQDN>",
  "OIDCScope" : ["https://<NOMAD_FQDN>"],
  "BoundAudiences": ["https://<NOMAD_FQDN>"],
  "AllowedRedirectURIs": [
    "https://<NOMAD_FQDN>:4646/ui/settings/tokens"
  ],
  "ListClaimMappings": {
    "roles": "roles"
  }
}

In access_token we have claim aud = https://<NOMAD_FQDN>, in id_token we have claim roles = [ .. ] and claim aud = https://<NOMAD_FQDN>.
This acces_token is not acceptable for https://<ADFS_FQDN>/adfs/userinfo due to 401 error

Audience 'https://<NOMAD_FQDN>' in the access token is not same as the identifier of the UserInfo relying party trust 'urn:microsoft:userinfo'

Config without scope option.

{
  "OIDCDiscoveryURL": "https://<ADFS_FQDN>/adfs",
  "OIDCClientID": "https://<NOMAD_FQDN>",
  "OIDCScope" : [ ],
  "BoundAudiences": ["https://<NOMAD_FQDN>"],
  "AllowedRedirectURIs": [
    "https://<NOMAD_FQDN>:4646/ui/settings/tokens"
  ],
  "ListClaimMappings": {
    "roles": "roles"
  }
}

In access_token we will have aud = urn:microsoft:userinfo, in id_token we have not claim roles (custom claims is not configurable for urn:microsoft:userinfo) and claim aud = https://<NOMAD_FQDN>.
UserInfo works fone, but roles is empty, we cannot find binding-rule.

eumikhailov · 2023-12-07T11:41:46Z

@tgross Hi!

I think there is only one way to fix this - disable UserInfo loading by ODIC option.
https://stackoverflow.com/a/45762227

DuendeArchive/IdentityServer4#812 (comment)

Should we change issue title to feature request OIDCDisableLoadUserInfo optional flag in OIDC config?

tgross · 2023-12-07T13:40:54Z

@eumikhailov let me review this and get back to you.

eumikhailov · 2023-12-26T07:25:21Z

Hi @tgross!
Are any updates here?

tgross · 2024-01-02T20:16:03Z

@eumikhailov ok I've had a look through this and although disabling the UserInfo fetch entirely as in your PR might work (and seems like not a bad option to have), isn't the underlying problem that you've got the wrong audience set?

Received invalid UserInfo request. Audience <cleint_id>' in the access token is not same as the identifier of the UserInfo relying party trust 'urn:microsoft:userinfo'

eumikhailov · 2024-01-02T21:01:55Z

@tgross yes, disabling the UserInfo fetch will solve this issue: no fetch, no issue with wrong aud :)

I've tried both settings and there is no way to set up urn:microsoft:userinfo audience and custom claims with roles in Microsoft OIDC provider:

If resource isn't passed here or in scope, AD FS uses a default resource urn:microsoft:userinfo. userinfo resource policies such as MFA, Issuance or authorization policy, can't be customized.

By the way Microsoft OIDC provider returns only sub claim in UserInfo endpoint
And https://learn.microsoft.com/en-us/windows-server/identity/ad-fs/overview/ad-fs-faq#i-m-trying-to-get-more-claims-on-the-userinfo-endpoint--but-it-s-only-returning-subject--how-can-i-get-more-claims-

tgross · 2024-01-02T22:03:36Z

I've tried both settings

In both the examples you linked to, your audience was "BoundAudiences": ["https://<NOMAD_FQDN>"],

eumikhailov · 2024-01-02T22:10:42Z

Yes, Nomad verifies BoundAudiences by id_token which always is https://<NOMAD_FQDN> in my cases (Application ID in Microsoft OIDC provider wizard).
All differences are in access_token audience which Nomad uses for UserInfo fetching.

tgross · 2024-01-04T13:48:59Z

Just a heads up that I've got someone configuring a ADFS install for me so that I can do some validation at my end here.

tgross · 2024-01-08T19:57:13Z

@eumikhailov I'm still having some trouble getting my ADFS to issue me correct claims, but I'm not hitting the UserInfo error you're hitting either. Instead I'm getting a response for UserInfo that only includes sub. I did run into two other bugs in the process: #19668 and #19667.

Does the Client Permissions tab of your Web API Properties include the checkbox for the profile, openid, and allatclaims? I'm pretty sure you also need these in your OIDC scope list as well.

eumikhailov · 2024-01-09T07:45:45Z

@tgross

UserInfo that only includes sub

It's by designed in Microsoft OIDC/ADFS - https://learn.microsoft.com/en-us/windows-server/identity/ad-fs/overview/ad-fs-faq#i-m-trying-to-get-more-claims-on-the-userinfo-endpoint--but-it-s-only-returning-subject--how-can-i-get-more-claims-
In my case Nomad does not get valid response from UserInfo, please verify OIDCScopes setting in Nomad OIDC config, It should be like https://nomad.local/openid.

Does the Client Permissions tab of your Web API Properties include the checkbox for the profile, openid, and allatclaims?

I've used following - allatclaims, email, openid, profile, there is my configs:

ADFS Settings
a. Create new ApplicationGroup WebAPI + Native Application
b. Setup ClientId, Redirects and scopes (I'm using nomad.local identity)
c. Config in YAML format (You can use same config with same IDs)

ApplicationGroupIdentifier: "nomad.local"
Name: "nomad.local"
Description: "Nomad ApplicationGroup"
  
NativeClientApplications:
    - Name: "nomad.local - Native application"
      ClientId: "de2658c8-cfbc-4d65-9b5e-bc3e85d1552d"
      Redirects:
          - http://localhost:4646/ui/settings/tokens
  
WebApiApplications:
    - Name: "nomad - Web API"
      Uri:
          - https://nomad.local/
      ACP: Permit every one
      Permissions:
          - Application: "nomad.local - Native application"
            Scopes: [allatclaims, email, openid, profile]
      TransformRules: >
          Name => Token: SID
          Body =>
            c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/primarysid"]
            => issue(claim = c);

Nomad Settings
a. Setup microsoft-oidc.json config (OIDCClientSecret is not required for ADFS Native Application)

{
    "OIDCDiscoveryURL": "https://<ADFS_FQDN>/adfs",
    "OIDCClientID": "de2658c8-cfbc-4d65-9b5e-bc3e85d1552d",
    "OIDCDisableUserInfo": true,
    "OIDCScopes": [
        "https://nomad.local/openid",
        "openid"
    ],
    "BoundAudiences": [
        "de2658c8-cfbc-4d65-9b5e-bc3e85d1552d",
        "https://nomad.local"
    ],
    "AllowedRedirectURIs": [
        "http://localhost:4646/ui/settings/tokens"
    ],
    "ClaimMappings": {
        "upn": "upn"
    },
    "ListClaimMappings": {
        "roles": "roles"
    }
}

b. Apply config in Nomad

nomad acl auth-method create -type=OIDC -name=microsoft-oidc -max-token-ttl=60m -token-locality=global -config=@microsoft-oidc.json

tgross · 2024-01-09T16:39:33Z

Ok, thanks so much for your patience with me on this @eumikhailov. With a lot of help from my colleague @Sokren, I've reproduced the bug and verified that your patch fixes it. I'll leave a couple minor comments there but I think we'll be able to wrap this up quickly now.

Here's how I reproduced, which I'm documenting here so we can maybe come back and write up a ADFS tutorial in the future.

In ADFS:

Go to Application Groups and add a new Application Group.
Give our application a name (ex "Nomad")
Select Server Application
Configure the client ID to any value we want (nomad.local in this example).
Add the redirect URLS:
- http://localhost:4649/oidc/callback
- http://localhost:4646/ui/settings/tokens
Create a shared secret and make sure you've copied it down.
Finish the wizard.
Double-click on the Application Group.
Select Add Application and add a Web API
For identifier, use the client from before (ex. nomad.local here)
Select Permit Everyone
Select allatclaims, email, openid, and profile
Finish the wizard.
Double-click on the Web API you just created.
Select Send LDAP Attributes as Claims
Name the claim rule (ex. "LDAP Group for Nomad")
Select Active Directory for the attribute store
For the LDAP Attribute, select Token-Groups - Unqualified Names
For the Outgoing Claim Type, select Group

Whew! Now on to Nomad. First, create a policy file:

// Grants read access to the namespace “default”.
namespace "default" {
  policy = "read"
}

// Grants read access to Nomad nodes.
node {
  policy = "read"
}

Create the policy and a role that uses it:

$ nomad acl policy apply engineering-read policy.hcl
$ nomad acl role create -name=engineering-read -policy=engineering-read

Create the auth method configuration file. The OIDCDiscoveryURL is set to the ADFS endpoint of my local ADFS VM, which has the appropriate cert set up for its domain. I've also configured a /etc/hosts value for that domain so that my host can reach the VM on the correct IP address.

Note the OIDCDisableUserInfo field has been added here (we get the 401 you reported if it's missing).

{
  "OIDCDiscoveryURL": "https://adfs.example.com/adfs",
  "OIDCClientID": "nomad.local",
  "OIDCClientSecret": "RvFWUU2HHgl_REDACTED",
  "BoundAudiences": ["nomad.local"],
  "OIDCDisableUserInfo": true,
  "OIDCScopes": ["openid", "profile", "allatclaims"],
  "AllowedRedirectURIs": [
    "http://localhost:4649/oidc/callback",
    "http://localhost:4646/ui/settings/tokens"
  ],
  "ListClaimMappings": {
    "group": "roles"
  }
}

Create the binding rule that binds to this auth method.

$ nomad acl binding-rule create \
    -auth-method=ADFS \
    -bind-type=role \
    -bind-name=engineering-read \
    -selector='engineering in list.roles'

And finally, login and see that we've successfully bound to the correct role!

$ nomad login -method=ADFS -oidc-callback-addr=localhost:4649
Successfully logged in via OIDC and ADFS

Accessor ID  = d9bfdf5d-f978-87e1-b477-4388a02b44b1
Secret ID    = 1ec9ff28-3841-cc44-7ea4-69d3ac6f544f
Name         = OIDC-ADFS
Type         = client
Global       = true
Create Time  = 2024-01-09 16:15:16.571920835 +0000 UTC
Expiry Time  = 2024-01-09 16:20:16.571920835 +0000 UTC
Create Index = 27
Modify Index = 27
Policies     = []

Roles
ID                                    Name
9db84e67-9855-2814-4f50-080270ae4759  engineering-read

…ovider (#19566) Add new optional `OIDCDisableUserInfo` setting for OIDC auth provider which disables a request to the identity provider to get OIDC UserInfo. This option is helpful when your identity provider doesn't send any additional claims from the UserInfo endpoint, such as Microsoft AD FS OIDC Provider: > The AD FS UserInfo endpoint always returns the subject claim as specified in the > OpenID standards. AD FS doesn't support additional claims requested via the > UserInfo endpoint Fixes #19318

tgross · 2024-01-09T18:42:37Z

#19566 has been merged and will ship in the next regular version of Nomad 1.7.x (with backports to 1.6.x and 1.5.x)

github-actions · 2025-01-02T02:15:22Z

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

eumikhailov added the type/bug label Dec 5, 2023

tgross added the theme/auth label Dec 5, 2023

tgross added the stage/waiting-reply label Dec 5, 2023

tgross self-assigned this Dec 5, 2023

tgross changed the title ~~There is no way to set up resource url-parameter for for self-hosted ADFS as OIDC provider~~ OIDC: add resource url-parameter for self-hosted ADFS Dec 5, 2023

tgross added type/enhancement stage/needs-discussion and removed type/bug labels Dec 5, 2023

tgross closed this as completed Dec 6, 2023

tgross reopened this Dec 8, 2023

tgross added stage/needs-investigation and removed stage/waiting-reply stage/needs-discussion labels Dec 8, 2023

eumikhailov mentioned this issue Dec 29, 2023

Add new optional OIDCDisableUserInfo setting for OIDC auth provider #19566

Merged

This was referenced Jan 8, 2024

OIDC callback server doesn't display errors query params #19667

Open

Updating OIDC scope of AuthMethod fails silently #19668

Closed

tgross closed this as completed in #19566 Jan 9, 2024

hc-github-team-nomad-core mentioned this issue Jan 9, 2024

Backport of Add new optional OIDCDisableUserInfo setting for OIDC auth provider into release/1.5.x #19686

Merged

tgross added this to the 1.7.x milestone Jan 9, 2024

This was referenced Jan 9, 2024

Backport of Add new optional OIDCDisableUserInfo setting for OIDC auth provider into release/1.6.x #19687

Merged

Backport of Add new optional OIDCDisableUserInfo setting for OIDC auth provider into release/1.7.x #19688

Merged

abpetrov mentioned this issue Feb 28, 2024

Unable to set OIDCDisableUserInfo to true #20040

Closed

tgross added this to Nomad - Community Issues Triage Jun 24, 2024

tgross moved this to Done in Nomad - Community Issues Triage Jun 24, 2024

github-actions bot locked as resolved and limited conversation to collaborators Jan 2, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OIDC: add `resource` url-parameter for self-hosted ADFS #19318

OIDC: add `resource` url-parameter for self-hosted ADFS #19318

eumikhailov commented Dec 5, 2023

tgross commented Dec 5, 2023

eumikhailov commented Dec 5, 2023

tgross commented Dec 5, 2023

eumikhailov commented Dec 6, 2023

tgross commented Dec 6, 2023

eumikhailov commented Dec 6, 2023

eumikhailov commented Dec 6, 2023

tgross commented Dec 6, 2023

eumikhailov commented Dec 6, 2023

eumikhailov commented Dec 6, 2023

tgross commented Dec 6, 2023

eumikhailov commented Dec 6, 2023 •

edited

Loading

eumikhailov commented Dec 7, 2023

tgross commented Dec 7, 2023

eumikhailov commented Dec 26, 2023

tgross commented Jan 2, 2024

eumikhailov commented Jan 2, 2024 •

edited

Loading

tgross commented Jan 2, 2024 •

edited

Loading

eumikhailov commented Jan 2, 2024 •

edited

Loading

tgross commented Jan 4, 2024

tgross commented Jan 8, 2024

eumikhailov commented Jan 9, 2024

tgross commented Jan 9, 2024

tgross commented Jan 9, 2024

github-actions bot commented Jan 2, 2025

OIDC: add resource url-parameter for self-hosted ADFS #19318

OIDC: add resource url-parameter for self-hosted ADFS #19318

Comments

eumikhailov commented Dec 5, 2023

Nomad version

Issue

Proposals for implementation

tgross commented Dec 5, 2023

eumikhailov commented Dec 5, 2023

tgross commented Dec 5, 2023

eumikhailov commented Dec 6, 2023

tgross commented Dec 6, 2023

eumikhailov commented Dec 6, 2023

eumikhailov commented Dec 6, 2023

tgross commented Dec 6, 2023

eumikhailov commented Dec 6, 2023

eumikhailov commented Dec 6, 2023

tgross commented Dec 6, 2023

eumikhailov commented Dec 6, 2023 • edited Loading

eumikhailov commented Dec 7, 2023

tgross commented Dec 7, 2023

eumikhailov commented Dec 26, 2023

tgross commented Jan 2, 2024

eumikhailov commented Jan 2, 2024 • edited Loading

tgross commented Jan 2, 2024 • edited Loading

eumikhailov commented Jan 2, 2024 • edited Loading

tgross commented Jan 4, 2024

tgross commented Jan 8, 2024

eumikhailov commented Jan 9, 2024

tgross commented Jan 9, 2024

tgross commented Jan 9, 2024

github-actions bot commented Jan 2, 2025

OIDC: add `resource` url-parameter for self-hosted ADFS #19318

OIDC: add `resource` url-parameter for self-hosted ADFS #19318

eumikhailov commented Dec 6, 2023 •

edited

Loading

eumikhailov commented Jan 2, 2024 •

edited

Loading

tgross commented Jan 2, 2024 •

edited

Loading

eumikhailov commented Jan 2, 2024 •

edited

Loading