#6155: OAuth 2.0 - Microsoft #6192

alejandratenorio · 2019-09-19T17:48:41Z

New Contributors

Welcome! New contributors should at least glance at CONTRIBUTING.md, especially the section on pull requests where we encourage you to reach out to other developers before you start coding. Also, please note that we measure code coverage and prefer you write unit tests. Pull requests can still be reviewed without tests or completion of the checklist outlined below. Thanks!

Related Issues

Closes OAuth 2.0 - Microsoft #6155

Pull Request Checklist

Unit tests completed
Integration tests: None
Deployment requirements, SQL updates, Solr updates, etc.: None
Documentation completed
Merged latest from "develop" branch and resolved conflicts

dataversebot · 2019-09-19T17:53:01Z

Can one of the admins verify this patch?

coveralls · 2019-09-19T17:56:15Z

Coverage decreased (-0.004%) to 19.451% when pulling ab8714e on CIMMYT:6155-OAuth-2.0-Microsoft into 96efe2f on IQSS:develop.

djbrooke · 2019-09-19T17:57:00Z

Thanks @alejandratenorio for the PR. We'll review this and I'll also add some documentation.

poikilotherm · 2019-09-19T18:04:18Z

Since the merge of gdcc/dataverse-kubernetes#87, I am working on #5991 and it's recent PR about updating ScribeJava. It might be a good idea to have that merged first, so it doesn't have to be me testing the Microsoft OAuth2... 😉

pdurbin

Hi! The main thing this pull request needs is documentation in doc/sphinx-guides/source/installation/oauth2.rst

@alejandratenorio do you want to work on this?

Also, I left a comment about JSON vs. XML.

pdurbin · 2019-09-19T18:06:36Z

...arvard/iq/dataverse/authorization/providers/oauth2/AbstractOAuth2AuthenticationProvider.java

@@ -116,6 +116,10 @@ public OAuth2UserRecord getUserRecord(String code, String state, String redirect
        final OAuthRequest request = new OAuthRequest(Verb.GET, userEndpoint, service);
        request.addHeader("Authorization", "Bearer " + accessToken.getAccessToken());
        request.setCharset("UTF-8");
+
+        // Microsoft
+        request.addHeader("Accept", "application/json");


I'm slightly concerned about this because doesn't ORCID use XML instead of JSON?

13:48
Hi!

We use this line because Microsoft identity platform needs this header when send
an authentication request. If this header is out, we receive an error.

We work in a fix for the ORCID XML problem.

@Gerafp perfect. Thanks! Please let us know when that fix is in.

Thanks @Gerafp - I'll move this PR back to Community Dev.

Hi.

We add a instruction for manage the authentication request when use Microsoft.

The branch has updated with our fix.

djbrooke · 2019-09-19T18:26:14Z

Hi @Gerafp and @alejandratenorio, can one of you please give me permission to push to this branch? I have some documentation edits ready to push. Thanks!

alejandratenorio · 2019-09-19T18:29:33Z

Hi @Gerafp and @alejandratenorio, can one of you please give me permission to push to this branch? I have some documentation edits ready to push. Thanks!

@djbrooke, sure, it's done.

…dataverse into 6155-OAuth-2.0-Microsoft

doc/sphinx-guides/source/installation/oauth2.rst

poikilotherm

Please be aware that in current releases of ScribeJava an upstream API client is available for this Microsoft endpoint. IMHO it would be a pity to refactor in #5997 and not use the advantages of an updated lib.

djbrooke · 2019-09-20T13:46:32Z

Thanks @Gerafp and @alejandratenorio for giving me push access! Please take a look at the docs I added and feel free to expand those sections if necessary. Also note that I did not add any info about the json template at line 55 in oauth.rst, as it seems that still needs to be finalized. Thanks again for this contribution!

pdurbin · 2019-09-20T18:01:46Z

Please be aware that in current releases of ScribeJava an upstream API client is available for this Microsoft endpoint.

I'm a little confused.

Currently Dataverse uses ScribeJava 3.1.0. According to https://github.com/scribejava/scribejava/tree/scribejava-3.1.0#supports-all-major-10a-and-20-oauth-apis-out-of-the-box the following Microsoft API is supported in 3.1.0:

Microsoft Live (https://login.live.com)

If we upgrade to ScribeJava 6.8.1 (the latest), https://github.com/scribejava/scribejava/tree/scribejava-6.8.1#supports-all-major-10a-and-20-oauth-apis-out-of-the-box indicates that a total of three Microsoft APIs are supported:

Microsoft Azure Active Directory (Azure AD) (http://azure.microsoft.com/)
Microsoft Azure Active Directory (Azure AD) 2.0 (http://azure.microsoft.com/)
Microsoft Live (https://login.live.com/)

Which of these three Microsoft APIs will Dataverse support when this pull request is merged? All of them? One of them?

alejandratenorio · 2019-09-20T18:23:17Z

Thanks @Gerafp and @alejandratenorio for giving me push access! Please take a look at the docs I added and feel free to expand those sections if necessary. Also note that I did not add any info about the json template at line 55 in oauth.rst, as it seems that still needs to be finalized. Thanks again for this contribution!

@djbrooke, I added info about microsoft.json file.

…osoft

…dataverse into 6155-OAuth-2.0-Microsoft Add fix for ORCID XML problem when a user is autenticated with Microsoft

pdurbin

Now that 9ef52c5 is in I believe the code is ready for QA so I'm approving this. The docs seem to be in pretty good shape too.

djbrooke · 2019-10-25T01:23:07Z

I have one or two more things to check here. Hoping to move over before standup. :)

…dataverse into 6155-OAuth-2.0-Microsoft

djbrooke · 2019-10-25T16:36:59Z

Thanks all, sorry for the delay. I made one last change to generalize an error message in ab8714e. I'm also fine with the buttons as they are. I'm going to move this over to QA now. I'll take a look at the docs in parallel.

@kcondon, I tested this successfully with an outlook.com account I created and I also tested this with my Harvard institutional account and the flow worked as I expected, in line with our other Oauth login options. I know that we talked about this a little bit this morning, but if you want to discuss in more detail let me know.

kcondon · 2019-10-25T19:58:56Z

Hi @Gerafp , I'm testing this pr and running into a bit of an issue. You've tested both public (microsoft live?) and institutional azure ad/oauth accounts, correct? When you did so, how did you get your client id and secret to configure Dataverse? My understanding from the Azure AD link provided in the docs is one needs to register the app (dataverse), with the Azure AD tenant (directory authenticator/IdP), and that is what gives you the id and secret. If that is the case, I understand about the institutional AD registration and am pursuing that with my local university. However, how does that work with a public service like Live? I'm reading about the /tenant versus the /common userEndpoint but not sure how/where to register to get the right credentials. Thanks for any advice. Kevin P.S. Is this implementation multi-tenant by default? I was looking at this: https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-convert-app-to-be-multi-tenant

Gerafp · 2019-10-25T21:39:17Z

Hi @Gerafp , I'm testing this pr and running into a bit of an issue. You've tested both public (microsoft live?) and institutional azure ad/oauth accounts, correct? When you did so, how did you get your client id and secret to configure Dataverse? My understanding from the Azure AD link provided in the docs is one needs to register the app (dataverse), with the Azure AD tenant (directory authenticator/IdP), and that is what gives you the id and secret. If that is the case, I understand about the institutional AD registration and am pursuing that with my local university. However, how does that work with a public service like Live? I'm reading about the /tenant versus the /common userEndpoint but not sure how/where to register to get the right credentials. Thanks for any advice. Kevin P.S. Is this implementation multi-tenant by default? I was looking at this: https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-convert-app-to-be-multi-tenant

Hi @kcondon.

You can registry your app in this link

https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/RegisteredApps

Documentation

https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app

When you registry you app, You obtain the client_id and secret for testing Dataverse.

djbrooke · 2019-10-25T22:07:35Z

Thanks @Gerafp we'll take another look at this on Monday!

kcondon · 2019-10-26T00:37:47Z

@Gerafp @djbrooke

So, I followed the above link and it takes me to the azure portal. My account does not have permission to register an application. This is what I had encountered before.

I had looked at the developer quick start guide earlier and that was helpful in better understanding how the accounts worked and how multi tenant (common) and single tenant (tenantid) endpoints worked to support a variety of microsoft identities.

What I think I learned is this:

To configure and use Dataverse with Microsoft Azure AD, you need an Azure subscription that allows creating and managing an Azure AD.
You need an Azure AD tenant, either create one or use an existing one. A tenant represents an organization or institution.
You need an Azure account that has permission to register an application for at least one tenant.
Once you register Dataverse, you will get a client id and secret. Update your microsoft.json file with this information.
All tenants support both the tenantId and common endpoints, so I think this means you can choose the scope of authentication, whether all Microsoft accounts, regardless of AD (multitenant), or microsoft accounts managed by a single organization (single tenant), by specifying the userEndpoint in microsoft.json, either https://login.microsoftonline.com/{tenantId} or https://login.microsoftonline.com/common. I don't know if login.microsoftonline.com is the literal value or whether it is the tenant name: harvard.microsoftonline.com
Now that the microsoft.json file is completed with credentials and endpoint, use the Dataverse api to add Microsoft Azure AD Oauth as an authentication provider.

Would you confirm these steps are correct? Also, please send the microsoft.json file(s) you used for institutional and public account testing, client id and secret removed.

If the above directions are correct, I think we should make the instructions more like these.

djbrooke · 2019-10-28T14:47:26Z

I've reached out to our contacts at Harvard to try and set up some test institutional creds.

alejandratenorio · 2019-10-28T17:51:18Z

@Gerafp @djbrooke

So, I followed the above link and it takes me to the azure portal. My account does not have permission to register an application. This is what I had encountered before.

I had looked at the developer quick start guide earlier and that was helpful in better understanding how the accounts worked and how multi tenant (common) and single tenant (tenantid) endpoints worked to support a variety of microsoft identities.

What I think I learned is this:

To configure and use Dataverse with Microsoft Azure AD, you need an Azure subscription that allows creating and managing an Azure AD.

You need an Azure AD tenant, either create one or use an existing one. A tenant represents an organization or institution.

You need an Azure account that has permission to register an application for at least one tenant.

Once you register Dataverse, you will get a client id and secret. Update your microsoft.json file with this information.

All tenants support both the tenantId and common endpoints, so I think this means you can choose the scope of authentication, whether all Microsoft accounts, regardless of AD (multitenant), or microsoft accounts managed by a single organization (single tenant), by specifying the userEndpoint in microsoft.json, either https://login.microsoftonline.com/{tenantId} or https://login.microsoftonline.com/common. I don't know if login.microsoftonline.com is the literal value or whether it is the tenant name: harvard.microsoftonline.com

Now that the microsoft.json file is completed with credentials and endpoint, use the Dataverse api to add Microsoft Azure AD Oauth as an authentication provider.

Would you confirm these steps are correct? Also, please send the microsoft.json file(s) you used for institutional and public account testing, client id and secret removed.

If the above directions are correct, I think we should make the instructions more like these.

Hi @kcondon,

What kind of account did you use? I used my personal account (live.com.mx) and my cgiar account, in both I have permission to register an application.

kcondon · 2019-10-28T17:56:58Z

@alejandratenorio When I clicked on the azure portal I was automatically logged in and under switch directory I saw Harvard University as a tenant. Harvard has a subscription, as I understand it, because they provide access to Microsoft cloud applications and we use outlook/web for email. My guess is I have a regular user account as defined by our organization. That is why I was emphasizing the permissions part. I do have a support request in to our IT group for more information. Also, at @pdurbin suggestion, I created a new microsoft account, accessed the portal and had even less permission -no tenants, no ability to click on switch directories.

Gerafp · 2019-10-28T19:40:13Z

@alejandratenorio When I clicked on the azure portal I was automatically logged in and under switch directory I saw Harvard University as a tenant. Harvard has a subscription, as I understand it, because they provide access to Microsoft cloud applications and we use outlook/web for email. My guess is I have a regular user account as defined by our organization. That is why I was emphasizing the permissions part. I do have a support request in to our IT group for more information. Also, at @pdurbin suggestion, I created a new microsoft account, accessed the portal and had even less permission -no tenants, no ability to click on switch directories.

Hi!

I dont know the reason becausse this happen, Can you give me access to a institutional account for check this situation?.

Personally, I have a institutional account with domain alumno.buap.mx and can register a new application in Azure AD.

djbrooke · 2019-10-28T21:05:10Z

Thanks @Gerafp. We unfortunately do not have the ability to edit any of this from our side, as the institutional account here is centrally managed. We're waiting to hear back from support and I'll let you know as soon as I have more information.

I also reached out to our local ops group to see if they have any ideas.

kcondon · 2019-10-29T18:04:12Z

@alejandratenorio @Gerafp OK, I was able to get the live account to work, thanks for the link. I was also able to get the account I create from Friday working. Not sure why it did not work when I tried it. I can also authenticate my institutional account when I specify the user endpoint: https://login.microsoftonline.com/common but I am unable to register an app with that account -presumably due to a local IT policy. So, this can be merged. Apologies for the confusion. Kevin

6155 - OAuth 2.0 - Microsoft

79ccd4c

pdurbin reviewed Sep 19, 2019

View reviewed changes

djbrooke self-assigned this Sep 19, 2019

djbrooke added 3 commits September 19, 2019 14:31

adding documentation

481dc81

updating documentation

246d2b8

Merge branch '6155-OAuth-2.0-Microsoft' of https://github.com/CIMMYT/…

efee662

…dataverse into 6155-OAuth-2.0-Microsoft

djbrooke assigned alejandratenorio and unassigned djbrooke Sep 19, 2019

pdurbin reviewed Sep 19, 2019

View reviewed changes

doc/sphinx-guides/source/installation/oauth2.rst Show resolved Hide resolved

adding microsoft to page title

1bf38c2

poikilotherm requested changes Sep 19, 2019

View reviewed changes

alejandratenorio added 2 commits September 20, 2019 09:55

adding documentation

ff84855

adding documentation

1f9630a

Gerafp added 2 commits September 20, 2019 15:27

Add fix for ORCID XML problem when an user is authenticated with Micr…

9ef52c5

…osoft

Merge branch '6155-OAuth-2.0-Microsoft' of https://github.com/CIMMYT/…

982bd88

…dataverse into 6155-OAuth-2.0-Microsoft Add fix for ORCID XML problem when a user is autenticated with Microsoft

pdurbin approved these changes Sep 20, 2019

View reviewed changes

pdurbin unassigned alejandratenorio Sep 20, 2019

djbrooke self-assigned this Sep 23, 2019

pdurbin unassigned Gerafp and alejandratenorio Oct 24, 2019

djbrooke self-assigned this Oct 25, 2019

Gerafp and others added 3 commits October 25, 2019 09:46

Update clases and .gitignore

44c37b4

Merge branch '6155-OAuth-2.0-Microsoft' of https://github.com/CIMMYT/…

38d88e2

…dataverse into 6155-OAuth-2.0-Microsoft

make error message non-ORCID specific

ab8714e

djbrooke removed their assignment Oct 25, 2019

djbrooke self-assigned this Oct 25, 2019

djbrooke assigned kcondon Oct 29, 2019

kcondon merged commit 4470369 into IQSS:develop Oct 29, 2019

pdurbin mentioned this pull request Oct 29, 2019

As a Dataverse admin, I would like to hook up my installation to an IDM/IAM #5974

Closed

5 tasks

djbrooke added this to the 4.18 milestone Oct 29, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

#6155: OAuth 2.0 - Microsoft #6192

#6155: OAuth 2.0 - Microsoft #6192

alejandratenorio commented Sep 19, 2019 •

edited by djbrooke

Loading

dataversebot commented Sep 19, 2019

coveralls commented Sep 19, 2019 •

edited

Loading

djbrooke commented Sep 19, 2019 •

edited

Loading

poikilotherm commented Sep 19, 2019

pdurbin left a comment

pdurbin Sep 19, 2019

Gerafp Sep 19, 2019

pdurbin Sep 19, 2019

djbrooke Sep 19, 2019

Gerafp Sep 20, 2019 •

edited

Loading

djbrooke commented Sep 19, 2019

alejandratenorio commented Sep 19, 2019

poikilotherm left a comment

djbrooke commented Sep 20, 2019

pdurbin commented Sep 20, 2019

alejandratenorio commented Sep 20, 2019

pdurbin left a comment

djbrooke commented Oct 25, 2019

djbrooke commented Oct 25, 2019

kcondon commented Oct 25, 2019

Gerafp commented Oct 25, 2019 •

edited

Loading

djbrooke commented Oct 25, 2019

kcondon commented Oct 26, 2019 •

edited

Loading

djbrooke commented Oct 28, 2019

alejandratenorio commented Oct 28, 2019

kcondon commented Oct 28, 2019

Gerafp commented Oct 28, 2019

djbrooke commented Oct 28, 2019 •

edited

Loading

kcondon commented Oct 29, 2019

#6155: OAuth 2.0 - Microsoft #6192

#6155: OAuth 2.0 - Microsoft #6192

Conversation

alejandratenorio commented Sep 19, 2019 • edited by djbrooke Loading

New Contributors

Related Issues

Pull Request Checklist

dataversebot commented Sep 19, 2019

coveralls commented Sep 19, 2019 • edited Loading

djbrooke commented Sep 19, 2019 • edited Loading

poikilotherm commented Sep 19, 2019

pdurbin left a comment

Choose a reason for hiding this comment

pdurbin Sep 19, 2019

Choose a reason for hiding this comment

Gerafp Sep 19, 2019

Choose a reason for hiding this comment

pdurbin Sep 19, 2019

Choose a reason for hiding this comment

djbrooke Sep 19, 2019

Choose a reason for hiding this comment

Gerafp Sep 20, 2019 • edited Loading

Choose a reason for hiding this comment

djbrooke commented Sep 19, 2019

alejandratenorio commented Sep 19, 2019

poikilotherm left a comment

Choose a reason for hiding this comment

djbrooke commented Sep 20, 2019

pdurbin commented Sep 20, 2019

alejandratenorio commented Sep 20, 2019

pdurbin left a comment

Choose a reason for hiding this comment

djbrooke commented Oct 25, 2019

djbrooke commented Oct 25, 2019

kcondon commented Oct 25, 2019

Gerafp commented Oct 25, 2019 • edited Loading

djbrooke commented Oct 25, 2019

kcondon commented Oct 26, 2019 • edited Loading

djbrooke commented Oct 28, 2019

alejandratenorio commented Oct 28, 2019

kcondon commented Oct 28, 2019

Gerafp commented Oct 28, 2019

djbrooke commented Oct 28, 2019 • edited Loading

kcondon commented Oct 29, 2019

alejandratenorio commented Sep 19, 2019 •

edited by djbrooke

Loading

coveralls commented Sep 19, 2019 •

edited

Loading

djbrooke commented Sep 19, 2019 •

edited

Loading

Gerafp Sep 20, 2019 •

edited

Loading

Gerafp commented Oct 25, 2019 •

edited

Loading

kcondon commented Oct 26, 2019 •

edited

Loading

djbrooke commented Oct 28, 2019 •

edited

Loading