[Urgent] Publishing Portal after new release breaks AAD

[ZarTrox]

Bug description

If one publishes a DevPortal Instance after the latest release (07.08.20) log-in via the AAD button does not work.
The page loads but nothing happens at all. Also one cannot navigate through the URL.

Reproduction steps

1. Go to the published version of a DevPortal Instance
2. Get directed to the Sign-In page
3. Press the AAD button
4. Enter your credentials
5. Wait
6. See that nothing happens
7. (If the Portal is resetted it is in a refresh loop)

Expected behavior

It should work like before the new release.

Is your portal managed or self-hosted?

Managed

Additional context

[TimMnz09]

We have the same issue with our developer portal after publishing portal, but we are using simple username & password authentication.

[mikebudzynski]

Thanks for reporting the issue, we are now investigating it.

[mikebudzynski]

@ZarTrox we identified an issue with AAD: #824 but:

1. We were able to reproduce it only in the self-hosted portals, not managed
2. The website doesn't reload infinitely as you show in the screenshot

Can you make sure there weren't any changes in your AAD / networking setup? If there weren't any, can you provide us with the service name and network traces via email to apimportalfeedback@microsoft.com?

@TimMenze what's the problem with the username and password sign-in? Can you provide us with reproduction steps? It doesn't seem to be related.

cc: @ygrik

[TimMnz09]

@mikebudzynski sure, please find some more information below.

Reproduction steps

1. Open Dev-Portal URL
2. Redirect to Sign-In page
3. Enter Username + Password
4. Press "Sign In" Button
5. Wait
6. Infinite Refresh Loop to Sign-In page

[ygrik]

@TimMenze did you try to remove cookies and clear session?

[AnRei123]

Yes. Some hours ago, I received that further instances for development are impacted know. They cannot log in anymore. The crazy thing is that I tested nearly all instances this morning, most of them still worked for me. We did the test with the inkognito mode of the Chrome browser. Once, additionally I cleared the cache of the browser. This is really a bad situation. We do not only use the portal by ourselves. Customers do also rely on them. Our product timeline depends on accessing the portals of several instances. Viele Grüße, Angelika

…

--- Original-Nachricht --- Von: Igor O Betreff: Re: [Azure/api-management-developer-portal] [Urgent] Publishing Portal after new release breaks AAD (#822) Datum: 11. August 2020, 19:28 An: Azure/api-management-developer-portal Cc: AnRei123, Manual @TimMenze <https://github.com/TimMenze> did you try to remove cookies and clear session? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#822 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOB7XZ3SCKYCNC3RZE3YZ7TSAF5TTANCNFSM4PZ23HUA> .

[TimMnz09]

@ygrik yes I did. Same problem :(

[MelHarbour]

We are also experiencing this issue with it looping after signin on AAD.

[LockTar]

Same problem here! No working developer portal anymore

[olandese]

we are experiencing the same problem. Not able to login with AAD anymore

[mikebudzynski]

We have hotfixed all regions, please republish your portal. Apologies for the inconvenience and disruption.

We'll create a new self-hosted release shortly.

[olandese]

I just did a republish, but the problem is still there (West-Europe)

[mikebudzynski]

We redeployed the fix, can you please check again? Please, republish the portal and, if it still doesn't work, clear the cookies and the session.

[AnRei123]

Unfortunately, for the two affected instances (managed) it still does not work. I cleared the cache in the Chrome browser. Closed the browser and opened it again. Published the current version. Opened the web url in the incognito mode. But still same behavior. Only the sign-in page is shown after adding the credentials to the sign-in dialog. For the legacy portal of the impacted instances, this issue does not show up. The sign-in for the legacy portal with the aad button is possible w/o any problem. This issue only occurs for two of our instances for the new dev portal after I have done content modifications and after I have published the content. I also checked the behavior with the Firefox browser this morning. Same issue.

[olandese]

same here, I republished the portal, open it in an incognito window but the problem is still there

[butsona]

We are also experiencing issues with sso. After clicking the AAD Signin the popup is shown. Once credentials are entered the popup disappears but the main form is not refresh. If we manually refresh the user is logged in. But users are unlikely to manually refresh.

Please can you make sure changes are tested before releasing. Especially when this effecting our clients.

[olandese]

I was able to solve the problem disabling the "Redirect anonymous users to sign-in page".

If the option is enabled after logging in with AAD, the users seems still to be anonymous and no Products/APIs are shown.
With the option disabled (and after logging in with AAD) we are able to see the APIs and Products.
In our setup the Products and APIs are shown only if an authenticated user is in a specific APIM group (Developers), that's why we can temporary disable the option.

Obviously this is not what we want, it's just a temporary work around

[olandese]

Is there any news about this issue?

[mikebudzynski]

The problem was on the backend side of API Management (cookie wasn't set on redirect) and the fix will be rolled out in the next few days.

[olandese]

The problem was on the backend side of API Management (cookie wasn't set on redirect) and the fix will be rolled out in the next few days.

I just republished the portal, but the problem is still there. Is the fix rolled out in West-Europe?

[azaslonov]

Hi @olandese, backend hasn't been deployed to West Europe yet, but it looks like this bug affects only specific services, so, to make sure this is the same case, could you please capture network traces into HAR file and send it to us apimportalfeedback@microsoft.com?

[tomilaakkonen]

@azaslonov It affects if you force anon users to login. If you disable the option then the problem doesn't occur. We are using West Europe and still having the issue.

[olandese]

@azaslonov It affects if you force anon users to login. If you disable the option then the problem doesn't occur. We are using West Europe and still having the issue.

Exactly!!!
I described it in a previous comment: #822 (comment)

[LockTar]

Why closed? Issues are still there...

[azaslonov]

Looks like it was auto-closed because of PR that associated with it.

[azaslonov]

@LcokTar, after the last hotfix I can't repro this anymore on any of my services. Can you please:

1. Ensure you have re-published it;
2. Send service name and network traces to apimportalfeedback@microsoft.com if the issue still there (we need traces from the moment you press sign-in to a one or to loop reloads);

[olandese]

Hi @olandese, backend hasn't been deployed to West Europe yet, but it looks like this bug affects only specific services, so, to make sure this is the same case, could you please capture network traces into HAR file and send it to us apimportalfeedback@microsoft.com?

Hi @azaslonov ,

I just sent an email with the HAR files and the name of the instances where we are have isues with.
As I told you before, the issue happens when we force anon user to login, see: #822 (comment)

[azaslonov]

Thanks, we'll look into this.

[LockTar]

@LcokTar, after the last hotfix I can't repro this anymore on any of my services. Can you please:

1. Ensure you have re-published it;
2. Send service name and network traces to apimportalfeedback@microsoft.com if the issue still there (we need traces from the moment you press sign-in to a one or to loop reloads);

@olandese and I work both on the same 4 instances (with the same issue) sow you have our traces now.

[butsona]

has this been rolled out globally. The portal is still not refreshing after login. We still have to manually refresh the page to complete login. We are using managed portal in uk south

[tomilaakkonen]

Any update on this?

@butsona how do you manually refresh? At least for me, f5 after login isn't working. Also clicking on the navigation bar doesn't do any difference.

[azaslonov]

UPDATE: The issue has been fixed. We're rolling out a hotfix for the backend. It should land everywhere this week. Thank you for patience everyone.

[tomilaakkonen]

At least in West Europe, we haven't got any updates/hotfixes.

[AnRei123]

Issue still exists at our location (West Europe). I am still not able to sign in to my published websites. Any news?

[AnRei123]

2 further Dev instances have been effected now. Seems that someone has by chance pressed the "publish" buttons for these instances. Please urgently come up with a fix!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[azaslonov]

@AnRei123, can you send us your service names?
We could try to prioritize them.

[AnRei123]

Here the names of the API Management service names:

• apiteamplaydeploytest
• apiteamplaydev

[azaslonov]

@AnRei123, can you please give it a try now?

[ErikMogensen]

It looks like we have the same problem with our instance in West Europe which has:
"PortalVersion":"202009011445",
"CodeVersion":"20200901084820",
"Version":"0.13.1048.0".

Our instance in West Central US also seems to have this problem. It has this version information:
"PortalVersion": "202008260847",
"CodeVersion": "20200812080849",
"Version": "0.13.1056.0"

Edit: Removed info about region updates and put that in a new issue #872

[thgossler]

The the Chrome browser Application session storage I'm see "Adal" values (see screenshot), and in the Azure Portal I'm seeing a warning that from June 2020 ADAL is not updated and supported anymore. Could it be that there is a problem?

[AnRei123]

Retested the two API Management services I mentioned above again:

• apiteamplaydeploytest: now WORKING again - thank you!

• apiteamplaydev: still NOT working

[azaslonov]

@AnRei123, the codebase is same for both services, so I suspect there is something cached. Can you please try to reset it:

If still doesn't help, please capture Network traces into HAR file: the login attempt itself and several reload; and send it to apimportalfeedback@microsoft.com

[AnRei123]

For the apiteamplaydev service, the published portal is still not working even after performing all the following steps:

• Clearing the browser cache
• Clearing the storage for the unregister service worker options on the application tab card
• Closing and reopening the browser
• Republishing again

@azaslonov: I have sent you the HAR file per email.

[AnRei123]

For the following internal APIM services the sign-in and routing to the homepage via the portal web URL is also not working:

• apiteamplayuxsource
• apiteamplayuxplayground
• apiteamplayuxdefaults
  Could you also fix the issues for these services?

[azaslonov]

Hi everyone, fix deployed, can you please re-publish and try again now?

[thgossler]

Great, it seems to be working now for us :-)

One thing I had to change in one of our instances was the homepage URL. It was set to the /signin URL as suggested in the tooltip (which is a bit confusing) which resulted in landing on the signin page every time again after sign-in. But this was not the case in other instances where it is now working also.

[ErikMogensen]

Yes @azaslonov, we are now able to log in to both of the portals we couldn't log in to previously. 👍

[AnRei123]

Thank you for solving this issue! Our portal sign-in works again as expected. Great work!

[azaslonov]

Thanks everyone for confirmation and your great patience.