[Bug]: Multiple errors when running b.oauth-bot

[Benjiiim]

Language

Javascript/Typescript

Version

latest (1.4.1)

Description

I'm running to two errors when trying to run the b.oauth-bot sample, once sending a message to the bot and expecting the consent prompt to appear.
The second error appears in a multi-tenant scenario where the Teams app is running in an other Entra ID/M365 tenant than the one used for App Registration.

Reproduction Steps

In the b.oauth-bot sample, with Teams AI JS 1.4.1 version.
Download the sample folder and open it in Visual Studio Code.
Teams toolkit configured with 2 accounts:
- Azure account
- M365 account in Entra ID tenant A
Debug with the local env with Debug (edge)
Select Azure subscription and ressource group through the command box picker
Wait for the provision and deploy steps to finished and keep the debug session active

Error 1
Launch the app by opening the link shared in the Output window with the same M365 account configured in Teams Toolkit (in Entra ID tenant A).
Add the bot the 1:1 conversation with "Add"
Open the Edge Dev Tools in the Network tab
Send a message to the bot hoping to get the consent prompt
See the HTTP 412 Precondition Failed errors with message Exception of type 'Library.Exceptions.AgentInvokeRemoteException' was thrown for POST requests https://teams.microsoft.com/api/chatsvc/emea/v1/agents/xxxxxxxxxx/invoke

Error 2
Open the Teams Dev Portal with same M365 account
Locate the app and Download app package
In Edge, with a Microsoft account from a separate M365 tenant (tenant B), sideload the app package in Teams
Add the bot the 1:1 conversation with "Add"
Open the Edve Dev Tools in the Network tab
Send a message to the bot hoping to get the consent prompt
See the HTTP 400 Bad Request errors with the following details:
- error invalid_resource
- error_description "AADSTS500011: The resource principal named api://botid-xxxxxxxxx was not found in the tenant named xxxx.com. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You might have sent your authentication request to the wrong tenant."
- error_codes 500011
- error_uri https://login.microsoftonline.com/error?code=500011

Regarding Error 2.
In the Entra ID App Registration manifest on the Entra ID Portal, I can see the signInAudience set to AzureADMultipleOrgs:

In the Bot configuration on the Azure portal, I can see that the Tenant ID is configured with common.

[corinagum]

For error 2, do you have multi-tenant selected for your bot *and app registration on azure portal?

[Benjiiim]

For error 2, do you have multi-tenant selected for your bot *and app registration on azure portal?

Yes. I edited the issue with screenshots.

[corinagum]

Hey @Benjiiim -- did you by any chance do any of the SSO steps manually? I ran the sample myself and was able to get error point 1 working for me:

If you're using TTK and start with no bot id or other data filled out in the /.env folder, TTK will do all of the provisioning for you.
Some notes:

• The first time I tried to run the sample today, I got a 'need to enter subscription' error. I cleared out all the prefilled data in the /.env folder and restarted from the beginning, and it worked.
• Once provisioning is complete, verify that OAUTH_CONNECTION_NAME is set in the .env file. For me, the value is graph-connection

For error point 2, my only suggestion is to verify that your bot is multi-tenant via the following UI in Azure Portal:

Originally I asked you about the app registration, but the sample didn't create one in my case. This is what my bot configuration page looks like:

Lastly, as in the readme, please ensure that you have the User.Read graph scope permission enabled.

[Benjiiim]

Thanks for your help.

I'm using the sample without any customization. I just extract the sample folder from the 1.4.1 release zip package and run the local Teams Toolkit env in Visual Studio Code. I tried as well with the 1.4.0 release zip package just in case with same results.

Regarding the first issue, I was able to see in the Azure Bot Service portal (on the channels page) that some "Internal errors" were happening with the Teams channel.
Unfortunately, nothing was logged in Visual Studio Code running the code locally.
I tried to step by step debug and arrived in the following part of the library code : BotAuthenticationBase.ts authenticate method

teams-ai/js/packages/teams-ai/src/authentication/BotAuthenticationBase.ts

Line 94 in 22a3887

// This could mean the user declined the consent prompt in the previous turn.

Regarding the comment about the consent prompt being declined, the consent prompt didn't appear at all in my case.

I finally found how to solve the issue: delete the browser cache (I was using Teams on Edge). Sign out and sign in back was not enough.
Without any other change, the consent message appeared on top of the compose box after the next message was sent.
I'm wondering if something can be done in the library to avoid the issue in the future or to ease the debug process but I can't reproduce anymore to give you more information. 😕

Regarding the second issue, multi-tenant is indeed activated in the bot. I sent the live manifest screenshot above and the settings is well reflected as "any org" in the Authentication page as you screenshot shows.
User.read is well configured in the app registration and in the OAuth configuration in the bot service.

May I ask you if you have been able to run the sample in a multi-tenant scenario by sideloading the zip package with a user account in an other tenant to give it a try? Does it work for you?

[corinagum]

I spoke with one of my teammates and your first issue regarding the consent prompt is a known issue. We have already alerted the Teams client team of it and filed an internal bug. Right now, the only workaround is exactly what you did with refreshing the cache of the app. Until Teams client fixes the issue, there isn't any other guidance we can provide.

Yes, I did try loading the app in a separate tenant and the app worked without issues.

[Benjiiim]

Thanks again for your time and for these insights @corinagum, very appreciated.

Good to know that the root cause of the first issue is identified.

I spent some time on the second issue today (multitenancy).
I think I have found the problem and from my perspective, this may be considered as a "sample bug" and this GitHub issue should be reopened with the right tags if you agree.

To make the b.oauth-bot sample work, we need to fix the azurebot.bicep file.
In the properties field of the botService resource, we need to add the msaAppType property with the 'Multitenant' value:

resource botService 'Microsoft.BotService/botServices@2021-03-01' = {
  kind: 'azurebot'
  location: 'global'
  name: botServiceName
  properties: {
    displayName: botDisplayName
    endpoint: 'https://${botAppDomain}/api/messages'
    msaAppId: botAadAppClientId
    msaAppType: 'MultiTenant'
  }
  sku: {
    name: botServiceSku
  }
}

From my understanding, without this configuration, the bot service will be created as single tenant for some reasons (default value was supposed to be multitenant as they have added singletenant and UserAssignedMSI later - microsoft/botbuilder-js#3906).
It might change the way the bot service asks a token to Entra ID, targeting the wrong tenant instead of targeting the common endpoint (even if Tenant Id is set to common in the OAuth Connection Settings of the bot, which is strange from my perspective but I may miss something...).

I'm not submitting a PR with this change as I would prefer to have your thoughts first, on various things:

• Does the above make sense and/or rings any bells? :-)
• Have you heard about a default value change from the Azure bot service team that might confirm my findings?
• Are other samples impacted? You might not want all samples to be configured as multi-tenant by default and might prefer to have single tenant by default with switch to multi-tenant documented. What about other langages?
• You might want to encourage people to use user-assigned managed identity to secure the communication between the bot backend and Azure bot service, which might require to create an other App registration for user authentication in the bot service oauth settings? But I'm not sure user-assigned managed identity here can be used in a multi-tenant scenario anyway. If it can, that might add a level of complexity you don't want to have to ease adoption.
• ...

[Benjiiim]

You might discard my previous message as I'm actually not able to make the sample work in a multi-tenant configuration event with msaAppType: 'MultiTenant' for the Azure bot resource. 😓

[Benjiiim]

I'm now starting to wonder if both errors are not related in some ways.

When granting admin consent manually, which means skipping the bot consent dialog for the user, authentication works great, in both cases:

• Granting admin consent in the home tenant, I'm not impacted anymore with the Library.Exceptions.AgentInvokeRemoteException, even if the cache has not been cleared.
• Granting admin consent in the distant tenant, I'm not impacted anymore with the AADSTS500011 error.

App Registrations and Bot provisioning/config look good (100% automatic through the Teams Toolkit workflow from the sample, I've just tried to add msaAppType: 'MultiTenant' for the bot resource but that doesn't seem to have any impact after all) and authentification works great as soon as the consent has been done.

I've deployed the sample more than 12 times in the past few days, in two different home tenants. I can reproduce 100% of the time.

Hopefully everything will be fixed when the Teams client issue you have identified a few weeks ago will be fixed but in the meantime, I'm starting to loose my hairs...

Thoughts would be appreciated.

[corinagum]

• The TeamsAdapter is already setting up credentials as Multi-tenant, so I lean towards saying your proposed change is redundant. Like I said before, when I ran this bot earlier this week, it worked out of the box and was already Multitenant. I just glanced through the JS samples, and it looks like they are all the same.
  
  However, I'm checking in with my team on their thoughts.
• Despite the fact that our samples are already quite opinionated, they are not intended to force our customers to adopt a particular stance on the myriad ways there are to create a bot. We have intentionally pushed back hard on changing our samples to use managed identity because of the overhead for both us and the customer and the same reason I stated earlier -- people just want to get these samples up and running to test things out. We have never intended for customers to publish these apps out of the box, with the possible exception of TeamsChefBot and OYD. We do not have plans to enforce managed identity, and it has already been discussed thoroughly internally. That being said, I believe Teams Toolkit is making a push to use managed identity on their samples (but only for published bots), so if you feel strongly about this, feel free to use their samples instead.

Please understand that our samples are intended to showcase features, and our focus is not geared towards making sure every sample works in every scenario that a customer might come up with. There are many things that are just not within our control, including admin permissions both on Azure and Teams, and that is something every customer will have to resolve themselves according to their organization's requirements.

The root cause of the Teams client issue has been found and fixed, though the ring train means it will be about a month before customers on public ring see the fix.

[Benjiiim]

Thanks a lot again for your time. That's really appreciated.

Regarding your comments on the samples, I do understand. I was thinking out loud when talking about Managed Identity. :-)

I would indeed be very interested to have your teams's thoughts about the relation between the adapter ConfigurationServiceClientCredentialFactoryOptions.MicrosoftAppType and the Azure bot resource msaAppType used during the creation of the bot service. From my perspective, they are two different things. A third thing being the AzureADMultipleOrgs value for the signInAudience property in the Entra ID app registration and a fourth being the fact that we use "common" value for the Tenant Id in the OAuth connection. 😅
It seems that adding msaAppType to MultiTenant when creating the Azure bot resource is not useful after all (it was a false lead followed during my investigations.). Multitenant might be the default value when nothing is provided.
However, something strange is happening: when defining explicitly msaAppType to MultiTenant at creation time, we can see this value in the Azure Portal Bot UI:

When nothing is provided in the ARM/Bicep file, we don't have the field at all in the UI (see your own screenshot above). I'm wondering if this is only a cosmetic difference or if there is more...
That being said, that's only me being curious here as that doesn't seem to impact the bot's behavior.

Great to know regarding the Teams client fix. I really hope that it will fix both consent dialog issues I'm having with the sample (and with the real app I'm building actually). Granting admin consent in order not to rely on the consent dialog in the user flow is a workaround not the solution.

Thanks again