[Bug] `msal_extensions.token_cache.PersistedTokenCache` is bypassed by `ConfidentialClientApplication`

[jiasli]

Describe the bug
AzureAD/microsoft-authentication-library-for-python#644 introduced a regression that msal_extensions.token_cache.PersistedTokenCache is bypassed by ConfidentialClientApplication.

To Reproduce

az login --service-principal --username ... --password ... --tenant ... --allow-no-subscriptions

az account get-access-token --scope https://management.azure.com//.default
...
  "expiresOn": "2024-04-11 19:25:43.000000",

az account get-access-token --scope https://management.azure.com//.default
...
  "expiresOn": "2024-04-11 19:26:03.000000",

Notice each time a new access token is retrieved, bypassing the token cache. Detailed analysis is provided at AzureAD/microsoft-authentication-library-for-python#644 (comment).

This causes a severe regression in OIDC authentication, so that no Azure CLI task can run longer than the OIDC token's 5-minute lifetime (Azure/azure-cli#28708 (comment)).

Expected behavior
Old access token from the token cache should be retrieved.

What you see instead
A new access token is retrieved.

The MSAL Python version you are using
1.28.0

Additional context
Add any other context about the problem here.

[bgavrilMS]

@rayluo - it looks like Azure CLI has been getting quite a lot of attention on this issue, including ICMs and the scenario is very much used right now. Any objection to gettting a release out asap?

[rayluo]

@rayluo - it looks like Azure CLI has been getting quite a lot of attention on this issue, including ICMs and the scenario is very much used right now. Any objection to gettting a release out asap?

@bgavrilMS , even if we ship this immediately, there are still some logistic steps on the Azure CLI side. After a discussion with Azure CLI team, we mutually agreed on the steps and timeline, described in the mitigation comment in the ICM.

[bgavrilMS]

@rayluo - it looks like Azure CLI has been getting quite a lot of attention on this issue, including ICMs and the scenario is very much used right now. Any objection to gettting a release out asap?

@bgavrilMS , even if we ship this immediately, there are still some logistic steps on the Azure CLI side. After a discussion with Azure CLI team, we mutually agreed on the steps and timeline, described in the mitigation comment in the ICM.

Thank you @rayluo and @jiasli