[BUG] Upgrading from Microsoft.AspNetCore.DataProtection unable to load existing keys

[RickBlouch]

Query/Question
Based on this breaking change I'm working through upgrading to newer Azure.Extensions.* components to be used for DataProtection.

We were previously using the following pacakges

      <PackageReference Include="Microsoft.AspNetCore.DataProtection.AzureStorage" Version="3.1.7" />
      <PackageReference Include="Microsoft.AspNetCore.DataProtection.AzureKeyVault" Version="3.1.7" />

And we're now testing with

      <PackageReference Include="Azure.Extensions.AspNetCore.DataProtection.Blobs" Version="1.0.1" />
      <PackageReference Include="Azure.Extensions.AspNetCore.DataProtection.Keys" Version="1.0.1" />
      <PackageReference Include="Azure.Identity" Version="1.2.2" />

I worked through the differences in the APIs and have the data protection registration successfully working with this service registration code

     services.AddDataProtection()
                .SetApplicationName("shared")
                .PersistKeysToAzureBlobStorage(blobStorageConnectionString, containerName, "data-protection-keys.xml")
                .ProtectKeysWithAzureKeyVault(dataProtectionKeyUri, credentials);

In a new environment where there are no pre-existing keys in a key ring, it's working as expected.

The problem is that when I roll this code out to an environment with an existing key ring, it's unable to read existing keys.

At first, the exception was

[11:50:24 ERR] An exception occurred while processing the key element '<key id="b2f42ac4-a1c4-4f49-adba-ed6a14fcade9" version="1" />'.
System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.AspNetCore.DataProtection.AzureKeyVault, Version=3.1.2.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.
File name: 'Microsoft.AspNetCore.DataProtection.AzureKeyVault, Version=3.1.2.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
   at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, StackCrawlMarkHandle stackMark, ObjectHandleOnStack assemblyLoadContext, Boolean loadTypeFromPartialName, ObjectHandleOnStack type, ObjectHandleOnStack keepalive)
   at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark, AssemblyLoadContext assemblyLoadContext, Boolean loadTypeFromPartialName)
   at System.RuntimeType.GetType(String typeName, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark)
   at System.Type.GetType(String typeName, Boolean throwOnError)
   at Microsoft.AspNetCore.DataProtection.SimpleActivator.CreateInstance(Type expectedBaseType, String implementationTypeName)
   at Microsoft.AspNetCore.DataProtection.TypeForwardingActivator.CreateInstance(Type expectedBaseType, String originalTypeName, Boolean& forwarded)
   at Microsoft.AspNetCore.DataProtection.TypeForwardingActivator.CreateInstance(Type expectedBaseType, String originalTypeName)
   at Microsoft.AspNetCore.DataProtection.ActivatorExtensions.CreateInstance[T](IActivator activator, String implementationTypeName)
   at Microsoft.AspNetCore.DataProtection.XmlEncryption.XmlEncryptionExtensions.DecryptElement(XElement element, IActivator activator)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.IInternalXmlKeyManager.DeserializeDescriptorFromKeyElement(XElement keyElement)

Which makes sense given that I had removed the reference to the old Microsoft.AspNetCore.DataProtection.* packages. My first thought was to add those references back in again so the internals could read the existing keys and we could run for a period of time until new keys were rolled with the new Azure.Extensions.AspNetCore.DataProtection.* packages. After I made that change, I started getting this exception though.

[11:55:12 ERR] An exception occurred while processing the key element '<key id="b2f42ac4-a1c4-4f49-adba-ed6a14fcade9" version="1" />'.
System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.AspNetCore.DataProtection.AzureKeyVault.AzureKeyVaultXmlDecryptor.DecryptAsync(XElement encryptedElement)
   at Microsoft.AspNetCore.DataProtection.AzureKeyVault.AzureKeyVaultXmlDecryptor.Decrypt(XElement encryptedElement)
   at Microsoft.AspNetCore.DataProtection.XmlEncryption.XmlEncryptionExtensions.DecryptElement(XElement element, IActivator activator)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.IInternalXmlKeyManager.DeserializeDescriptorFromKeyElement(XElement keyElement)

In the key ring XML, the following XML node is the culprit

<encryptedSecret decryptorType="Microsoft.AspNetCore.DataProtection.AzureKeyVault.AzureKeyVaultXmlDecryptor, Microsoft.AspNetCore.DataProtection.AzureKeyVault, Version=3.1.2.0, Culture=neutral, PublicKeyToken=adb9793829ddae60" xmlns="http://schemas.asp.net/2015/03/dataProtection">

If I manually update the decryptorType attribute on existing keys to the newer decryption class, it's able to read existing keys.

<encryptedSecret decryptorType="Azure.Extensions.AspNetCore.DataProtection.Keys.AzureKeyVaultXmlDecryptor, Azure.Extensions.AspNetCore.DataProtection.Keys, Version=1.0.1.0, Culture=neutral, PublicKeyToken=92742159e12e44c8" xmlns="http://schemas.asp.net/2015/03/dataProtection">

I'm wondering if

• I missed a configuration that should allow for this scenario to be handled
• There is a better path to upgrade that doesn't involve losing all existing keys, or manually modifying every key ring to have the newer decryptorType attribute.

Thank you

[jsquire]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

[pakrym]

I missed a configuration that should allow for this scenario to be handled

Nope, there is no such option.

There is a better path to upgrade that doesn't involve losing all existing keys, or manually modifying every key ring to have the newer decryptorType attribute.

Yeah, this is a bug, migration is supposed to be completely drop-in.

I'll provide you with a temporary workaround and then ship it as a feature ASAP.

[pakrym]

@RickBlouch can you please try grabbing this file: https://github.com/pakrym/azure-sdk-for-net/blob/pakrym/support-reading-existing-keys/sdk/extensions/Azure.Extensions.AspNetCore.DataProtection.Keys/src/DecryptorTypeForwardingActivator.cs

and adding this line to your configure services method:

            services.AddSingleton<IActivator, DecryptorTypeForwardingActivator>();

[RickBlouch]

@pakrym The change works as expected, thank you.

What's the best way to track when this will be live so that I can back out this workaround?

[pakrym]

The release would happen at the end of this week or beginning of the next.