Program incorrectly handles crypto key when running under Linux .NET 9 but is fine under .NET 8

[Dave-Lowndes]

Description

This is a duplicate of this report on developercommunity.

The attached C# .NET program (see the developercommunity bug report) throws an exception when built for .NET 9 and is run on Linux platforms. I’ve tried Mint, Ubuntu (WSL), & Raspberry Pi (ARM64), all behave the same. However its fine on those when built for .NET 8 (and .NET 7) and similarly under Windows for .NET 7, 8, & 9.

The issue seems to arise in loading the private key in the call to ImportPkcs8PrivateKey. As noted in comments in the code, after this call in the good scenario, the rsa.KeySize is 3080, when run under Linux & .NET9, it’s 3073.

Presumably something in the Linux implementation has altered between .NET 8 & 9 that’s giving rise to this issue?

Here’s an example run of a .NET 8 and .NET 9 versions under Ubuntu (WSL):

david@MyPC:~/test$ ./EncDecTest8
Net V8.0.11 OS: Unix 4.4.0.19041
Private Key Length: 1793; Public Key Length: 420

Read 1793 bytes; privateKey Length: 1793
rsa KeySize: 3080; rsa alg: RSA; rsa sig alg: http://www.w3.org/2000/09/xmldsig#rsa-sha1
Hello World

david@MyPC:~/test$ ./EncDecTest9
.Net V9.0.0 OS: Unix 4.4.0.19041
Private Key Length: 1793; Public Key Length: 420

Read 1793 bytes; privateKey Length: 1793
rsa KeySize: 3073; rsa alg: RSA; rsa sig alg: http://www.w3.org/2000/09/xmldsig#rsa-sha1
Unhandled exception. System.Security.Cryptography.CryptographicException: The length of the data to decrypt is not valid for the size of this key.
   at System.Security.Cryptography.RSACryptoServiceProvider.Decrypt(Byte[] rgb, Boolean fOAEP)
   at EncDecTest.Program.Decrypt(ReadOnlySpan`1 privateKey, String data)
   at EncDecTest.Program.Main(String[] args)
Aborted (core dumped)

Reproduction Steps

Build the program (for .NET 9) and run it on a Linux platform. You should get the exception noted in the description.
Rebuild the program for .NET8 and run it on a Linux platform. It should work.
Note that the .NET 9 build will work fine on Windows.

Expected behavior

The .NET 9 version should work as the .NET 8 one does.

Actual behavior

.NET 9 version doesn't work.

Regression?

Yes, works fine for .NET8 on Linux platforms.

Known Workarounds

None - other than not to use .NET 9.

Configuration

Works on Linux platforms (x64 & ARM) when built for .NET 8, doesn't when built for .NET 9.

Other information

My presumption is that it's due to something having changed in the underlying Linux libraries between .NET8 & 9 - but that's just my uneducated guess.

[vcsjones]

The key here is a tad peculiar. You can reproduce this without .NET by taking the content of the private key and feeding it in to OpenSSL:

openssl pkey -in rsa.key -text -noout`

and the result is Private-Key: (3073 bit, 2 primes). In that respect, what .NET sees and what OpenSSL see are the same in terms of key. And yes, it's a 3073-bit RSA key.

The part that changed in .NET 8 to 9 is how we determine the key size. In .NET 8, we used EVP_PKEY_get_size which doesn't really give you the key size... it gives you the size of a buffer big enough to hold the key modulus. So that results in it getting "rounded up" when it gets converted to bits.

In .NET 9, it was changed to EVP_PKEY_get_bits, which gives an exact significant bit-count.

1. OpenSSL in this case is now saying "Your RSA's public modulus is 3073-bits", which is correct.
2. Other platforms are saying "I need 385 bytes to contain the modulus... and if I multiply that by 8, that's 3080-bits." That, by another interpretation, is correct, but less correct by my opinion.

Okay, so now we know why some platforms say 3080 and some say 3073, and why both are kinda "right".

RSACryptoServiceProvider.Decrypt(content, true) requires that the modulus and the content be exactly the same size. Unfortunately, this check will always fail for you.

Okay Kevin, thanks for the details but how do I get unblocked?

The easiest way to get unblocked is to change your Decrypt to

1. Use RSA.Create() instead of new RSACryptoServiceProvider()
2. Change rsa.Decrypt(dataBytes, true) to rsa.Decrypt(dataBytes, RSAEncryptionPadding.OaepSHA1);

RSACryptoServiceProvider is a compatibility / legacy type and shouldn't be used. RSA.Create() is the "right" thing to use.

Given that, you should also make the same changes to Encrypt.

The other thing you can do, if it is possible, is to change your RSA key to be more "standard" by not having a "not a multiple of 8 bits" problem. In theory it should work, but in practice a number of platforms will not always like an RSA modulus that does not have the most significant bit set.

[Dave-Lowndes]

Thanks Kevin, a quick test replacing the 2 points you mentioned works (on WSL at least; I presume other platforms will also work).
I came to appreciate the key was a bit odd when I couldn't recreate one in the same form with newer tools. Unfortunately, it can't easily be changed retrospectively.
I can fix the code that uses the key so that it will work though.
Thanks for that.

[vcsjones]

I presume other platforms will also work

It should! If it doesn’t then I think that is something I would like to know more about.

I can fix the code that uses the key so that it will work though.

This is probably the “right” fix. RSACryptoServiceProvider exists for compatibility, and instead RSA.Create should be favored (as well as the Encrypt and Decrypt that take RSAEncryptionPadding). The .NET Framework implementation of RSA has the same check for key size.

Do please let me know if the better API works for you on all platforms. My quick check against Windows 11 and macOS says it should.

[Dave-Lowndes]

Do please let me know if the better API works for you on all platforms. My quick check against Windows 11 and macOS says it should.

@vcsjones Yes, the changes to use RSA.Create work on Mint, Raspberry Pi, and WSL.

Thanks again.

[vcsjones]

Thanks for checking. In that case, I am going to close this out.

While it was a behavioral change between 8 and 9, the appetite to make changes to compatibility types is low, and using the recommended APIs resolved the problem.

Feel free to re-open this issue if you think there are still concerns that are unaddressed.