X509Certificate2 Fails to load Pfx files that contain a 25519 key/cert instead reports wrong password

[annerajb]

Description

Issue

The x509certificate2 class fails loading a pfx file which contains a ed25519 private key and it's certificate (+ chain)

The real failure seems to be here (it's super hard to know 100% since visual studio 2019 does not load the openssl native shims and just optimized assembly)

The oid of the private key is: "1.3.101.112" which corresponds to the RFC oid for ED25519
https://cryptography.io/en/latest/x509/reference.html#cryptography.x509.oid.SignatureAlgorithmOID.ED25519

From reading it seems that support for 25519 has been requested since 2015 #14741

Could this be implemented today at least with openssl on linux I need to use it with SslStream and SecureStream and I can't override the x509certificate2 class to use bouncycastle or any other library due to the library forbidding overloads/overrides.

reproducing

All it takes for it to fail is to try calling the constructor like this
new X509Certificate2("server_chain25519.pfx", "qwerty");

Attached a text file that is a bashscript to generate the pfx file on the same format with the whole chain + private key and same password being used.
generate_25519_certs.txt

Configuration

Project With the sdk=Microsoft.net.sdk.web
Target Framework: net 5.0
Running using docker mcr.microsoft.com/dotnet/aspnet:5.0-buster-slim

Regression?

Not sure my guess is this never worked before.

Other information

Stacktraces

Inner Stack trace:

Message: A certificate referenced a private key which was already referenced, or could not be loaded.

   at Internal.Cryptography.Pal.UnixPkcs12Reader.BuildCertsWithKeys(CertBagAsn[] certBags, AttributeAsn[][] certBagAttrs, CertAndKey[] certs, Int32 certBagIdx, SafeBagAsn[] keyBags, RentedSubjectPublicKeyInfo[] publicKeyInfos, AsymmetricAlgorithm[] keys, Int32 keyBagIdx)
   at Internal.Cryptography.Pal.UnixPkcs12Reader.Decrypt(ReadOnlySpan`1 password, ReadOnlyMemory`1 authSafeContents)
   at Internal.Cryptography.Pal.UnixPkcs12Reader.VerifyAndDecrypt(ReadOnlySpan`1 password, ReadOnlyMemory`1 authSafeContents)
   at Internal.Cryptography.Pal.UnixPkcs12Reader.Decrypt(SafePasswordHandle password)

Outer stack trace

System.Security.Cryptography.CryptographicException
  HResult=0x80070056
  Message=The certificate data cannot be read with the provided password, the password may be incorrect.
  Source=System.Security.Cryptography.X509Certificates
  StackTrace:
   at Internal.Cryptography.Pal.UnixPkcs12Reader.Decrypt(SafePasswordHandle password) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/Internal/Cryptography/Pal.Unix/UnixPkcs12Reader.cs:line 230
   at Internal.Cryptography.Pal.PkcsFormatReader.TryReadPkcs12(OpenSslPkcs12Reader pfx, SafePasswordHandle password, Boolean single, ICertificatePal& readPal, List`1& readCerts) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/Internal/Cryptography/Pal.Unix/PkcsFormatReader.cs:line 303
   at Internal.Cryptography.Pal.PkcsFormatReader.TryReadPkcs12(ReadOnlySpan`1 rawData, SafePasswordHandle password, Boolean single, ICertificatePal& readPal, List`1& readCerts, Exception& openSslException) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/Internal/Cryptography/Pal.Unix/PkcsFormatReader.cs:line 290
   at Internal.Cryptography.Pal.OpenSslX509CertificateReader.FromFile(String fileName, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/Internal/Cryptography/Pal.Unix/OpenSslX509CertificateReader.cs:line 87
   at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(String fileName, String password, X509KeyStorageFlags keyStorageFlags) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509Certificate.cs:line 153
   at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(String fileName, String password) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509Certificate2.cs:line 126
   at KestrelTcpDemo.Program.CreateWebHostBuilder(String[] args) in C:\Users\annerajb\Source\Repos\MultiProtocolAspNetCore\KestrelTcpDemo\Program.cs:line 42
   at KestrelTcpDemo.Program.Main(String[] args) in C:\Users\annerajb\Source\Repos\MultiProtocolAspNetCore\KestrelTcpDemo\Program.cs:line 30

  This exception was originally thrown at this call stack:
    Internal.Cryptography.Pal.UnixPkcs12Reader.BuildCertsWithKeys(System.Security.Cryptography.Asn1.Pkcs12.CertBagAsn[], System.Security.Cryptography.Asn1.AttributeAsn[][], Internal.Cryptography.Pal.UnixPkcs12Reader.CertAndKey[], int, System.Security.Cryptography.Asn1.Pkcs12.SafeBagAsn[], Internal.Cryptography.Pal.UnixPkcs12Reader.RentedSubjectPublicKeyInfo[], System.Security.Cryptography.AsymmetricAlgorithm[], int) in UnixPkcs12Reader.cs
    Internal.Cryptography.Pal.UnixPkcs12Reader.Decrypt(System.ReadOnlySpan<char>, System.ReadOnlyMemory<byte>) in UnixPkcs12Reader.cs
    Internal.Cryptography.Pal.UnixPkcs12Reader.VerifyAndDecrypt(System.ReadOnlySpan<char>, System.ReadOnlyMemory<byte>) in UnixPkcs12Reader.cs
    Internal.Cryptography.Pal.UnixPkcs12Reader.Decrypt(Microsoft.Win32.SafeHandles.SafePasswordHandle) in UnixPkcs12Reader.cs

Inner Exception 1:
CryptographicException: A certificate referenced a private key which was already referenced, or could not be loaded.

[ghost]

Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Issue

The x509certificate2 class fails loading a pfx file which contains a ed25519 private key and it's certificate (+ chain)

The real failure seems to be here (it's super hard to know 100% since visual studio 2019 does not load the openssl native shims and just optimized assembly)

The oid of the private key is: "1.3.101.112" which corresponds to the RFC oid for ED25510
https://cryptography.io/en/latest/x509/reference.html#cryptography.x509.oid.SignatureAlgorithmOID.ED25519

From reading it seems that support for 25519 has been requested since 2015 #14741

Could this be implemented today at least with openssl on linux I need to use it with SslStream and SecureStream and I can't override the x509certificate2 class to use bouncycastle or any other library due to the library forbidding overloads/overrides.

reproducing

All it takes for it to fail is to try calling the constructor like this
new X509Certificate2("server_chain25519.pfx", "qwerty");

Attached a text file that is a bashscript to generate the pfx file on the same format with the whole chain + private key and same password being used.
generate_25519_certs.txt

Configuration

Project With the sdk=Microsoft.net.sdk.web
Target Framework: net 5.0
Running using docker mcr.microsoft.com/dotnet/aspnet:5.0-buster-slim

Regression?

Not sure my guess is this never worked before.

Other information

Stacktraces

Inner Stack trace:

Message: A certificate referenced a private key which was already referenced, or could not be loaded.

   at Internal.Cryptography.Pal.UnixPkcs12Reader.BuildCertsWithKeys(CertBagAsn[] certBags, AttributeAsn[][] certBagAttrs, CertAndKey[] certs, Int32 certBagIdx, SafeBagAsn[] keyBags, RentedSubjectPublicKeyInfo[] publicKeyInfos, AsymmetricAlgorithm[] keys, Int32 keyBagIdx)
   at Internal.Cryptography.Pal.UnixPkcs12Reader.Decrypt(ReadOnlySpan`1 password, ReadOnlyMemory`1 authSafeContents)
   at Internal.Cryptography.Pal.UnixPkcs12Reader.VerifyAndDecrypt(ReadOnlySpan`1 password, ReadOnlyMemory`1 authSafeContents)
   at Internal.Cryptography.Pal.UnixPkcs12Reader.Decrypt(SafePasswordHandle password)

Outer stack trace

System.Security.Cryptography.CryptographicException
  HResult=0x80070056
  Message=The certificate data cannot be read with the provided password, the password may be incorrect.
  Source=System.Security.Cryptography.X509Certificates
  StackTrace:
   at Internal.Cryptography.Pal.UnixPkcs12Reader.Decrypt(SafePasswordHandle password) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/Internal/Cryptography/Pal.Unix/UnixPkcs12Reader.cs:line 230
   at Internal.Cryptography.Pal.PkcsFormatReader.TryReadPkcs12(OpenSslPkcs12Reader pfx, SafePasswordHandle password, Boolean single, ICertificatePal& readPal, List`1& readCerts) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/Internal/Cryptography/Pal.Unix/PkcsFormatReader.cs:line 303
   at Internal.Cryptography.Pal.PkcsFormatReader.TryReadPkcs12(ReadOnlySpan`1 rawData, SafePasswordHandle password, Boolean single, ICertificatePal& readPal, List`1& readCerts, Exception& openSslException) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/Internal/Cryptography/Pal.Unix/PkcsFormatReader.cs:line 290
   at Internal.Cryptography.Pal.OpenSslX509CertificateReader.FromFile(String fileName, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/Internal/Cryptography/Pal.Unix/OpenSslX509CertificateReader.cs:line 87
   at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(String fileName, String password, X509KeyStorageFlags keyStorageFlags) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509Certificate.cs:line 153
   at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(String fileName, String password) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509Certificate2.cs:line 126
   at KestrelTcpDemo.Program.CreateWebHostBuilder(String[] args) in C:\Users\annerajb\Source\Repos\MultiProtocolAspNetCore\KestrelTcpDemo\Program.cs:line 42
   at KestrelTcpDemo.Program.Main(String[] args) in C:\Users\annerajb\Source\Repos\MultiProtocolAspNetCore\KestrelTcpDemo\Program.cs:line 30

  This exception was originally thrown at this call stack:
    Internal.Cryptography.Pal.UnixPkcs12Reader.BuildCertsWithKeys(System.Security.Cryptography.Asn1.Pkcs12.CertBagAsn[], System.Security.Cryptography.Asn1.AttributeAsn[][], Internal.Cryptography.Pal.UnixPkcs12Reader.CertAndKey[], int, System.Security.Cryptography.Asn1.Pkcs12.SafeBagAsn[], Internal.Cryptography.Pal.UnixPkcs12Reader.RentedSubjectPublicKeyInfo[], System.Security.Cryptography.AsymmetricAlgorithm[], int) in UnixPkcs12Reader.cs
    Internal.Cryptography.Pal.UnixPkcs12Reader.Decrypt(System.ReadOnlySpan<char>, System.ReadOnlyMemory<byte>) in UnixPkcs12Reader.cs
    Internal.Cryptography.Pal.UnixPkcs12Reader.VerifyAndDecrypt(System.ReadOnlySpan<char>, System.ReadOnlyMemory<byte>) in UnixPkcs12Reader.cs
    Internal.Cryptography.Pal.UnixPkcs12Reader.Decrypt(Microsoft.Win32.SafeHandles.SafePasswordHandle) in UnixPkcs12Reader.cs

Inner Exception 1:
CryptographicException: A certificate referenced a private key which was already referenced, or could not be loaded.

Author:	annerajb
Assignees:	-
Labels:	area-System.Security, untriaged
Milestone:	-

[bartonjs]

We don't have any way of representing the EdDSA keys internally, so we think that the private key blob is invalid.

We'd need to add plumbing to get the certificate to understand that it has an OpenSSL EdDSA key so that it can pass it back to OpenSSL from SslStream. (Workarounds would be possible by writing a custom loader using Pkcs12Info, P/Invoking to OpenSSL to load a EdDSA key object, and using private reflection to force the cert object to know about the private key... but since that involves private reflection it isn't anything that we'd support or guarantee works across updates).

[annerajb]

We don't have any way of representing the EdDSA keys internally, so we think that the private key blob is invalid.

We'd need to add plumbing to get the certificate to understand that it has an OpenSSL EdDSA key so that it can pass it back to OpenSSL from SslStream. (Workarounds would be possible by writing a custom loader using Pkcs12Info, P/Invoking to OpenSSL to load a EdDSA key object, and using private reflection to force the cert object to know about the private key... but since that involves private reflection it isn't anything that we'd support or guarantee works across updates).

Started looking into what would we needed to implement it properly.
The native crypto interop needed new functions to create raw public and private keys.

They where added on openssl 1.1.1 so I had to Install on the dotnet runtime image libssl-dev.

It seems that it may make sense to also create a opensslrawkey class similar to openssleckey.

Since the openssl raw function has uses outside of 25519.

Seems like this would require a api review .since I need to add a new eddsa class which is public and I needed to change it to be able to correctly parse the private key asn.1 format since the existing ecdsa parser fails since the format is different.

[vcsjones]

It seems that it may make sense to also create a opensslrawkey class similar to openssleckey.

I think the intention with EdDSA in OpenSSL is to use PKCS8 / SPKI export functionality, so I would think byte[] would work and the existing members from AsymmetricAlgorithm would work.

Seems like this would require a api review

Yes, it would.

A concern I have is the inability to provide similar functionality on Windows and macOS. While one could theoretically add EdDSA primitive to the S.S.C.OpenSsl package, making it useful in X509Certificate2 would likely mean doing it in such a way that Windows and MacOS could see new public APIs that won't work for them. It would be unfortunate for you to spent a lot of time on this if it was later determined that it cannot be added until at least Windows provides similar functionality.

[filipnavara]

A concern I have is the inability to provide similar functionality on Windows and macOS.

Valid concern. macOS has ed25519 APIs in CryptoKit so in theory that could be done on new enough systems (10.15+). Windows can do ed25519 calculation on custom EC curve but it's hard to make it into something interoperable and useful since it requires both coordinates for the public key and it's likely slow.

I, for one, would really like to see some APIs for EdDSA/Ed25519 (and X25519, which is already tracked in other issues). Even if the default implementation would not be provided on Windows I could use the same API shape and plug-in my NSec-based implementation instead. Obviously it would not be ideal situation but it would still be better than not having the APIs at all.

[vcsjones]

macOS has ed25519 APIs in CryptoKit

Yeah, I really want to figure out the "right" way to wire in / light up CryptoKit for macOS. The Swift-only bindings continues to be the source of trouble.

Though it's worth keeping in mind that supporting the primitive and supporting it in TLS / SslStream as the original issue asked for are different things. To my knowledge, though CryptoKit supports the primitive, SecureTransport and the newer Network framework do not, at least the last time I checked. (And neither CNG/SymCrypto or SChannel do).

From reading it seems that support for 25519 has been requested since 2015

While the Ed25519 and such have existed for a bit of time, RFC 8410 was only published in 2018.