RSAOpenSSL doesn't appear to support HSM-managed keys?

[andyhopp]

Description

I have an OpenSSL dynamic engine installed that manages keys on an HSM, but when I attempt to use a PFX to create an X509Certificate2 that uses that key, I am presented with an error:
Error occurred during a cryptographic operation.

HasPrivateKey on the X509Certificate2 returns True.

A sample stack trace is below:
Unhandled exception. System.Security.Cryptography.CryptographicException: Error occurred during a cryptographic operation.
at System.Security.Cryptography.RSAOpenSsl.TrySignHash(ReadOnlySpan1 hash, Span1 destination, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding, Boolean allocateSignature, Int32& bytesWritten, Byte[]& signature)
at System.Security.Cryptography.RSAOpenSsl.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
at System.Security.Cryptography.RSAPKCS1SignatureFormatter.CreateSignature(Byte[] rgbHash)
...

Reproducing this error will be tricky unless you have access to an HSM. I am happy to volunteer the use of mine.

Configuration

~/.dotnet/dotnet --list-sdks
3.1.300 [/home/ec2-user/.dotnet/sdk]
Amazon Linux 2, x64

Is there a way to specify the engine that will be used by System.Security.Cryptography.OpenSsl through an appSetting, environment variable, etc? From my naive review of the code, it appears that the native interop EnsureOpenSslInitialized() method only calls OPENSSL_add_all_algorithms_conf().

[ghost]

Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq
Notify danmosemsft if you want to be subscribed.

[andyhopp]

Update: I have implemented a class that uses the ENGINE_* functions to load/initialize/set as the default provider for (RSA/DSA/Cipher/etc.) and would like to contribute this back. Is there someone who would be willing to coach me on this?

[vcsjones]

I assume you mean add new public APIs and classes for other developers to consume, right? If that's the case - the place to start is to go through the API Review Process with a proposal. Basically open up a new issue with un-implemented methods for your new public APIs, give a few examples of usage, etc. An example proposal is here.

[bartonjs]

Even with the API proposal from #37383, how would you expect the PFX load to understand to associate the private key to an HSM? Or are you just hoping it creates a new key, instead of accessing an existing persisted key?

[andyhopp]

Terribly sorry; I had responded to the thread via email, but GitHub didn't attach it
The HSMs I'm familiar with actually use the private key payload as a "pointer" to the HSM-stored key material, and the cryptographic operations are handled by the dynamic engine (or CNG on Windows). This is handled transparently by OpenSSL/CNG, so no code modifications need to be made by the developer (beyond adding the dynamic engine API described above). This works for both PKCS12 and X509Store certs.

[bartonjs]

Do you have an example of such a "pointer"? We don't use OpenSSL's PKCS12/PFX loader any more, but manually associate the X.509 and PKCS8 data.

[vcsjones]

Is there a strong use case for loading dynamic engines programmatically as opposed to adding it to openssl.cnf?

Something like:

openssl_conf = openssl_init

[ openssl_init ]
engines = engine_section

[ engine_section ]
myhsm = myhsm_section

[ myhsm_section ]
engine_id = myhsm
dynamic_path = /usr/lib/engines/myhsm.so
default_algorithms = ALL
init = 1

I think that would be enough to get an dynamic engine loaded and initialized.

[bartonjs]

Given that the author didn't respond to the request for more information 2 years ago, I'm going ahead and closing this.

[benlongo]

Could this be reopened? I think the issue still stands

[bartonjs]

@benlongo This issue never had enough information to investigate it, so reopening will do no good. If you have information to add you are welcome to add it here (which might lead to reopening it), or just create a new issue.

[lukeschlather]

I'm using an HSM with Google's libpkcs11. https://cloud.google.com/kms/docs/reference/pkcs11-library

You configure your openssl with the engine to use a specific Google KMS keyring. Then you can create a key on the keyring, and reference it by name. Google has docs for setting it up. The other APIs I've seen take the name of a certificate file, and the name of a keyfile. If the keyfile name looks like "pkcs11:object=" then it passes the keyname off to the engine.

Do you have an example of how you "manually associate the X.509 and PKCS8 data?"

[bartonjs]

@lukeschlather The way to associate them if you already have a key and a cert, is cert.CopyWithPrivateKey(key). My reference to "manually" above is that when reading a PFX (with a private key) we see the certificate data, and the private key as PKCS#8 data.

And it's a request that if there's a format that OpenSSL supports for associating a PFX with a named key (HSM/TPM/external-file/whatever) that someone share with us a PFX that does that.

[lukeschlather]

Does PFX support storing a reference to a key rather than the key itself, or is the assumption that the PFX would just have the key data be ignored?

I am also confused a bit by cert.CopyWithPrivateKey(key) - I see that API but it's unclear to me if there's a constructor for System.Security.Cryptography.ECDiffieHellman which will take an object reference. It looks like it explicitly requires an operating system filehandle? I think I would want something like System.Security.Cryptography.OpenSSLEngineKey(keyStringId)

[bartonjs]

it looks like it explicitly requires an operating system filehandle

it requires a SafeEvpPKeyHandle, representing an EVP_PKEY*.

Does PFX support storing a reference to a key rather than the key itself

As far as I know, no.

I think I would want something like System.Security.Cryptography.OpenSSLEngineKey(keyStringId)

That's #55356 (not yet approved)

[lukeschlather]

I'm trying to do this:

DynamicEngine engine = new DynamicEngine("pkcs11", "libkmsp11.so");
engine.Initialize();
engine.SetDefaults(EngineDefaults.ECDSA);

SafeEvpPKeyHandle privateKeyHandle = engine.GetPrivKey("key-name");
ECDiffieHellmanOpenSsl privateKey = new ECDiffieHellmanOpenSsl(privateKeyHandle);
X509Certificate2 publicKey = X509Certificate2.CreateFromPemFile("certificate.pem");

X509Certificate2 certificate = publicKey.CopyWithPrivateKey(privateKey);

I am working from https://github.com/Dvergatal/OpenSsl.DynamicEngine (though I've made some minor changes trying to wrap my head around this.)

I'm not entirely sure I'm properly loading the private key, but it's moot because it looks like X509Certificate2.CreateFromPemFile("certificate.pem") doesn't like being invoked with a pem which does not include a private key.

It fails with this error:

Unhandled exception. System.Security.Cryptography.CryptographicException: The key contents do not contain a PEM, the content is malformed, or the key does not match the certificate.
   at System.Security.Cryptography.X509Certificates.X509Certificate2.ExtractKeyFromPem[TAlg](ReadOnlySpan`1 keyPem, String[] labels, Func`1 factory, Func`2 import)
   at System.Security.Cryptography.X509Certificates.X509Certificate2.CreateFromPem(ReadOnlySpan`1 certPem, ReadOnlySpan`1 keyPem)
   at System.Security.Cryptography.X509Certificates.X509Certificate2.CreateFromPemFile(String certPemFilePath, String keyPemFilePath)

Which also suggests loading a mismatched key would cause a similar error. Just to verify it wasn't something weird about the preexisting cert I'm trying to load, I generated a fresh self-signed cert with:

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 365

and just trying to load that cert in isolation with X509Certificate2("cert.pem") yields the same error.

[lukeschlather]

I'm not entirely sure what is going on with CreateFromPemFile but I realized that my pem includes a chain of 3 certificates. I converted it to a pfx which didn't help, it still contained 3 certificates, and the X509Certificate2 constructor inexplicably only loads the root cert. It's good that it fails, but really it should just refuse to do anything rather than picking the last cert in the collection.

Anyway loading the collection is the way to go. I was mislead because until recently Kestrel only supported providing a single X509Certificate2 object rather than

            X509Certificate2Collection collection = new X509Certificate2Collection();
            collection.ImportFromPemFile(PEM_PATH);

And I can attach the dynamic engine key with X509Certificate2 certificate = collection[0].CopyWithPrivateKey(privateKey);