Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenSSL3 provider support #540

Open
wants to merge 36 commits into
base: master
Choose a base branch
from
Open

OpenSSL3 provider support #540

wants to merge 36 commits into from

Conversation

patzol768
Copy link

Hi Everyone,
here are some changes to begin supporting OpenSSL3 providers.

Key generation, sign + verify, encrypt + decrypt are available, though requires a lot-lot-lot of additional testing and maybe some fixes. Please check the prov-openssl.sh in tests. I have no access to real HSM devices, hence everything was made with Opencrytoki's SW token.

patzol768 and others added 30 commits January 3, 2023 00:34
* Missing function declarations
* Incorrect automake rule
* return values for non-void return
* provider lib compiles only if OpenSSL3
* obj list for MS builds
* unused variables
* signedness in loops
* do not compile provider library if not OpenSSL3
* Revert autoconfig required version number
* Adding basic OpenSSL 3 provider capabilities

* Remove unwanted file

* Remove unwanted files

* Correcting typos

* Correcting CI check failures

* Missing function declarations
* Incorrect automake rule

* Correcting CI errors

* return values for non-void return
* provider lib compiles only if OpenSSL3
* obj list for MS builds

* Correcting CI issues

* unused variables
* signedness in loops
* do not compile provider library if not OpenSSL3

* Correcting accidental change

* Revert autoconfig required version number

* Fix DES reference

* Revert configure.ac accidental changes

* Deprecated algorithm warning corrected
* Adding basic OpenSSL 3 provider capabilities

* Remove unwanted file

* Remove unwanted files

* Correcting typos

* Correcting CI check failures

* Missing function declarations
* Incorrect automake rule

* Correcting CI errors

* return values for non-void return
* provider lib compiles only if OpenSSL3
* obj list for MS builds

* Correcting CI issues

* unused variables
* signedness in loops
* do not compile provider library if not OpenSSL3

* Correcting accidental change

* Revert autoconfig required version number

* Fix DES reference

* Revert configure.ac accidental changes

* Deprecated algorithm warning corrected

* Fix module unload crash, add some tests

* Test fix and error reporting fixes

* Fix Windows modules check

* Fix Mac compile errors
@mtrojnar
Copy link
Member

This PR is too large to merge as a single unit. Please split it into individual features, beginning with the libp11 API extension, and submit each feature as a separate pull request.

@mtrojnar
Copy link
Member

mtrojnar commented Nov 2, 2024

@patzol768 Are you still interested in getting this feature merged? It would be great to have it merged, but there are way too many unrelated changes in this PR. I'm fine with merging an unstable new feature as long as it doesn't break the existing functionalities, which are used for business-critical applications. Therefore, any changes to the existing code need to be justified, reviewed and thoroughly tested.

@dengert
Copy link
Member

dengert commented Nov 9, 2024

Nice to see mods to libp11 to use OpenSSL 3 providers as engines will be deprecated some time soon. There is at least one other project that is developing a PKCS11 provider: https://github.com/latchset/pkcs11-provider It is quite active with multiple contributors.

I would suggest that any names for modules or env names or used in opensc,cnf files, include "libp11" or "LIBP11"in the names. For example The name pkcs11prov could be libp11prov or `pkcs11-libp11prov' of something to
The engines and providers already are very confusing.

I am building on Ubuntu-24.04 with OpenSSL 3.3.2 to install in /opt/ossl-3.3.2/ and note that OpenSSL installs modules in
/opt/ossl-3.3.1/lib/ossl-modules legacy.so' but this PR installs pkcs11prov.so in/opt/ossl-3.3.1/lib. configure.acsome code to install engines in engines-3. Can the configure be changed to store intoossl-modules`?

PKCS11FORCELOGIN=1 \
FORCELOGIN=1 \
PKCS11MODULE=/opt/ossl-active/lib/opensc-pkcs11.so \
PKCS11VERBOSE=1 \
VERBOSE=1 \
OPENSSL_MODULES=/opt/ossl-active/lib/ossl-modules /opt/ossl-active/bin/openssl req -verbose -provider legacy -provider pkcs11prov -sha256 -new -key 'pkcs11prov:type=private;id=01;pin-value=123456' -out cards/8103.myreq.9A.pem -text
Using configuration from /tmp/genreq.131011.openssl.conf
Could not open file or uri for loading private key from pkcs11prov:type=private;id=01;pin-value=123456
801BAD21857B0000:error:80000002:system library:file_open:No such file or directory:../openssl/providers/implementations/storemgmt/file_store.c:263:calling stat(pkcs11prov:type=private;id=01;pin-value=123456)
801BAD21857B0000:error:1608010C:STORE routines:inner_loader_fetch:unsupported:../openssl/crypto/store/store_meth.c:360:No store loader found. For standard store loaders you need at least one of the default or base providers available. Did you forget to load them? Info: Global default library context, Scheme (pkcs11prov : 0), Properties (<null>)

The opensc-debug.log shows opensc-pkcs11.so was loaded, and accessed the card and last PKCS11 line was:
"C_GetTokenInfo: C_GetTokenInfo(0) returns CKR_OK"

Looks like OpenSSL is expecting pkcs11prov to have a store loader

@dengert
Copy link
Member

dengert commented Nov 9, 2024

And a rebase of this PR would be good.

@dengert
Copy link
Member

dengert commented Nov 9, 2024

I have gotten further using gdb --args /opt/ossl-3.3.1/bin/openssl req -verbose -provider pkcs11prov -provider default -sha256 -new -key pkcs11://pkcs11:type=private\;id=%01 -out cards/8103.myreq.9A.pem -text

Thread 1 "openssl" hit Breakpoint 1, EVP_MD_CTX_copy_ex (out=0x5555556fcb20, in=0x5555556f4d80) at ../openssl/crypto/evp/digest.c:643
643	            ERR_raise(ERR_LIB_EVP, EVP_R_NOT_ABLE_TO_COPY_CTX);
(gdb) where
#0  EVP_MD_CTX_copy_ex (out=0x5555556fcb20, in=0x5555556f4d80) at ../openssl/crypto/evp/digest.c:643
#1  0x00007ffff7a1265c in EVP_DigestSignFinal (ctx=0x5555556f4d80, sigret=0x5555556eb260 "k\360=", siglen=0x7fffffffd048) at ../openssl/crypto/evp/m_sigver.c:550
#2  0x00007ffff78d6ff1 in ASN1_item_sign_ctx (it=<optimized out>, algor1=<optimized out>, algor1@entry=0x5555556eaa98, algor2=<optimized out>, algor2@entry=0x0, signature=0x5555556fc440, data=data@entry=0x5555556eaa60, ctx=ctx@entry=0x5555556f4d80) at ../openssl/crypto/asn1/a_sign.c:275
#3  0x00007ffff7bab9de in X509_REQ_sign_ctx (x=x@entry=0x5555556eaa60, ctx=ctx@entry=0x5555556f4d80) at ../openssl/crypto/x509/x_all.c:141
#4  0x0000555555604344 in do_X509_REQ_sign (x=x@entry=0x5555556eaa60, pkey=pkey@entry=0x5555556a2c80, md=md@entry=0x7fffffffdf6c "sha256", sigopts=sigopts@entry=0x0) at ../openssl/apps/lib/apps.c:2372
#5  0x00005555555d2841 in req_main (argc=<optimized out>, argv=<optimized out>) at ../openssl/apps/req.c:886
#6  0x00005555555c70a1 in do_cmd (prog=prog@entry=0x55555567b0d0, argc=argc@entry=13, argv=argv@entry=0x7fffffffdac0) at ../openssl/apps/openssl.c:426
#7  0x000055555559ed78 in main (argc=<optimized out>, argv=<optimized out>) at ../openssl/apps/openssl.c:307
(gdb) p in
$14 = (const EVP_MD_CTX *) 0x5555556f4d80
(gdb) p *in
$15 = {reqdigest = 0x7ffff7d15500 <sha256_md>, digest = 0x5555556e24f0, engine = 0x0, flags = 0, md_data = 0x0, pctx = 0x5555556792f0, update = 0x0, algctx = 0x5555556fc8c0, fetched_digest = 0x5555556e24f0}
(gdb) p *in.digest
$16 = {type = 672, pkey_type = 0, md_size = 32, flags = 0, origin = 0, init = 0x0, update = 0x0, final = 0x0, copy = 0x0, cleanup = 0x0, block_size = 64, ctx_size = 0, md_ctrl = 0x0, name_id = 102, type_name = 0x5555556e9ff0 "SHA2-256", description = 0x0, prov = 0x55555567a150, refcnt = {val = 4}, newctx = 0x7ffff7e815e7 <p11_digest_SHA256_newctx>, dinit = 0x7ffff7e81075 <p11_digest_init>, dupdate = 0x7ffff7e81130 <p11_digest_update>, dfinal = 0x7ffff7e811bf <p11_digest_final>, dsqueeze = 0x0, digest = 0x7ffff7e8160a <p11_digest_SHA256_digest>, freectx = 0x7ffff7e80ebe <p11_digest_freectx>, dupctx = 0x7ffff7e80f70 <p11_digest_dupctx>, get_params = 0x7ffff7e81663 <p11_digest_SHA256_get_params>, set_ctx_params = 0x0, get_ctx_params = 0x0, gettable_params = 0x7ffff7e81449 <p11_digest_gettable_params>, settable_ctx_params = 0x0, gettable_ctx_params = 0x0}

This appears as the provider is calling the pkcs11 module to do the SHA256 digest for the CSR.

3985 P:137512; T:0x140737352706944 16:28:20.126 [opensc-pkcs11] pkcs15-pin.c:730:sc_pkcs15_get_pin_info: returning with: 0 (Success)
3986 P:137512; T:0x140737352706944 16:28:20.126 [opensc-pkcs11] pkcs11-session.c:303:C_GetSessionInfo: C_GetSessionInfo(0x5555556f6810) = CKR_OK
3987 P:137512; T:0x140737352706944 16:28:20.126 [opensc-pkcs11] pkcs11-object.c:543:C_DigestInit: C_DigestInit(hSession=0x5555556f6810)
3988 P:137512; T:0x140737352706944 16:28:20.126 [opensc-pkcs11] mechanism.c:299:sc_pkcs11_md_init: called
3989 P:137512; T:0x140737352706944 16:28:20.126 [opensc-pkcs11] misc.c:268:session_start_operation: called
3990 P:137512; T:0x140737352706944 16:28:20.127 [opensc-pkcs11] misc.c:269:session_start_operation: Session 0x5555556f6810, type 3
3991 P:137512; T:0x140737352706944 16:28:20.127 [opensc-pkcs11] mechanism.c:319:sc_pkcs11_md_init: returning with: 0 (Success)
3992 P:137512; T:0x140737352706944 16:28:20.127 [opensc-pkcs11] pkcs11-object.c:548::C_DigestInit: C_DigestInit() = CKR_OK
3993 P:137512; T:0x140737352706944 16:34:30.435 [opensc-pkcs11] misc.c:290:session_get_operation: called
3994 P:137512; T:0x140737352706944 16:34:30.435 [opensc-pkcs11] mechanism.c:339:sc_pkcs11_md_update: returning with: 0 (Success)
3995 P:137512; T:0x140737352706944 16:34:30.436 [opensc-pkcs11] pkcs11-object.c:615:C_DigestUpdate: C_DigestUpdate() = CKR_OK
3996 P:137512; T:0x140737352706944 16:35:25.097 [opensc-pkcs11] pkcs11-session.c:58:C_OpenSession: C_OpenSession(0x0)
3997 P:137512; T:0x140737352706944 16:35:25.097 [opensc-pkcs11] slot.c:472:slot_get_token: Slot(id=0x0): get token
3998 P:137512; T:0x140737352706944 16:35:25.097 [opensc-pkcs11] slot.c:490:slot_get_token: Slot-get-token returns OK
3999 P:137512; T:0x140737352706944 16:35:25.097 [opensc-pkcs11] pkcs11-session.c:94:C_OpenSession: C_OpenSession handle: 0x5555556fcb70
4000 P:137512; T:0x140737352706944 16:35:25.097 [opensc-pkcs11] pkcs11-session.c:97:C_OpenSession: C_OpenSession() = CKR_OK

And that is as much of the opensc-debug.log that has been written to disk. What looks strange is extra calls to C_OpenSession as C_DigestInit, C_DigestUpdate and C_DigestFinal (which is not in the log) should be run using the same session. This looks like a bug.

The OpenSC PKCS11 module lists these mechanisms, note that since there is no "hw" bit set, the pkcs11 module is doing the operation in software, i.e. CK_MECHANISM_INFO flag CKF_HW is not set. So should the digests be done by OpenSSL i.e. default-provider rather then the pkcs11-provider?

@dengert
Copy link
Member

dengert commented Nov 10, 2024

Using OpenSSL 3.3.1 and adding digest_disabled = 1 and rand_disabled = 1 to openssl.cnf a CSR was created using OpenSC module with a PIV card!

@mouse07410
Copy link
Contributor

Using OpenSSL 3.3.1 and adding digest_disabled = 1 and rand_disabled = 1 to openssl.cnf a CSR was created using OpenSC module with a PIV card!

You mean - adding to the pkcs11 provider section? Perhaps, posting here a direct quote from your openssl.cnf would be useful?

@dengert
Copy link
Member

dengert commented Nov 10, 2024

@mouse07410, I just got it working once and need to clean up what I did, as the openssl.cnf file was based on 1.1.x to be used only for openssl req called from some scripts I used to generate CSR for test PIV cards: 9A, 9C, 9D and 9E which differ in usage and extensions.

I basically followed:
https://github.com/patzol768/libp11/tree/master?tab=readme-ov-file#pkcs-11-module-configuration---openssl3-provider
and used a combination of ENV variables and provider params found here:
https://github.com/patzol768/libp11/blob/master/src/prov_front.c#L204-L218
and here:
https://github.com/patzol768/libp11/blob/master/src/prov_front.c#L150-L176
As noted in the previous comments, I had to add

The build built the engine and installed in DESTDIR/lib/engines-3/pkcs11.so and libpkcs11.so -> pkcs11.so
and built the provider and installed in DESTDIR/lib/pkcs11prov.so but OpenSSL installs other providers in DESTDIR/lib/openssl-modules So I copied pkcs11prov.so to DESTDIR/lib/openssl-modules

So it looks like this PR adds provider support, without changing engine support! Don't know if both libp11 engine and provider can be used in the same application. I think you could have both in the same openssl.cnf But one or the other should workin an application.

I spent the last few days seeing if this PR works or not, and it works for the case, and it looks like it could easily be used by others libp11 users to convert from engine to provider.

And it looks like using this provider with some simple additional mods to libp11 it could also support Ed25519, X25519 Ed448 and X448 too. See OpenSC/OpenSC#3090

The changed to opensc.cnf looked like:

[providers_sect]
pkcs11 = pkcs11_sect
#default = default_sect
#legacy = legacy_sect

[pkcs11_sect]
# Check prov_front.c for supported parameters
#
# Some parameters could be overridden with environment variable value. See prov_front.c
# Like: PKCS11MODULE, PKCS11VERBOSE, PKCS11FORCELOGIN, ... 
PKCS11VERBOSE = 1 
PKCS11FORCELOGIN = 1 
#
# alternative name (if we were compiling the provider as pkcs11.so, we won't need this)
identity = pkcs11prov
# name of the libp11 pkcs11 provider
####module = @@OPENSSL_PROVIDER@@
# like MODULE_PATH in case of the libp11 pkcs11 engine
######pkcs11module = @@MODULE@@
# libp11 log level (0 - off, >0 - on) 
verbose = 1 

# force login to the PKCS#11 module
force_login = 1 

#use OpenSSL digest commands
digest_disabled = 1 

# do not use pkcs11 module  for random number
rand_disabled = 1 

And run this command where: ossl-active is ossl-3.3.1

PKCS11FORCELOGIN=1 \
FORCELOGIN=1 \
PKCS11MODULE=/opt/ossl-active/lib/opensc-pkcs11.so \
PKCS11VERBOSE=1 \
VERBOSE=1 \
OPENSSL_MODULES=/opt/ossl-active/lib/ossl-modules \
gdb --args  /opt/ossl-3.3.1/bin/openssl req -verbose -provider pkcs11prov -provider default -sha256 -new -key pkcs11://pkcs11:type=private\;id=%01 -out cards/8103.myreq.9A.pem -text

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants