Raise the minimum RSA modulus size to 1024 bits.

[ryancdotorg]

The first publicly known instance of a 512 bit RSA modulus being factored was in 1999.

In 2009, an individual cracked a 512 bit RSA modulus by themself with a single PC.

The "Factoring as a Service" paper, showing that such keys could be broken in a few hours for tens of dollars, came out in 2015.

In 2018, OpenSSL v1.1.1 set its minimum RSA modulus size to 512 bits. This is still the minimum as of OpenSSL v3.3.1.

People still use 512 bit RSA keys without realizing that they're catastrophically weak.

This should be easy to stop.

I propose the following deprecation stages:

1. OpenSSL limits RSA key generation to a minimum of 1024 bits, but still allows such keys to be loaded/used.
2. OpenSSL drops support for verifying signatures for RSA keys smaller than 1024 bits.
3. OpenSSL fully drops support for RSA keys smaller than 1024 bits.

A more conservative approach could temporarily include an option in openssl.conf to temporarily re-enable dangerously weak keys while systems are updated.

[nhorman]

(1) seems pretty reasonable to do right now
(2) and (3) seem plausible, but I imagine that would likely want to be done on a major release boundary.

Would you be willing to submit a PR for this? (1) is likely pretty straightforward to accomplish by updating RSA_MIN_MODULUS_BITS

[ryancdotorg]

RSA_MIN_MODULUS_BITS is checked in ossl_rsa_sp800_56b_check_public, which is in turn called from ossl_rsa_validate_public.

Doing some tests though, it doesn't seem like anything breaks when trying to load/use a 512 bit RSA key when compiled with RSA_MIN_MODULUS_BITS 1024.

I can definitely submit a PR to make that change, though I'd like to understand under what circumstances ossl_rsa_sp800_56b_check_public is called.

Might also be worth updating DH_MIN_MODULUS_BITS at the same time.

[t8m]

openssl/project#809

[gilles-peskine-arm]

As a data point on what the ecosystem is doing, Mbed TLS enforces 1024 bits for generation since 2023, i.e. it's already at stage 1. Our motivating example was a confusion between bits and bytes; it also solved the problem of confusing sensible ECC or symmetric bit-sizes with RSA bit-sizes. I've put stages 2/3 up for consideration on our planning board for our next major release.