TLS1.2 should support RSA-PSS certificates

[lrstewart]

Problem:

s2n-tls restricts the rsa_pss_pss signature algorithms to TLS1.3 by setting the minimum protocol version to TLS1.3.

However, the RFC says:

Implementations that advertise support for RSASSA-PSS (which is
mandatory in TLS 1.3) MUST be prepared to accept a signature using
that scheme even when TLS 1.2 is negotiated. In TLS 1.2,
RSASSA-PSS is used with RSA cipher suites.

I think we interpreted that as TLS1.2 needing to support rsa_pss_rsae (RSA certs + PSS padding), not as TLS1.2 needing to also support rsa_pss_pss (RSA-PSS certs + PSS padding). However, I now think that it means that a cipher suite that specifies RSA auth can also use an RSA-PSS certificate for auth. The RFC is not very exact in its use of the term "RSASSA-PSS".

I verified that openssl supports RSA-PSS certs with RSA-auth cipher suites in TLS1.2. openssl s_client -connect localhost:8000 -tls1_2 can connect to openssl s_server -accept localhost:8000 -tls1_2 -cert tests/pems/rsa_pss_2048_sha256_leaf_cert.pem -key tests/pems/rsa_pss_2048_sha256_leaf_key.pem, negotiating "ECDHE-RSA-AES256-GCM-SHA384". That means openssl interpreted the RFC as requiring TLS1.2 support for RSA-PSS certs via RSA-auth cipher suites.

Additionally, the use of RSA-PSS certs for client auth wouldn't even be restricted by any auth limitations of TLS1.2 cipher suites.

Solution:

We should update TLS1.2 to support RSA-PSS for both client auth and for server auth when a cipher suite is chosen that supports RSA auth and a key exchange algorithm other than RSA kex. RSA-PSS certs can't be used for RSA kex-- that's kind of the point of RSA-PSS.

Note: double check how this would interact with our support for openssl-1.0.2 as a libcrypto, which doesn't support RSA-PSS certs. We already support PSS padding with TLS1.2, which openssl-1.0.2 also doesn't support, so it's probably fine.

• Does this change what S2N sends over the wire? Yes, the server will offer rsa_pss_pss signature algorithms in the TLS1.2 certificate request message.
• Does this change any public APIs? No.
• Which versions of TLS will this impact? TLS1.2.

Requirements / Acceptance Criteria:

What must a solution address in order to solve the problem? How do we know the solution is complete?

• RFC links: Links to relevant RFC(s)
• Related Issues: Link any relevant issues
• Will the Usage Guide or other documentation need to be updated?
• Testing: How will this change be tested? Call out new integration tests, functional tests, or particularly interesting/important unit tests.
  • Will this change trigger SAW changes? Changes to the state machine, the s2n_handshake_io code that controls state transitions, the DRBG, or the corking/uncorking logic could trigger SAW failures.
  • Should this change be fuzz tested? Will it handle untrusted input? Create a separate issue to track the fuzzing work.

Out of scope:

Is there anything the solution will intentionally NOT address?