Implement ECDSA Certificate generation

[Sean-Der]

By default generate certificates with NIST P-256. Allow
RSA to still be used by with dtls_generate_rsa_private_key
config option.

Resolves #1978

[lminiero]

Thanks @Sean-Der, you rock! I'll make a couple of tests, and merge in case I don't see issues 👍

[lminiero]

Looks good! I noticed a couple of things I'd like tweaks on (e.g., name of the property and the section it is in), but nothing that needs to wait, so I'll take care of those after I merge.

Thanks again!

[fippo]

this shouldn't take more than SSL_CTX_set_ecdh_auto(somectx, 1);
(if SSL_CTX_set_ecdh_auto is supported. other wise you need a bit work)

paging @ibc for cross-pollination

[lminiero]

@fippo this patch is for generating ECDSA certificates, though. Is that what SSL_CTX_set_ecdh_auto is for?

[ibc]

We already do SSL_CTX_set_ecdh_auto() in mediasoup (whatever it means).

[ibc]

PLEASE STAY AT HOME

[fippo]

ah... it does take more than SSL_CTX_set_ecdh_auto (which is a nice things to do so i'll provide a patch shortly) so what this does looks good.
Lucky me generating the certs with a single line of JS before passing them on :-)

[ibc]

Mmm, I don't remember the rationale about my code, but it does this:

// Enable ECDH ciphers.
// DOC: http://en.wikibooks.org/wiki/OpenSSL/Diffie-Hellman_parameters
// NOTE: https://code.google.com/p/chromium/issues/detail?id=406458
// NOTE: https://bugs.ruby-lang.org/issues/12324
//
// Nothing to be done in OpenSSL >= 1.1.0.
#if (OPENSSL_VERSION_NUMBER >= 0x10100000L)
// For OpenSSL >= 1.0.2.
#elif (OPENSSL_VERSION_NUMBER >= 0x10002000L)
		SSL_CTX_set_ecdh_auto(DtlsTransport::sslCtx, 1);
// Older versions.
#else
		ecdh = EC_KEY_new_by_curve_name(NID_X9_62_prime256v1);

		if (!ecdh)
		{
			LOG_OPENSSL_ERROR("EC_KEY_new_by_curve_name() failed");

			goto error;
		}

		if (SSL_CTX_set_tmp_ecdh(DtlsTransport::sslCtx, ecdh) != 1)
		{
			LOG_OPENSSL_ERROR("SSL_CTX_set_tmp_ecdh() failed");

			goto error;
		}

		EC_KEY_free(ecdh);
		ecdh = nullptr;
#endif

so, for OpenSSL >= 1.0.2 the only that must be done is:

SSL_CTX_set_ecdh_auto(sslCtx, 1);

WHAT'S WRONG WITH THIS?

[lminiero]

We're talking about different things: we've had the same code to enable ECDHE ciphers as well for quite some time already. This patch here is used to generate the certificate Janus itself will use in DTLS: you either specify an existing certificate in the configuration file, or ask Janus to generate one at startup, and this is what this patch affects. Before our certificates were generated using RSA, thanks to a contribution by @saghul, while @Sean-Der now extended it to support generating ECDSA certificates too.

[ibc]

Ah, I see, thanks. Updating my code too.

[ibc]

So, any reason to keep the RSA code other than allowing older OpenSSL versions?

[lminiero]

It's there mostly as a fallback, because it's what we've used for a long time. Since this is a new feature and I'm not 100% sure it works with all OpenSSL versions, I thought it might be safer to keep it there.

[ibc]

Careful here. BN_free(bne); is always called, even if bne = BN_new(); was not called above.

[fippo]

i've had left a papertrail in my own changes. I'm a bureaucrat.
This was added to webrtc in https://bugs.chromium.org/p/webrtc/issues/detail?id=5795 in 2016 and i've been running in production with just ECDSA for months without (reported) issues.

[ibc]

So I've made the same changes to mediasoup by making it always generate certs with EC keys instead of RSA:

versatica/mediasoup@cc0dccc?ts=2

It seems to work and blablabla. Thanks, author!