[p2p] Why rollout homemade transport security instead of TLS/Noise?

[patrick-ogrady]

Link: https://github.com/cometbft/cometbft/blob/26f43ce6c6a482db9488425f2dbc448a3e95f3e6/p2p/conn/secret_connection.go#L57-L66

[patrick-ogrady]

Great question (and thanks for sharing some context from Cosmos)!

From the reasons stated in the questions above, it's unclear why risking rolling out new transport security is worth it. Is this because of the need to reuse consensus identity for peering without translation? To my knowledge, one can simultaneously use ed25510 keys for both peer and consensus identities.
It may require translating and using self-signed certificates for TLS, but the problem is unclear, especially compared to the risk of creating a new security protocol. I favor reducing layers of abstraction and dependencies but not for something as low-level and critical as transport security, especially when the standart is widely adopted and has been through thousands of expert eyes.

TLS may sound like a "no brainer" but I have found it to be significantly more trouble than its worth over the last few years. Things got so bad, we (mostly Stephen) ended up implementing our own TLS certificate generation/parsing:

• https://github.com/ava-labs/avalanchego/blob/75fc684534b4db2c03e6317efb26317a3adfcd95/staking/tls.go#L23-L75
• https://github.com/ava-labs/avalanchego/blob/75fc684534b4db2c03e6317efb26317a3adfcd95/staking/parse.go#L55-L167

Like many of the tradeoffs made by commonware-*, they are born from pain, late nights, and banging my head against the wall (not out of a desire to be "novel" or "cool"). TLS has a TON of functionality (IMHO too much for its own good and way too much for byzantine-robust p2p).

[patrick-ogrady]

I believe robust specifications need to be simple and each TLS specification (TLSv1, TLSv1.1, TLSv1.2, TLSv1.3) is about ~100 pages. To support the "required functionality" of TLS, implementations are massive and often times have serious vulnerabilities (e.g. many of the Golang security releases over the past few years have been TLS/x509 fixes). To get some sense of the upkeep (even just in go1.20, check out the crypto/tls section of Filippo's blogs: https://words.filippo.io/dispatches/go1-20/).

To support much of its functionality, TLS employs a flexible certificate scheme (which allows for all sorts of fancy shit, like the crypto schemes you've mentioned) and protocol negotiation (to determine which options should be used to instantiate the transport layer). In a mission-critical application, you don't want either of these. You want 1 way to connect with rigidly defined cryptographic rules (which may be outside of the list TLS supports) where each participant sends 1 message to instantiate an encrypted channel (not the multiple messages TLS requires, which can be gamed in all sorts of crazy ways).

Last but not least, TLS implementations aren't meant to be deterministically simulated. Everything in commonware-* can be simulated deterministically without a custom hypervisor (using commonware-runtime). In practice, this makes comprehensive testing/efficient debugging much more attainable.

Hope that helps...but no, definitely not going the TLS, QUIC, Noise, etc. route. You can obviously introduce your own instance of commonware-p2p to do so though (if you hold a different opinion).

[patrick-ogrady]

To make reuse of the connection stream provided by commonware-p2p::authenticated easier, I created an issue to move it into its own crate: #139