add a WebTransport spec #404

marten-seemann · 2022-04-04T10:24:51Z

This PR adds a basic spec for WebTransport, based on https://pl-strflt.notion.site/Enabling-WebTransport-across-the-libp2p-network-c6849c7252a1469a828a6c3e4b6abcad.

Most important things to watch out for when reviewing:

The multiaddr format: The multiaddr contains the hashes of the certificates used, e.g. /ip4/1.2.3.4/udp/443/quic/webtransport/<hash1><hash2>. Is this a sane construction?
We need to run a libp2p handshake on the WebTransport connection to verify peer IDs.

UPDATE 2022-07-26: Updated the link to the Notion page. I was mistaken in believing that the previous link was already public.

webtransport/README.md

willscott · 2022-04-05T08:24:09Z

webtransport/README.md

+
+## Securing Streams
+
+All streams other than the stream used for the security handshake are protected using Salsa20. Two (symmetric) keys are derived from the master secrect established during the handshake, one for sending and for receiving on the stream.


It's unfortunate that there's no concept of the client identity at the transport level, so that we don't need an additional wrapping of encryption.

We should see if that's an API that can be proposed at the spec level.

We don't need that and we don't need double encryption. We can sufficiently identify the connection with the server certificate. Basically:

The client knows it's talking to a server with a key that matches the certificate hash.

The client then sends a challenge of (cert_hash, client_pid, server_pid, client_salt)

The server responds with server_salt, SIG(HASH(cert_hash || client_pid || server_pid || client_salt || server_salt)) to authenticate their end.

The client verifies this, then signs the same thing (maybe it signs the signature too?).

In step 3, the server ties their peer ID to the connection.

In step 4, the client ties their peer ID to the connection.

The client knows it's talking to a server with a key that matches the certificate hash.

That is correct iff it received the certificate hash in a signed peer record. Otherwise, the MITM could have generated a multiaddr containing its own cert hash.
Is that something we can require? Does js-libp2p support signed peer records?

The client then sends a challenge of (cert_hash, client_pid, server_pid, client_salt)

Note: Unfortunately, the client doesn't know which cert_hash was used by WebTransport. It only knows that one of the hashes was considered valid, so it would have to send all of the cert hashes to the server.

The server responds with server_salt, SIG(HASH(cert_hash || client_pid || server_pid || client_salt || server_salt)) to authenticate their end.

It is crucial that the server validates that it owns all cert hashes, otherwise a MITM attack is trivial. Furthermore (unless the certificate hash was signed in step 1), the client has to trust that the server actually performs this verification. Note that this kind of trust is not necessary when using TLS or Noise.

Here's a diagram of that handshake:

@Kubuxu Would you mind taking a look at this proposal (the handshake that @Stebalien suggested in particular)?

That is correct iff it received the certificate hash in a signed peer record. Otherwise, the MITM could have generated a multiaddr containing its own cert hash.

It doesn't need to be in a signed peer record, the server signs (and, as you noted, validates) this in step 2.

Note: Unfortunately, the client doesn't know which cert_hash was used by WebTransport. It only knows that one of the hashes was considered valid, so it would have to send all of the cert hashes to the server.

Makes sense.

It is crucial that the server validates that it owns all cert hashes, otherwise a MITM attack is trivial. Furthermore (unless the certificate hash was signed in step 1), the client has to trust that the server actually performs this verification. Note that this kind of trust is not necessary when using TLS or Noise.

Yes, the server needs to check it. But it's definitely not the clients problem if the server doesn't check. The server could just as easily leak their key.

It doesn't need to be in a signed peer record, the server signs (and, as you noted, validates) this in step 2.

Fair enough. As long as the client trusts the server to perform this check, we're good. What makes me a little bit nervous is that the client on its own can't verify that the connection isn't MITM'ed, which is different from the security guarantees you get from a regular TLS handshake. We probably can live with it (and as you say, the server could leak the key anyway), but it is a somewhat weaker security guarantee.

Let's set goals for this protocol, it needs to bind: the current channel and both PeerIDs.
TLS provides facilities for this by means of ExportKeyingMaterial. Result from it is unique per established channel, so if someone were to try to MitM the session, ExportKeyingMaterial result will be different on either side.

I think, but I need someone else to check it, we can just do:
Both sides compiute: ekm = ExportKeyingMaterial("libp2p-auth"), ekm is now the same shared secret on both sides, if they are using the same channel.
Then the Client sends the signature Signature(ekm || ServerPeerID, ClientPrivKey) and its pubkey. Server does the same, sends Signature(ekm || ClientPeerID, ServerPrivKey) and its pubkey.

Then to verify both sides verify that pubkeys match PeerIDs and they verify if the signature received is of ekm || MyPeerID. The ekm is never transmitted over wire, the MyPeerID is not transferred but a local value.

If either side uncovers an invalid signature, they terminate the connection.
If signature checks pass, the connection is assumed secure.

This binds the WebTransport TLS channel to both PeerIDs.
Binding to ekm prevents MitM and binding to other side PeerID prevents impersonation.
The local side PeerID binding is not necessary as that is achieved by the signature.

Questions is whether something like ExportKeyingMaterial is available on the browser end. I think the answer is yes.

EDIT: add other side PeerID binding.

PubKeys can also be exchanged out of band, IIRC libp2p has some other facilities for it.

ExportKeyingMaterial is unavailable on the browser side making my proposal moot.

I think @Stebalien 's proposal can work. It is just there are a lot of details to work out that I don't feel comfortable deciding on (in essence we are rolling our own secure channel protocol, just without encryption).

An alternative to that proposal could be to reuse existing implementation of Noise secure channel to establish a secure session within WebTransport, for the purpose of co-authentication and authentication of the cert_hash.
The proposal would be to start a libp2p-noise channel over WebCrypot and add cert_hash into the Data field of HandshakePayload. An input of a predicate to assert onto the Data field would need to be added into libp2p-noise.

I think this is the lowest friction and risk solution.

webtransport/README.md

Stebalien · 2022-04-10T13:43:03Z

webtransport/README.md

+
+## Securing Streams
+
+All streams other than the stream used for the security handshake are protected using Salsa20. Two (symmetric) keys are derived from the master secrect established during the handshake, one for sending and for receiving on the stream.


We don't need that and we don't need double encryption. We can sufficiently identify the connection with the server certificate. Basically:

The client knows it's talking to a server with a key that matches the certificate hash.

The client then sends a challenge of (cert_hash, client_pid, server_pid, client_salt)

The server responds with server_salt, SIG(HASH(cert_hash || client_pid || server_pid || client_salt || server_salt)) to authenticate their end.

The client verifies this, then signs the same thing (maybe it signs the signature too?).

In step 3, the server ties their peer ID to the connection.

In step 4, the client ties their peer ID to the connection.

webtransport/README.md

marten-seemann · 2022-04-17T19:38:55Z

I updated the spec, incorporating the ideas of @Stebalien and @Kubuxu for securing the connection and avoiding double-encryption:

I introduced a /certhash multiaddr component, which can be repeated to transmit multiple certificate hashes.
We now use Noise to perform the libp2p handshake, and transmit the list of certificate hashes the client was willing to accept as (unencrypted, but integrity-protected) payload of the first handshake message.
The server verifies that it possesses certificates matching ALL of the certificate hashes.
This also means that we won't be able to accept CA-signed certificates any more, as the Noise handshake described above is needed to tie the outer (WebTransport, TLS) connection to the Noise handshake. This simplifies the spec.

Kubuxu · 2022-04-22T10:18:31Z

If we really wanted we could accept CA-signed certs but as self-signed certs are the primary use case, we might as well use just self-signed.

Kubuxu · 2022-04-22T10:21:15Z

webtransport/README.md

+}
+```
+
+On receipt of the `e` message, the server MUST verify the list of certificate hashes. If the list is empty, it MUST fail the handshake. For every certificate in the list, it checks if it possesses a certificate with the corresponding hash. If so, it continues with the handshake. However, if there is even a single certificate hash in the list that it cannot associate with a certificate, it MUST abort the handshake.


Thinking about it more, there is a small issue with it.
It would be trivial for me to make any node impossible to connect to via WebTransport by, as 3rd party, announcing additional multiaddr with some random certhash.

I don't have good solution for this, especially if WebTransport browser APIs don't give us any feedback on which certhash was used.

Not sure if I understand the attack you're suggesting. You're saying that an attacker can add a /certhash/<attacker controlled hash> to a WebTransport multiaddr, right? How is it different from an attacker modifying a TCP or QUIC address, changing the port number? That would prevent the victim from establishing a TCP / QUIC connection to that node as well.

I guess the answer to that is: the only multiaddr you can actually trust is an address transferred in a signed peer record. For all other addresses, the best you can do is try to connect and see what happens.

I guess it is no different than specifying a bad port on the same IP address for a given peer id.

In face of this, we have to ensure that multiaddrs won't be merged on the client side.

SgtPooki · 2022-04-30T15:25:55Z

I introduced a /certhash multiaddr component, which can be repeated to transmit multiple certificate hashes.

Can we convert the '/certhash' pairs into an IPLD structure and do something like '/certhashs/<ipld_cid>' or something similar?

MarcoPolo · 2022-05-04T20:30:43Z

I introduced a /certhash multiaddr component, which can be repeated to transmit multiple certificate hashes.

Can we convert the '/certhash' pairs into an IPLD structure and do something like '/certhashs/<ipld_cid>' or something similar?

I don't think we'd want to do that since then we'd have to fetch the certhash someway (since we only have the cid). Also I'm not sure what we would gain from the extra overhead of having a struct here. We also don't want to introduce an IPLD dependency for every implementation that wants to add this.

Is there something I'm missing?

SgtPooki · 2022-05-09T18:17:25Z

Is there something I'm missing?

The CID shouldn't require lookup if it's an identity CID, other than that, probably not? ¯\_(ツ)_/¯

webtransport/README.md

marten-seemann · 2022-09-28T20:20:01Z

Thank you everyone for the great discussions on this PR!

We've shipped (experimental) WebTransport support based on this specification in go-libp2p v0.23.

I've resolved all remaining nits in the specification. Would be grateful for another round of review and many approvals :)

MarcoPolo · 2022-09-29T21:49:46Z

webtransport/README.md

+
+## Certificates
+
+Since most libp2p nodes don't possess a TLS certificate signed by a Certificate Authority, servers use a self-signed certificates. According to the [w3c WebTransport certification](https://www.w3.org/TR/webtransport/), the validity of the certificate MUST be at most 14 days, and must not use an RSA key. Nodes then include the hash of one (or more) certificates in their multiaddr (see [Addressing](#addressing)).


MUST, it's part of the webtransport spec: https://www.w3.org/TR/webtransport/#web-transport-configuration

It's "MUST NOT", not "MUST not" ;)

This entire section needs a rewrite though. It is possible to use a CA-signed certificate.

I rewrote this section. Please take another look.

webtransport/README.md

elenaf9 · 2022-09-29T21:12:32Z

webtransport/README.md

+WebTransport is a way for browsers to establish a stream-multiplexed and bidirectional connection to servers using QUIC.
+
+The WebTransport protocol is currently under development at the IETF. Chrome has implemented and shipped support for [draft-02](https://datatracker.ietf.org/doc/draft-ietf-webtrans-http3/), and Firefox [is working](https://bugzilla.mozilla.org/show_bug.cgi?id=1709355) on WebTransport support.


If I understand it correctly, when we talk about the WebTransport protocol in this doc, we actually mean the WebTransport over HTTP-3 protocol as described in draft-ietf-webtrans-http3.
WebTransport as specified in draft-ietf-webtrans-overview is a protocol framework and not a single protocol. I think we should add a note/ sentence on that. At least for me it was a bit confusing.

Yeah. Not the best naming from the IETF's side here...
Everybody thinks of WebTransport over HTTP/3 when WebTransport, but then there's the HTTP/2 fallback that not a lot of people care about to begin with...

I've added some text (and links) to clarify things. Let me know if that's less confusing now.

webtransport/README.md

elenaf9 · 2022-09-29T21:48:06Z

webtransport/README.md

+* `/ip4/1.2.3.4/udp/443/quic/webtransport/certhash/<hash1>`
+* `/ip6/fe80::1ff:fe23:4567:890a/udp/1234/quic/webtransport/certhash/<hash1>/certhash/<hash2>/certhash/<hash3>`


Why do we call it /certhash, instead of just adding a more general /multihash? The latter could then also be used in the future by other protocols that need to include a multihash in the adress for whatever reason.

There's some discussion here: multiformats/multiaddr#130. But I think the important part is that we can be a bit more specific here to highlight that this is a certificate hash rather than a hash for some other (?) thing. We also don't have a use case for a generic hash, but do have this use case for the certificate hash.

Vote for more specific field name too. it consumes less encoding space and allows easier extension.

I think that being specific pays off here, by making us more future-proof.
Imagine we want to add another hash-like component to a multiaddr in the future. We'd now use /hash for the certificate hash and for that new thing. How would we know which of these is a certificate hash and which of them the new thing?

MarcoPolo · 2022-09-29T21:49:46Z

webtransport/README.md

+
+## Certificates
+
+Since most libp2p nodes don't possess a TLS certificate signed by a Certificate Authority, servers use a self-signed certificates. According to the [w3c WebTransport certification](https://www.w3.org/TR/webtransport/), the validity of the certificate MUST be at most 14 days, and must not use an RSA key. Nodes then include the hash of one (or more) certificates in their multiaddr (see [Addressing](#addressing)).


MUST, it's part of the webtransport spec: https://www.w3.org/TR/webtransport/#web-transport-configuration

julian88110

Good write-up. Some comments and questions in line.

julian88110 · 2022-09-29T22:33:41Z

webtransport/README.md

+
+In order to verify end-to-end encryption of the connection, the peers need to establish that no MITM intercepted the connection. To do so, the server MUST include the certificate hash of the currently used certificate as well as the certificate hashes of all future certificates it has already advertised to the network in the `webtransport_certhashes` Noise extension (see Noise Extension section of the [Noise spec](/noise/README.md)). The hash of recently used, but expired certificates SHOULD also be included.
+
+On receipt of the `webtransport_certhashes` extension, the client MUST verify that the certificate hash of the certificate that was used on the connection is contained in the server's list. If the client was willing to accept multiple certificate hashes, but cannot determine which certificate was actually used to establish the connection (this will commonly be the case for browser clients), it MUST verify that all certificate hashes are contained in the server's list. If verification fails, it MUST abort the handshake.


Is it necessary to specify how the peer should detect and retire expired certificate? or is it true that the webtransport connections are shortlived so that the expiration is usually not an issue?

julian88110 · 2022-09-29T22:35:45Z

webtransport/README.md

+* `/ip4/1.2.3.4/udp/443/quic/webtransport/certhash/<hash1>`
+* `/ip6/fe80::1ff:fe23:4567:890a/udp/1234/quic/webtransport/certhash/<hash1>/certhash/<hash2>/certhash/<hash3>`


Vote for more specific field name too. it consumes less encoding space and allows easier extension.

julian88110 · 2022-09-29T22:38:09Z

webtransport/README.md

+## Security Handshake
+
+Unfortunately, the self-signed certificate doesn't allow the nodes to authenticate each others' peer IDs. It is therefore necessary to run an additional libp2p handshake on a newly established WebTransport connection.
+The first stream that the client opens on a new WebTransport session is used to perform a libp2p handshake using Noise (https://github.com/libp2p/specs/tree/master/noise). The client SHOULD start the handshake right after sending the CONNECT request, without waiting for the server's response.


This is just a curious question regarding handshake starting right after Connect. Is connect referring to establishing a socket level connection?

This is WebTransport CONNECT request as described in https://datatracker.ietf.org/doc/html/draft-ietf-webtrans-http3-03#section-3.2.

Co-authored-by: Elena Frank <elena.frank@protonmail.com>

marten-seemann · 2022-09-30T09:07:03Z

Thank you for this in-depth review, @elenaf9!

I rewrote the "Certificates" section to allow the use of CA-signed certificates.

webtransport/README.md

marten-seemann · 2022-10-12T19:19:16Z

Thank you to everyone involved here! This was a big effort, and we now have a decent spec, implemented and released by go-libp2p, with another implementation in js-libp2p demonstrating interoperability.

Given that this PR has received 2 approvals from members of the libp2p team at PL, I'm going to merge it now.
This does NOT mean that this spec is perfect - both editorial as well as protocol changes are still possible via PRs to this repo.

mxinden · 2022-10-12T19:36:17Z

🚀 big step for the project!

Small nit, can you add an entry to https://github.com/libp2p/specs#protocols?

marten-seemann requested review from vyzo, BigLep, lidel, Stebalien, willscott, aschmahmann and mxinden April 4, 2022 10:24

willscott reviewed Apr 5, 2022

View reviewed changes

mriise reviewed Apr 6, 2022

View reviewed changes

webtransport/README.md Outdated Show resolved Hide resolved

webtransport/README.md Outdated Show resolved Hide resolved

webtransport/README.md Outdated Show resolved Hide resolved

melekes reviewed Apr 7, 2022

View reviewed changes

webtransport/README.md Show resolved Hide resolved

Stebalien reviewed Apr 10, 2022

View reviewed changes

marten-seemann requested a review from MarcoPolo April 11, 2022 18:33

MarcoPolo reviewed Apr 12, 2022

View reviewed changes

webtransport/README.md Outdated Show resolved Hide resolved

This was referenced Apr 16, 2022

implement an early data API libp2p/go-libp2p-noise#110

Open

add the WebTransport effort to the roadmap #407

Merged

marten-seemann requested review from Stebalien and Kubuxu April 17, 2022 19:39

Kubuxu reviewed Apr 22, 2022

View reviewed changes

BigLep assigned marten-seemann Apr 27, 2022

marten-seemann force-pushed the webtransport branch from 6aec66d to 27d4bb6 Compare May 23, 2022 14:25

mxinden reviewed May 24, 2022

View reviewed changes

webtransport/README.md Outdated Show resolved Hide resolved

MarcoPolo reviewed May 25, 2022

View reviewed changes

webtransport/README.md Outdated Show resolved Hide resolved

marten-seemann mentioned this pull request May 26, 2022

webrtc/: Add libp2p WebRTC browser-to-server spec #412

Merged

19 tasks

walkerlj0 mentioned this pull request Jun 1, 2022

Add updated libp2p video protocol/launchpad#135

Closed

marten-seemann mentioned this pull request Jun 13, 2022

add WebTransport multiaddr components multiformats/go-multiaddr#176

Merged

marten-seemann requested review from julian88110 and lidel September 28, 2022 20:18

p-shahi mentioned this pull request Sep 29, 2022

roadmap: move WebTransport to Done section #460

Merged

MarcoPolo approved these changes Sep 29, 2022

View reviewed changes

elenaf9 reviewed Sep 29, 2022

View reviewed changes

MarcoPolo reviewed Sep 29, 2022

View reviewed changes

julian88110 reviewed Sep 29, 2022

View reviewed changes

marten-seemann added 2 commits September 30, 2022 11:38

webtransport: soften language around URL multiaddr encoding

5ab529d

webtransport: clarify that WebTransport over HTTP/3 is meant

1219d07

libp2p deleted a comment from elenaf9 Sep 30, 2022

marten-seemann and others added 4 commits September 30, 2022 01:49

webtransport: fix typo

4f498a1

Co-authored-by: Elena Frank <elena.frank@protonmail.com>

webtransport: clarify certificate regeneration logic

53d98bd

webtransport: fix typos

5271135

Co-authored-by: Elena Frank <elena.frank@protonmail.com>

webtransport: allow use of CA-signed certificates

bb90d58

marten-seemann requested review from elenaf9 and julian88110 September 30, 2022 09:07

elenaf9 approved these changes Oct 5, 2022

View reviewed changes

webtransport/README.md Outdated Show resolved Hide resolved

webtransport/README.md Outdated Show resolved Hide resolved

webtransport/README.md Outdated Show resolved Hide resolved

webtransport/README.md Outdated Show resolved Hide resolved

BigLep mentioned this pull request Oct 5, 2022

Enable WebTransport by default ipfs/kubo#9292

Closed

mxinden mentioned this pull request Oct 6, 2022

transport/webtransport: Add WebTransport protocol libp2p/rust-libp2p#2993

Open

marten-seemann mentioned this pull request Oct 12, 2022

Tracking issue: Finish WebTransport Support libp2p/go-libp2p#1827

Closed

10 tasks

marten-seemann added 2 commits October 12, 2022 21:06

address minor issues raised in code review

f9f8613

clarify that servers with a CA-signed certificate don't use /certhash

0a9a7bb

marten-seemann merged commit 12f9a31 into master Oct 12, 2022

marten-seemann deleted the webtransport branch October 12, 2022 19:21

lidel mentioned this pull request Oct 14, 2022

feat: update ./multicodec/table.csv multiformats/go-multicodec#71

Merged


		## Securing Streams

		All streams other than the stream used for the security handshake are protected using Salsa20. Two (symmetric) keys are derived from the master secrect established during the handshake, one for sending and for receiving on the stream.


		## Certificates

		Since most libp2p nodes don't possess a TLS certificate signed by a Certificate Authority, servers use a self-signed certificates. According to the [w3c WebTransport certification](https://www.w3.org/TR/webtransport/), the validity of the certificate MUST be at most 14 days, and must not use an RSA key. Nodes then include the hash of one (or more) certificates in their multiaddr (see [Addressing](#addressing)).

		WebTransport is a way for browsers to establish a stream-multiplexed and bidirectional connection to servers using QUIC.

		The WebTransport protocol is currently under development at the IETF. Chrome has implemented and shipped support for [draft-02](https://datatracker.ietf.org/doc/draft-ietf-webtrans-http3/), and Firefox [is working](https://bugzilla.mozilla.org/show_bug.cgi?id=1709355) on WebTransport support.

		* `/ip4/1.2.3.4/udp/443/quic/webtransport/certhash/<hash1>`
		* `/ip6/fe80::1ff:fe23:4567:890a/udp/1234/quic/webtransport/certhash/<hash1>/certhash/<hash2>/certhash/<hash3>`


		In order to verify end-to-end encryption of the connection, the peers need to establish that no MITM intercepted the connection. To do so, the server MUST include the certificate hash of the currently used certificate as well as the certificate hashes of all future certificates it has already advertised to the network in the `webtransport_certhashes` Noise extension (see Noise Extension section of the [Noise spec](/noise/README.md)). The hash of recently used, but expired certificates SHOULD also be included.

		On receipt of the `webtransport_certhashes` extension, the client MUST verify that the certificate hash of the certificate that was used on the connection is contained in the server's list. If the client was willing to accept multiple certificate hashes, but cannot determine which certificate was actually used to establish the connection (this will commonly be the case for browser clients), it MUST verify that all certificate hashes are contained in the server's list. If verification fails, it MUST abort the handshake.

add a WebTransport spec #404

add a WebTransport spec #404

Conversation

marten-seemann commented Apr 4, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kubuxu Apr 13, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kubuxu Apr 13, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marten-seemann commented Apr 17, 2022

Kubuxu commented Apr 22, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SgtPooki commented Apr 30, 2022

MarcoPolo commented May 4, 2022

SgtPooki commented May 9, 2022 • edited Loading

marten-seemann commented Sep 28, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elenaf9 Sep 29, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

julian88110 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marten-seemann commented Sep 30, 2022

marten-seemann commented Oct 12, 2022

mxinden commented Oct 12, 2022

marten-seemann commented Apr 4, 2022 •

edited

Loading

Kubuxu Apr 13, 2022 •

edited

Loading

Kubuxu Apr 13, 2022 •

edited

Loading

SgtPooki commented May 9, 2022 •

edited

Loading

elenaf9 Sep 29, 2022 •

edited

Loading