diff --git a/.DS_Store b/.DS_Store new file mode 100644 index 00000000..77fc116c Binary files /dev/null and b/.DS_Store differ diff --git a/config.toml b/config.toml index 7144de4d..add10022 100644 --- a/config.toml +++ b/config.toml @@ -2,8 +2,9 @@ baseURL = "https://docs.libp2p.io" relativeURLs = true languageCode = "en-us" title = "libp2p Documentation" +summaryLength = 20 -theme = "hugo-theme-learn" +theme = ["hugo-theme-learn"] [params] themeVariant = "libp2p" @@ -18,12 +19,19 @@ url = "https://libp2p.io" weight = 1 [[menu.shortcuts]] -name = " libp2p Github" +name = " libp2p GitHub" url = "https://github.com/libp2p/libp2p" weight = 2 +[[menu.shortcuts]] +name = " libp2p Specs" +url = "https://github.com/libp2p/specs" +weight = 3 + [[menu.shortcuts]] name = " Discussion Forums" url = "https://discuss.libp2p.io" -weight = 3 +weight = 4 +[markup.goldmark.renderer] + unsafe = true # Allow HTML in md files diff --git a/content/.DS_Store b/content/.DS_Store new file mode 100644 index 00000000..abdf6fc3 Binary files /dev/null and b/content/.DS_Store differ diff --git a/content/_index.md b/content/_index.md index f30ed201..129c8774 100644 --- a/content/_index.md +++ b/content/_index.md @@ -1,29 +1,16 @@ --- -title: libp2p documentation portal +title: libp2p Documentation Portal menuTitle: libp2p weight: 1 -pre: "1. " +pre: " " chapter: true --- -# libp2p Documentation Portal +# Welcome to the Documentation Portal -Welcome to the libp2p documentation portal! Whether you’re just learning how to build peer-to-peer systems with libp2p, want to dive into peer-to-peer concepts and solutions, or are looking for detailed reference +Welcome to the libp2p Documentation Portal! Whether you’re just learning how to build peer-to-peer systems with +libp2p, want to dive into peer-to-peer concepts and solutions, or are looking for detailed reference information, this is the place to start. {{% children description="true" %}} -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} - -### Related Projects - -libp2p began as part of the [IPFS](https://ipfs.io) project, and is still an essential component of IPFS. As such, libp2p composes well with the abstractions and tools provided by other projects in the IPFS "family". Check their individual sites for specific information and references: - -- [IPFS](https://ipfs.io) is the InterPlanetary File System, which uses libp2p as its networking layer. -- [Multiformats](https://multiformats.io) is a variety of *self-describing* data formats. -- [IPLD](https://ipld.io) is a set of tools for describing links between content-addressed data, like IPFS files, Git commits, or Ethereum blocks. -- [The Permissive License Stack](https://protocol.ai/blog/announcing-the-permissive-license-stack) is a licensing strategy for software development that embraces open-source values. diff --git a/content/community/_index.md b/content/community/_index.md index 33ec44d6..f7ab5b1a 100644 --- a/content/community/_index.md +++ b/content/community/_index.md @@ -1,11 +1,13 @@ --- title: "Community" -weight: 10 -pre: ' ' +weight: 5 +pre: ' ' +chapter: true +summary: Get in touch with other members working on or with libp2p, learn how to contribute, and become an active participant in the libp2p community. --- -Get in touch with other members of the libp2p community who are building tools and applications with libp2p! You can ask questions, discuss new ideas, or get support for problems at [libp2p discussion forums](https://discuss.libp2p.io/), but you can also [hop on IRC](/community/irc/) for a quick chat. +# Get Involved with libp2p -See the other links in the community section for more information about meetings, events, apps people are building, and more. +Find out how to get involved with libp2p and join the community. -Information about contributing to libp2p and about other software projects in the community are also hosted here. +{{% children description="true"%}} diff --git a/content/discuss/_index.md b/content/community/community.md similarity index 80% rename from content/discuss/_index.md rename to content/community/community.md index 6cd8c534..3e29577f 100644 --- a/content/discuss/_index.md +++ b/content/community/community.md @@ -1,9 +1,13 @@ --- -title: "Help & Discussion" -weight: 100 -pre: " " +title: "Get Involved" +weight: 1 +pre: ' ' +chapter: true +summary: Get involved and participate in the libp2p community. --- +# Get Involved + We love talking about libp2p, and we'd be happy to have you in the mix. Visit our [discussion forums](https://discuss.libp2p.io) to ask questions about [using libp2p](https://discuss.libp2p.io/c/users), discuss exciting [research](https://discuss.libp2p.io/c/research), and [stay up to date with important announcments](https://discuss.libp2p.io/c/news). diff --git a/content/contributing/_index.md b/content/community/contribute.md similarity index 89% rename from content/contributing/_index.md rename to content/community/contribute.md index 80be193d..ed828a86 100644 --- a/content/contributing/_index.md +++ b/content/community/contribute.md @@ -1,26 +1,31 @@ --- -title: "Get Involved" -weight: 10 -pre: ' ' +title: "How to Contribute" +weight: 2 +pre: ' ' +chapter: true +summary: Learn how you can contribute to libp2p. --- +# How to Contribute + So you want to contribute to libp2p and the peer-to-peer ecosystem? Here is a quick listing of things we need help with and how you can get started. Even if what you want to do is not listed here, we probably accept contributions for it! If you're unsure, please open a issue. ## Areas of contribution -- [Areas of contribution](#areas-of-contribution) - - [Code](#code) - - [Documentation](#documentation) - - [Support](#support) - - [Testing](#testing) - - [Design](#design) - - [Issues / Triaging](#issues--triaging) - - [Community](#community) - - [Applications](#applications) - - [Protocol Design](#protocol-design) - - [Research](#research) +- [How to Contribute](#how-to-contribute) + - [Areas of contribution](#areas-of-contribution) + - [Code](#code) + - [Documentation](#documentation) + - [Support](#support) + - [Testing](#testing) + - [Design](#design) + - [Issues / Triaging](#issues--triaging) + - [Community](#community) + - [Applications](#applications) + - [Protocol Design](#protocol-design) + - [Research](#research) ### Code diff --git a/content/concepts/_index.md b/content/concepts/_index.md index de391551..99a216a5 100644 --- a/content/concepts/_index.md +++ b/content/concepts/_index.md @@ -1,16 +1,15 @@ --- -title: "Core Components" +title: Core Components weight: 2 -pre: ' ' +pre: ' ' +chapter: true +aliases: /reference/specs/ +summary: libp2p covers a lot of ground, and may involve unfamiliar terminology and concepts. This section goes over the foundational concepts involved in libp2p. --- -libp2p covers a lot of ground, and may involve unfamiliar terminology and concepts. -This section goes over the foundational concepts involved in libp2p. +# Learn about libp2p -{{% children %}} +libp2p covers a lot of ground, and may involve unfamiliar terminology and concepts. +This chapter goes over the foundational concepts involved in libp2p. -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} +{{% children description="true"%}} diff --git a/content/concepts/publish-subscribe/fanout_forget.png b/content/concepts/assets/publish-subscribe/fanout_forget.png similarity index 100% rename from content/concepts/publish-subscribe/fanout_forget.png rename to content/concepts/assets/publish-subscribe/fanout_forget.png diff --git a/content/concepts/publish-subscribe/fanout_forget.svg b/content/concepts/assets/publish-subscribe/fanout_forget.svg similarity index 100% rename from content/concepts/publish-subscribe/fanout_forget.svg rename to content/concepts/assets/publish-subscribe/fanout_forget.svg diff --git a/content/concepts/publish-subscribe/fanout_grafting_preference.png b/content/concepts/assets/publish-subscribe/fanout_grafting_preference.png similarity index 100% rename from content/concepts/publish-subscribe/fanout_grafting_preference.png rename to content/concepts/assets/publish-subscribe/fanout_grafting_preference.png diff --git a/content/concepts/publish-subscribe/fanout_grafting_preference.svg b/content/concepts/assets/publish-subscribe/fanout_grafting_preference.svg similarity index 100% rename from content/concepts/publish-subscribe/fanout_grafting_preference.svg rename to content/concepts/assets/publish-subscribe/fanout_grafting_preference.svg diff --git a/content/concepts/publish-subscribe/fanout_initial_pick.png b/content/concepts/assets/publish-subscribe/fanout_initial_pick.png similarity index 100% rename from content/concepts/publish-subscribe/fanout_initial_pick.png rename to content/concepts/assets/publish-subscribe/fanout_initial_pick.png diff --git a/content/concepts/publish-subscribe/fanout_initial_pick.svg b/content/concepts/assets/publish-subscribe/fanout_initial_pick.svg similarity index 100% rename from content/concepts/publish-subscribe/fanout_initial_pick.svg rename to content/concepts/assets/publish-subscribe/fanout_initial_pick.svg diff --git a/content/concepts/publish-subscribe/fanout_message_send.png b/content/concepts/assets/publish-subscribe/fanout_message_send.png similarity index 100% rename from content/concepts/publish-subscribe/fanout_message_send.png rename to content/concepts/assets/publish-subscribe/fanout_message_send.png diff --git a/content/concepts/publish-subscribe/fanout_message_send.svg b/content/concepts/assets/publish-subscribe/fanout_message_send.svg similarity index 100% rename from content/concepts/publish-subscribe/fanout_message_send.svg rename to content/concepts/assets/publish-subscribe/fanout_message_send.svg diff --git a/content/concepts/publish-subscribe/full_message_forward.png b/content/concepts/assets/publish-subscribe/full_message_forward.png similarity index 100% rename from content/concepts/publish-subscribe/full_message_forward.png rename to content/concepts/assets/publish-subscribe/full_message_forward.png diff --git a/content/concepts/publish-subscribe/full_message_forward.svg b/content/concepts/assets/publish-subscribe/full_message_forward.svg similarity index 100% rename from content/concepts/publish-subscribe/full_message_forward.svg rename to content/concepts/assets/publish-subscribe/full_message_forward.svg diff --git a/content/concepts/publish-subscribe/full_message_network.png b/content/concepts/assets/publish-subscribe/full_message_network.png similarity index 100% rename from content/concepts/publish-subscribe/full_message_network.png rename to content/concepts/assets/publish-subscribe/full_message_network.png diff --git a/content/concepts/publish-subscribe/full_message_network.svg b/content/concepts/assets/publish-subscribe/full_message_network.svg similarity index 100% rename from content/concepts/publish-subscribe/full_message_network.svg rename to content/concepts/assets/publish-subscribe/full_message_network.svg diff --git a/content/concepts/publish-subscribe/full_message_send.png b/content/concepts/assets/publish-subscribe/full_message_send.png similarity index 100% rename from content/concepts/publish-subscribe/full_message_send.png rename to content/concepts/assets/publish-subscribe/full_message_send.png diff --git a/content/concepts/publish-subscribe/full_message_send.svg b/content/concepts/assets/publish-subscribe/full_message_send.svg similarity index 100% rename from content/concepts/publish-subscribe/full_message_send.svg rename to content/concepts/assets/publish-subscribe/full_message_send.svg diff --git a/content/concepts/publish-subscribe/gossip_deliver.png b/content/concepts/assets/publish-subscribe/gossip_deliver.png similarity index 100% rename from content/concepts/publish-subscribe/gossip_deliver.png rename to content/concepts/assets/publish-subscribe/gossip_deliver.png diff --git a/content/concepts/publish-subscribe/gossip_deliver.svg b/content/concepts/assets/publish-subscribe/gossip_deliver.svg similarity index 100% rename from content/concepts/publish-subscribe/gossip_deliver.svg rename to content/concepts/assets/publish-subscribe/gossip_deliver.svg diff --git a/content/concepts/publish-subscribe/graft_prune.png b/content/concepts/assets/publish-subscribe/graft_prune.png similarity index 100% rename from content/concepts/publish-subscribe/graft_prune.png rename to content/concepts/assets/publish-subscribe/graft_prune.png diff --git a/content/concepts/publish-subscribe/graft_prune.svg b/content/concepts/assets/publish-subscribe/graft_prune.svg similarity index 100% rename from content/concepts/publish-subscribe/graft_prune.svg rename to content/concepts/assets/publish-subscribe/graft_prune.svg diff --git a/content/concepts/publish-subscribe/maintain_graft.png b/content/concepts/assets/publish-subscribe/maintain_graft.png similarity index 100% rename from content/concepts/publish-subscribe/maintain_graft.png rename to content/concepts/assets/publish-subscribe/maintain_graft.png diff --git a/content/concepts/publish-subscribe/maintain_graft.svg b/content/concepts/assets/publish-subscribe/maintain_graft.svg similarity index 100% rename from content/concepts/publish-subscribe/maintain_graft.svg rename to content/concepts/assets/publish-subscribe/maintain_graft.svg diff --git a/content/concepts/publish-subscribe/maintain_prune.png b/content/concepts/assets/publish-subscribe/maintain_prune.png similarity index 100% rename from content/concepts/publish-subscribe/maintain_prune.png rename to content/concepts/assets/publish-subscribe/maintain_prune.png diff --git a/content/concepts/publish-subscribe/maintain_prune.svg b/content/concepts/assets/publish-subscribe/maintain_prune.svg similarity index 100% rename from content/concepts/publish-subscribe/maintain_prune.svg rename to content/concepts/assets/publish-subscribe/maintain_prune.svg diff --git a/content/concepts/publish-subscribe/message_delivered_to_all.png b/content/concepts/assets/publish-subscribe/message_delivered_to_all.png similarity index 100% rename from content/concepts/publish-subscribe/message_delivered_to_all.png rename to content/concepts/assets/publish-subscribe/message_delivered_to_all.png diff --git a/content/concepts/publish-subscribe/message_delivered_to_all.svg b/content/concepts/assets/publish-subscribe/message_delivered_to_all.svg similarity index 100% rename from content/concepts/publish-subscribe/message_delivered_to_all.svg rename to content/concepts/assets/publish-subscribe/message_delivered_to_all.svg diff --git a/content/concepts/publish-subscribe/metadata_only_network.png b/content/concepts/assets/publish-subscribe/metadata_only_network.png similarity index 100% rename from content/concepts/publish-subscribe/metadata_only_network.png rename to content/concepts/assets/publish-subscribe/metadata_only_network.png diff --git a/content/concepts/publish-subscribe/metadata_only_network.svg b/content/concepts/assets/publish-subscribe/metadata_only_network.svg similarity index 100% rename from content/concepts/publish-subscribe/metadata_only_network.svg rename to content/concepts/assets/publish-subscribe/metadata_only_network.svg diff --git a/content/concepts/publish-subscribe/network_packet_structure.png b/content/concepts/assets/publish-subscribe/network_packet_structure.png similarity index 100% rename from content/concepts/publish-subscribe/network_packet_structure.png rename to content/concepts/assets/publish-subscribe/network_packet_structure.png diff --git a/content/concepts/publish-subscribe/network_packet_structure.svg b/content/concepts/assets/publish-subscribe/network_packet_structure.svg similarity index 100% rename from content/concepts/publish-subscribe/network_packet_structure.svg rename to content/concepts/assets/publish-subscribe/network_packet_structure.svg diff --git a/content/concepts/publish-subscribe/request_gossiped_message.png b/content/concepts/assets/publish-subscribe/request_gossiped_message.png similarity index 100% rename from content/concepts/publish-subscribe/request_gossiped_message.png rename to content/concepts/assets/publish-subscribe/request_gossiped_message.png diff --git a/content/concepts/publish-subscribe/request_gossiped_message.svg b/content/concepts/assets/publish-subscribe/request_gossiped_message.svg similarity index 100% rename from content/concepts/publish-subscribe/request_gossiped_message.svg rename to content/concepts/assets/publish-subscribe/request_gossiped_message.svg diff --git a/content/concepts/publish-subscribe/state.png b/content/concepts/assets/publish-subscribe/state.png similarity index 100% rename from content/concepts/publish-subscribe/state.png rename to content/concepts/assets/publish-subscribe/state.png diff --git a/content/concepts/publish-subscribe/state.svg b/content/concepts/assets/publish-subscribe/state.svg similarity index 100% rename from content/concepts/publish-subscribe/state.svg rename to content/concepts/assets/publish-subscribe/state.svg diff --git a/content/concepts/publish-subscribe/subscribe_graft.png b/content/concepts/assets/publish-subscribe/subscribe_graft.png similarity index 100% rename from content/concepts/publish-subscribe/subscribe_graft.png rename to content/concepts/assets/publish-subscribe/subscribe_graft.png diff --git a/content/concepts/publish-subscribe/subscribe_graft.svg b/content/concepts/assets/publish-subscribe/subscribe_graft.svg similarity index 100% rename from content/concepts/publish-subscribe/subscribe_graft.svg rename to content/concepts/assets/publish-subscribe/subscribe_graft.svg diff --git a/content/concepts/publish-subscribe/subscribed_peers.png b/content/concepts/assets/publish-subscribe/subscribed_peers.png similarity index 100% rename from content/concepts/publish-subscribe/subscribed_peers.png rename to content/concepts/assets/publish-subscribe/subscribed_peers.png diff --git a/content/concepts/publish-subscribe/subscribed_peers.svg b/content/concepts/assets/publish-subscribe/subscribed_peers.svg similarity index 100% rename from content/concepts/publish-subscribe/subscribed_peers.svg rename to content/concepts/assets/publish-subscribe/subscribed_peers.svg diff --git a/content/concepts/publish-subscribe/subscription_list_change.png b/content/concepts/assets/publish-subscribe/subscription_list_change.png similarity index 100% rename from content/concepts/publish-subscribe/subscription_list_change.png rename to content/concepts/assets/publish-subscribe/subscription_list_change.png diff --git a/content/concepts/publish-subscribe/subscription_list_change.svg b/content/concepts/assets/publish-subscribe/subscription_list_change.svg similarity index 100% rename from content/concepts/publish-subscribe/subscription_list_change.svg rename to content/concepts/assets/publish-subscribe/subscription_list_change.svg diff --git a/content/concepts/publish-subscribe/subscription_list_first_connect.png b/content/concepts/assets/publish-subscribe/subscription_list_first_connect.png similarity index 100% rename from content/concepts/publish-subscribe/subscription_list_first_connect.png rename to content/concepts/assets/publish-subscribe/subscription_list_first_connect.png diff --git a/content/concepts/publish-subscribe/subscription_list_first_connect.svg b/content/concepts/assets/publish-subscribe/subscription_list_first_connect.svg similarity index 100% rename from content/concepts/publish-subscribe/subscription_list_first_connect.svg rename to content/concepts/assets/publish-subscribe/subscription_list_first_connect.svg diff --git a/content/concepts/publish-subscribe/subscriptions_local_view.png b/content/concepts/assets/publish-subscribe/subscriptions_local_view.png similarity index 100% rename from content/concepts/publish-subscribe/subscriptions_local_view.png rename to content/concepts/assets/publish-subscribe/subscriptions_local_view.png diff --git a/content/concepts/publish-subscribe/subscriptions_local_view.svg b/content/concepts/assets/publish-subscribe/subscriptions_local_view.svg similarity index 100% rename from content/concepts/publish-subscribe/subscriptions_local_view.svg rename to content/concepts/assets/publish-subscribe/subscriptions_local_view.svg diff --git a/content/concepts/publish-subscribe/types_of_peering.png b/content/concepts/assets/publish-subscribe/types_of_peering.png similarity index 100% rename from content/concepts/publish-subscribe/types_of_peering.png rename to content/concepts/assets/publish-subscribe/types_of_peering.png diff --git a/content/concepts/publish-subscribe/types_of_peering.svg b/content/concepts/assets/publish-subscribe/types_of_peering.svg similarity index 100% rename from content/concepts/publish-subscribe/types_of_peering.svg rename to content/concepts/assets/publish-subscribe/types_of_peering.svg diff --git a/content/concepts/publish-subscribe/unsubscribe_prune.png b/content/concepts/assets/publish-subscribe/unsubscribe_prune.png similarity index 100% rename from content/concepts/publish-subscribe/unsubscribe_prune.png rename to content/concepts/assets/publish-subscribe/unsubscribe_prune.png diff --git a/content/concepts/publish-subscribe/unsubscribe_prune.svg b/content/concepts/assets/publish-subscribe/unsubscribe_prune.svg similarity index 100% rename from content/concepts/publish-subscribe/unsubscribe_prune.svg rename to content/concepts/assets/publish-subscribe/unsubscribe_prune.svg diff --git a/content/concepts/content-routing.md b/content/concepts/content-routing.md deleted file mode 100644 index 8d212cbc..00000000 --- a/content/concepts/content-routing.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: Content Routing -weight: 5 ---- - -This article is coming soon! - -Please [refer to this issue](https://github.com/libp2p/docs/issues/23) to track the progress and make suggestions. diff --git a/content/reference/dos-mitigation.md b/content/concepts/dos-mitigation.md similarity index 88% rename from content/reference/dos-mitigation.md rename to content/concepts/dos-mitigation.md index a9f2a13b..521e9fac 100644 --- a/content/reference/dos-mitigation.md +++ b/content/concepts/dos-mitigation.md @@ -1,37 +1,45 @@ --- -title: "DOS Mitigation" -weight: 3 +title: "DoS Mitigation" +weight: 11 +pre: ' ' +chapter: true +aliases: /reference/dos-mitigation/ +summary: DoS mitigation is an essential part of any peer-to-peer application. Learn how to design protocols to be resilient to malicious peers. --- -DOS mitigation is an essential part of any P2P application. We need to design +# DoS Mitigation + +DoS mitigation is an essential part of any P2P application. We need to design our protocols to be resilient to malicious peers. We need to monitor our application for signs of suspicious activity or an attack. And we need to be able to respond to an attack. Here we'll cover how we can use libp2p to achieve the above goals. -# Table of contents - -- [What we mean by a DOS attack](#what-we-mean-by-a-dos-attack) -- [Incorporating DOS mitigation from the start](#incorporating-dos-mitigation-from-the-start) - - [Limit the number of connections your application needs](#limit-the-number-of-connections-your-application-needs) - - [Transient Connections](#transient-connections) - - [Limit the number of concurrent streams per connection your protocol needs](#limit-the-number-of-concurrent-streams-per-connection-your-protocol-needs) - - [Reduce blast radius](#reduce-blast-radius) - - [Fail2ban](#fail2ban) - - [Leverage the resource manager to limit resource usage (go-libp2p only)](#leverage-the-resource-manager-to-limit-resource-usage-go-libp2p-only) - - [Rate limiting incoming connections](#rate-limiting-incoming-connections) -- [Monitoring your application](#monitoring-your-application) -- [Responding to an attack](#responding-to-an-attack) - - [Who’s misbehaving?](#whos-misbehaving) - - [How to block a misbehaving peer](#how-to-block-a-misbehaving-peer) - - [How to automate blocking with fail2ban](#how-to-automate-blocking-with-fail2ban) - - [Example screen recording of fail2ban in action](#example-screen-recording-of-fail2ban-in-action) - - [Setting Up fail2ban](#setting-up-fail2ban) - - [Deny specific peers or create an allow list of trusted peers](#deny-specific-peers-or-create-an-allow-list-of-trusted-peers) -- [Summary](#summary) - -# What we mean by a DOS attack +## Table of contents + +- [DoS Mitigation](#dos-mitigation) + - [Table of contents](#table-of-contents) + - [What we mean by a DOS attack](#what-we-mean-by-a-dos-attack) + - [Incorporating DOS mitigation from the start](#incorporating-dos-mitigation-from-the-start) + - [Limit the number of connections your application needs](#limit-the-number-of-connections-your-application-needs) + - [Transient Connections](#transient-connections) + - [Limit the number of concurrent streams per connection your protocol needs](#limit-the-number-of-concurrent-streams-per-connection-your-protocol-needs) + - [Reduce blast radius](#reduce-blast-radius) + - [Fail2ban](#fail2ban) + - [Leverage the resource manager to limit resource usage (go-libp2p only)](#leverage-the-resource-manager-to-limit-resource-usage-go-libp2p-only) + - [Rate limiting incoming connections](#rate-limiting-incoming-connections) + - [Monitoring your application](#monitoring-your-application) + - [Responding to an attack](#responding-to-an-attack) + - [Who’s misbehaving?](#whos-misbehaving) + - [How to block a misbehaving peer](#how-to-block-a-misbehaving-peer) + - [How to automate blocking with fail2ban](#how-to-automate-blocking-with-fail2ban) + - [Example screen recording of fail2ban in action](#example-screen-recording-of-fail2ban-in-action) + - [Setting Up fail2ban](#setting-up-fail2ban) + - [Deny specific peers or create an allow list of trusted peers](#deny-specific-peers-or-create-an-allow-list-of-trusted-peers) + - [Summary](#summary) + +## What we mean by a DOS attack A DOS attack is any attack that can cause your application to crash, stall, or otherwise fail to respond normally. An attack is considered viable if it takes @@ -63,7 +71,7 @@ stay up and healthy. In the next section we'll cover some design strategies you should incorporate into your protocol to make sure your application stays up and healthy. -# Incorporating DOS mitigation from the start +## Incorporating DOS mitigation from the start The general strategy is to use the minimum amount of resources as possible and make sure that there's no untrusted amplification mechanism (e.g. an untrusted @@ -74,7 +82,7 @@ below). Below are some more specific recommendations -## Limit the number of connections your application needs +### Limit the number of connections your application needs Each connection has a resource cost associated with it. A connection will usually represent a peer and a set of protocols with each their own resource @@ -96,7 +104,6 @@ multiple knobs here that do similar things, so take care to set these. We know this is not ideal and we are tracking this issue [here](https://github.com/libp2p/go-libp2p/issues/1640), contributions welcome. - In rust-libp2p handlers should implement [`connection_keep_alive`](https://docs.rs/libp2p/latest/libp2p/swarm/trait.ConnectionHandler.html#tymethod.connection_keep_alive) to define when a connection can be closed. The swarm will close connections when @@ -114,7 +121,7 @@ and passing it to the js-libp2p users should read the section on [connection limits](https://github.com/libp2p/js-libp2p/blob/master/doc/LIMITS.md#connection-limits) in the js-libp2p docs. -## Transient Connections +### Transient Connections When a connection is first established to libp2p but before that connection has been tied to a specific peer (before security and muxer have been negotiated), @@ -135,7 +142,7 @@ In rust-libp2p you can tune this with `ConnectionLimits` as explained above. Similarly js-libp2p users can adjust the `maxIncomingPendingConnections` value in the [connection limits](https://github.com/libp2p/js-libp2p/blob/master/doc/LIMITS.md#connection-limits) as explained in the js-libp2p docs. -## Limit the number of concurrent streams per connection your protocol needs +### Limit the number of concurrent streams per connection your protocol needs Each stream has some resource cost associated with it. Depending on the transport and multiplexer, this can be bigger or smaller. Design your protocol @@ -179,7 +186,7 @@ If we add a limit in this protocol of say 10 streams, then method 1 will mean we can only have 10 concurrent RPC calls, while method 2 would let us have a much larger number of concurrent RPC calls. -## Reduce blast radius +### Reduce blast radius If you can split up your libp2p application into multiple separate processes you can increase the resiliency of your overall system. For example, your node may @@ -187,15 +194,14 @@ have to help achieve consensus and respond to user queries. By splitting this up into two processes you now rely on the OS’s guarantee that the user query process won’t take down the consensus process. -## Fail2ban +### Fail2ban If you can log when a peer is misbehaving or is malicious, you can then hook up those logs to fail2ban and have fail2ban manage your firewall to automatically block misbehaving nodes. go-libp2p includes some built-in support for this use case. More details below. - -## Leverage the resource manager to limit resource usage (go-libp2p only) +### Leverage the resource manager to limit resource usage (go-libp2p only) go-libp2p includes a powerful [resource manager](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager) that keeps track @@ -204,7 +210,7 @@ within your protocol implementation to make sure you don't allocate more than some predetermined amount of memory per connection. It's basically a resource accounting abstraction that you can make use of in your own application. -## Rate limiting incoming connections +### Rate limiting incoming connections Depending on your use case, it can help to limit the rate of inbound connections. You can use go-libp2p's @@ -215,17 +221,15 @@ Gater](https://github.com/prysmaticlabs/prysm/blob/63a8690140c00ba6e3e4054cac3f3 js-libp2p has a similar [connection gater](https://github.com/libp2p/js-libp2p/blob/master/doc/CONFIGURATION.md#configuring-connection-gater) that can be configured on node start up and also allows you to [drop connections from peers that try to open too many connections too quickly](https://github.com/libp2p/js-libp2p/blob/master/doc/LIMITS.md#inbound-connection-threshold). -# Monitoring your application +## Monitoring your application Once we've designed our protocols to be resilient to DOS attacks and deployed them, we then need to monitor our application both to verify our mitigation works and to be alerted if a new attack vector is exploited. - Monitoring is implementation specific, so consult the links below to see how your implementation does it. - For rust-libp2p look at the [libp2p-metrics crate](https://github.com/libp2p/rust-libp2p/tree/master/misc/metrics). For go-libp2p resource usage take a look at the OpenCensus metrics exposed by the resource @@ -238,12 +242,12 @@ This work is being tracked in issue js-libp2p collects various system metrics, please see the [metrics documentation](https://github.com/libp2p/js-libp2p/blob/master/doc/METRICS.md) for more information. -# Responding to an attack +## Responding to an attack When you see that your node is being attacked (e.g. crashing, stalling, high cpu usage), then the next step is responding to the attack. -## Who’s misbehaving? +### Who’s misbehaving? To answer the question of which peer is misbehaving and harming you, go-libp2p exposes a [canonical log @@ -275,7 +279,7 @@ If the logs are too verbose you can also exclude components trace logging `DEBUG the value of the variable is consulted at runtime so you can alter the amount or type of logging while your application is running. -## How to block a misbehaving peer +### How to block a misbehaving peer Once you’ve identified the misbehaving peer, you can block them with `iptables` or `ufw`. Here we’ll outline how to block the peer with `ufw`. You can get the @@ -286,7 +290,7 @@ ip address of the peer from the sudo ufw deny from 1.2.3.4 ``` -## How to automate blocking with fail2ban +### How to automate blocking with fail2ban You can hook up [fail2ban](https://www.fail2ban.org) to automatically block connections from these misbehaving peers if they emit this @@ -349,14 +353,13 @@ RestartSec=1min User=ipfs ``` -### Example screen recording of fail2ban in action +#### Example screen recording of fail2ban in action - [fail2ban+go-libp2p screen recording](/images/fail2bango-libp2p.mp4) -### Setting Up fail2ban +#### Setting Up fail2ban We’ll focus on the specifics around fail2ban and go-libp2p here. The steps to take are: @@ -384,7 +387,7 @@ Status for the jail: go-libp2p-weird-behavior-iptables Then you’re good to go! You’ve successfully set up a go-libp2p jail. -## Deny specific peers or create an allow list of trusted peers +### Deny specific peers or create an allow list of trusted peers The go-libp2p [resource manager](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager) can accept a list of trusted multiaddrs and can use a different set of limits in @@ -396,7 +399,7 @@ for more details. js-libp2p provides a straightforward allow and deny list mechanism with its [connection manager limits](https://github.com/libp2p/js-libp2p/blob/master/doc/LIMITS.md#allowdeny-lists). -# Summary +## Summary Mitigating DOS attacks is hard because an attacker needs only one flaw, while a protocol developer needs to cover _all_ their bases. Libp2p provides some tools diff --git a/content/concepts/fundamentals/_index.md b/content/concepts/fundamentals/_index.md new file mode 100644 index 00000000..6dc80b43 --- /dev/null +++ b/content/concepts/fundamentals/_index.md @@ -0,0 +1,11 @@ +--- +title: "Fundamentals" +weight: 1 +pre: ' ' +chapter: true +summary: Learn about the core pillars that compose each libp2p peer and a libp2p network. +--- + +# Key Components of libp2p + +{{% children description="true"%}} diff --git a/content/concepts/addressing.md b/content/concepts/fundamentals/addressing.md similarity index 88% rename from content/concepts/addressing.md rename to content/concepts/fundamentals/addressing.md index c86dcc03..aa7c258e 100644 --- a/content/concepts/addressing.md +++ b/content/concepts/fundamentals/addressing.md @@ -1,8 +1,14 @@ --- -title: Addressing -weight: 6 +title: "Addressing" +weight: 1 +pre: ' ' +chapter: true +aliases: /concepts/addressing/ +summary: Flexible networks need flexible addressing systems. Since libp2p is designed to work across a wide variety of networks, we need a way to work with a lot of different addressing schemes in a consistent way. --- +# Addressing + Flexible networks need flexible addressing systems. Since libp2p is designed to work across a wide variety of networks, we need a way to work with a lot of different addressing schemes in a consistent way. A `multiaddress` (often abbreviated `multiaddr`), is a convention for encoding multiple layers of addressing information into a single "future-proof" path structure. It [defines][spec_multiaddr] human-readable and machine-optimized encodings of common transport and overlay protocols and allows many layers of addressing to be combined and used together. diff --git a/content/concepts/peers.md b/content/concepts/fundamentals/peers.md similarity index 92% rename from content/concepts/peers.md rename to content/concepts/fundamentals/peers.md index 888d2eab..5be83c8d 100644 --- a/content/concepts/peers.md +++ b/content/concepts/fundamentals/peers.md @@ -1,8 +1,19 @@ --- -title: All about Peers -weight: 4 +title: "Peers" +weight: 3 +pre: ' ' +chapter: true +aliases: + - /concepts/peer-id/ + - /concepts/peers/ +summary: Peers are what make up a libp2p network. They include unique identifiers and storage methods to facilitate peer-to-peer connections in libp2p. --- +# Peers + +Peers are what make up a libp2p network. We'll go over their key +properties in this guide. + ## Peer ID A Peer Identity (often written `PeerID`) is a unique reference to a specific diff --git a/content/concepts/protocols.md b/content/concepts/fundamentals/protocols.md similarity index 97% rename from content/concepts/protocols.md rename to content/concepts/fundamentals/protocols.md index 7cc924b6..4079d38e 100644 --- a/content/concepts/protocols.md +++ b/content/concepts/fundamentals/protocols.md @@ -1,8 +1,14 @@ --- -title: Protocols -weight: 3 +title: "Protocols" +weight: 2 +pre: ' ' +chapter: true +aliases: /concepts/protocols/ +summary: There are protocols everywhere you look when you're writing network applications, and libp2p is especially thick with them. --- +# Protocols + There are protocols everywhere you look when you're writing network applications, and libp2p is especially thick with them. diff --git a/content/reference/glossary.md b/content/concepts/glossary.md similarity index 99% rename from content/reference/glossary.md rename to content/concepts/glossary.md index b70443d9..858555d8 100644 --- a/content/reference/glossary.md +++ b/content/concepts/glossary.md @@ -1,8 +1,13 @@ --- title: "Glossary" -weight: 1 +weight: 12 +pre: ' ' +chapter: true +summary: A compiled list of words, phrases, and abbreviations for libp2p. --- +# Glossary + ### Boot Node New nodes in a p2p network often make their initial connection to the p2p network through a set of nodes known as boot nodes. Information (e.g. addresses) about these boot nodes is e.g. embedded in an application binary or provided as a configuration option. @@ -108,7 +113,6 @@ In the peer-to-peer model, accepting connections from other peers is often just ### NAT Traversal - NAT traversal refers to the process of establishing connections with other machines across a [NAT](#nat) boundary. When crossing the boundary between IP networks (e.g. from a local network to the global internet), a [Network Address Translation](#nat) process occurs which maps addresses from one space to another. For example, my home network has an internal range of IP addresses (10.0.1.x), which is part of a range of addresses that are reserved for private networks. If I start a program on my computer that listens for connections on its internal address, a user from the public internet has no way of reaching me, even if they know my public IP address. This is because I haven't made my router aware of my program yet. When a connection comes in from the internet to my public IP address, the router needs to figure out which internal IP to route the request to, and to which port. @@ -117,7 +121,6 @@ There are many ways to inform one's router about services you want to expose. Fo In some cases, automatic NAT traversal is impossible, often because multiple layers of NAT are involved. In such cases, we still want to be able to communicate, and we especially want to be reachable and allow other peers to [dial in](#dial) and use our services. This is the one of the motivations for [Circuit Relay](#circuit-relay), which is a protocol involving a "relay" peer that is publicly reachable and can route traffic on behalf of others. Once a relay circuit is established, a peer behind an especially intractable NAT can advertise the relay circuit's [multiaddr](#multiaddr), and the relay will accept incoming connections on our behalf and send us traffic via the relay. - ### Node The word "node" is quite overloaded in general programming contexts, and this is especially the case in peer-to-peer networking circles. @@ -251,5 +254,4 @@ In libp2p, `transport` refers to the technology that lets us move bits from one Note that in some environments such as javascript running in the browser, not all transports will be available. In such cases, it may be possible to establish a [Circuit Relay](#circuit-relay) with the help of a peer that can support many common transports. Such a relay can act as a "transport adapter" of sorts, allowing peers that can't communicate with each other directly to interact. For example, a peer in the browser that can only make websocket connections could relay through a peer able to make TCP connections, which would enable communication with a wider variety of peers. - [js-docs-home]: /reference/ diff --git a/content/concepts/hole-punching.md b/content/concepts/hole-punching.md index 39dfb54d..4d62c84f 100644 --- a/content/concepts/hole-punching.md +++ b/content/concepts/hole-punching.md @@ -1,11 +1,12 @@ --- -title: Hole Punching -weight: 3 +title: "Hole Punching" +weight: 8 +pre: ' ' +chapter: true +summary: In most configurations, both public and non-public nodes can dial connections to other public nodes. However, it's not possible to establish a connection from the public internet to a non-public node. --- -## Background - -### Types of nodes +# Hole Punching Nodes on a peer-to-peer network can be categorized into two groups: public and non-public. Public nodes are those nodes that have unobstructed @@ -15,7 +16,7 @@ as well as mobile phones. In most configurations, both public and non-public nodes can dial connections to other public nodes. However, it's not possible to establish a connection from the public internet to a non-public node. -### How can a node become dialable despite being behind a firewall and/or NAT? +## Dialing a non-public node Here are a few methods that nodes can use to dial a non-public node: diff --git a/content/concepts/multiplex/_index.md b/content/concepts/multiplex/_index.md new file mode 100644 index 00000000..cc4f1635 --- /dev/null +++ b/content/concepts/multiplex/_index.md @@ -0,0 +1,34 @@ +--- +title: "Stream Multiplexing" +weight: 4 +pre: ' ' +chapter: true +aliases: /concepts/stream-multiplexing/ +summary: Stream Multiplexing is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed so it can be output and used by separate applications. +--- + +# Stream bytes + +Stream Multiplexing (_stream muxing_) is a way of sending multiple streams of data over one +communication link. It combines multiple signals into one unified signal so it can be transported +'over the wires', then it is demulitiplexed (_demuxed_) so it can be output and used by separate +applications. This is done to share a single TCP connection using unique port numbers to distinguish +streams, between the multiple proceeses (such as kademlia and gossipsub) used by applications (such as IPFS) +to make connection and transmission more efficient. + +With muxing, libp2p applications may have many separate streams of communication between peers, as well as +have multiple concurrent streams open at the same time with a peer. Stream multiplexing allows us to initialize +and use the same [transport](/concepts/transport/) connection across the lifetime of our interaction with a peer. +With muxing, we also only need to deal with [NAT traversal](nat) once to be able to open as many +streams as we need, since they will all share the same underlying transport connection. Applications can enable +support for multiple multiplexers, which will allow you to fall back to a widely-supported multiplexer if a preferred +choice is not supported by a remote peer. + +{{% notice "info" %}} +libp2p's multiplexing happens at the application layer, meaning it's not provided by the +operating system's network stack. However, developers writing libp2p applications rarely need to i +nteract with stream multiplexers directly, except during initial configuration to control which +modules are enabled. +{{% /notice %}} + +{{% children description="true"%}} diff --git a/content/concepts/multiplex/mplex.md b/content/concepts/multiplex/mplex.md new file mode 100644 index 00000000..e0d819ad --- /dev/null +++ b/content/concepts/multiplex/mplex.md @@ -0,0 +1,18 @@ +--- +title: "mplex" +weight: 2 +pre: ' ' +chapter: true +summary: mplex is a protocol developed for libp2p. +--- + +# mplex + +mplex is a protocol developed for libp2p. The [spec](https://github.com/libp2p/specs/tree/master/mplex) defines a simple protocol for multiplexing that is widely supported across libp2p language implementations: + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} + diff --git a/content/concepts/multiplex/quic.md b/content/concepts/multiplex/quic.md new file mode 100644 index 00000000..f84fa697 --- /dev/null +++ b/content/concepts/multiplex/quic.md @@ -0,0 +1,19 @@ +--- +title: "QUIC" +weight: 4 +pre: ' ' +chapter: true +summary: QUIC is a transport protocol that contains a native stream multiplexer. +--- + +# QUIC + +QUIC is a [transport](/concepts/transport/) protocol that contains a "native" stream multiplexer. +libp2p will automatically use the native multiplexer for streams using a QUIC transport. View the [QUIC +section](/concepts/transport/quic/) to learn about QUIC. + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/multiplex/switch.md b/content/concepts/multiplex/switch.md new file mode 100644 index 00000000..e43ff067 --- /dev/null +++ b/content/concepts/multiplex/switch.md @@ -0,0 +1,15 @@ +--- +title: "Switch" +weight: 1 +pre: ' ' +chapter: true +summary: libp2p maintains some state about known peers and existing connections in a component known as the switch +--- + +# Switch + +libp2p maintains some state about known peers and existing connections in a component known as the switch (or "swarm", depending on the implementation). The switch provides a dialing and listening interface that abstracts the details of which stream multiplexer is used for a given connection. + +When configuring libp2p, applications enable stream muxing modules, which the switch will use when dialing peers and listening for connections. If the remote peers support any of the same stream muxing implementations, the switch will select and use it when establishing the connection. If you dial a peer that the switch already has an open connection to, the new stream will automatically be multiplexed over the existing connection. + +Reaching agreement on which stream multiplexer to use happens early in the connection establishment process. Peers use [protocol negotiation](/concepts/fundamentals/protocols/#protocol-negotiation/) to agree on a commonly supported multiplexer, which upgrades a "raw" transport connection into a muxed connection capable of opening new streams. diff --git a/content/concepts/multiplex/yamux.md b/content/concepts/multiplex/yamux.md new file mode 100644 index 00000000..e695ec3d --- /dev/null +++ b/content/concepts/multiplex/yamux.md @@ -0,0 +1,19 @@ +--- +title: "yamux" +weight: 3 +pre: ' ' +chapter: true +summary: yamux is a multiplexing protocol designed by Hashicorp. +--- + +# yamux + +[yamux](https://github.com/hashicorp/yamux) is a multiplexing protocol designed by [Hashicorp](https://www.hashicorp.com/). +yamux offers more sophisticated flow control than mplex, and can scale to thousands of multiplexed streams over a single +connection. + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/nat.md b/content/concepts/nat.md deleted file mode 100644 index de32d6c0..00000000 --- a/content/concepts/nat.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -title: NAT Traversal -weight: 2 ---- - -The internet is composed of countless networks, bound together into shared address spaces by foundational [transport protocols](/concepts/transport/). - -As traffic moves between network boundaries, it's very common for a process called Network Address Translation to occur. Network Address Translation (NAT) maps an address from one address space to another. - -NAT allows many machines to share a single public address, and it is essential for the continued functioning of the IPv4 protocol, which would otherwise be unable to serve the needs of the modern networked population with its 32-bit address space. - -For example, when I connect to my home wifi, my computer gets an IPv4 address of `10.0.1.15`. This is part of a range of IP addresses reserved for internal use by private networks. When I make an outgoing connection to a public IP address, the router replaces my internal IP with its own public IP address. When data comes back from the other side, the router will translate back to the internal address. - -While NAT is usually transparent for outgoing connections, listening for incoming connections requires some configuration. The router listens on a single public IP address, but any number of machines on the internal network could handle the request. To serve requests, your router must be configured to send certain traffic to a specific machine, usually by mapping one or more TCP or UDP ports from the public IP to an internal one. - -While it's usually possible to manually configure routers, not everyone that wants to run a peer-to-peer application or other network service will have the ability to do so. - -We want libp2p applications to run everywhere, not just in data centers or on machines with stable public IP addresses. To enable this, here are the main approaches to NAT traversal available in libp2p today. - -### Automatic router configuration - -Many routers support automatic configuration protocols for port forwarding, most commonly [UPnP][wiki_upnp] or [nat-pmp.][wiki_nat-pmp] - -If your router supports one of those protocols, libp2p will attempt to automatically configure a port mapping that will allow it to listen for incoming traffic. This is usually the simplest option if supported by the network and libp2p implementation. - -{{% notice "info" %}} -Support for automatic NAT configuration varies by libp2p implementation. -Check the [current implementation status](https://libp2p.io/implementations/#nat-traversal) for details. -{{% /notice %}} - -### Hole-punching (STUN) - -When an internal machine "dials out" and makes a connection to a public address, the router will map a public port to the internal IP address to use for the connection. In some cases, the router will also accept *incoming* connections on that port and route them to the same internal IP. - -libp2p will try to take advantage of this behavior when using IP-backed transports by using the same port for both dialing and listening, using a socket option called [`SO_REUSEPORT`](https://lwn.net/Articles/542629/). - -If our peer is in a favorable network environment, they will be able to make an outgoing connection and get a publicly-reachable listening port "for free," but they might never know it. Unfortunately, there's no way for the dialing program to discover what port was assigned to the connection on its own. - -However, an external peer can tell us what address they observed us on. We can then take that address and advertise it to other peers in our [peer routing network](/concepts/peer-routing/) to let them know where to find us. - -This basic premise of peers informing each other of their observed addresses is the foundation of [STUN][wiki_stun] (Session Traversal Utilities for NAT), which [describes][rfc_stun] a client / server protocol for discovering publicly reachable IP address and port combinations. - -One of libp2p's core protocols is the [identify protocol][spec_identify], which allows one peer to ask another for some identifying information. When sending over their [public key](/concepts/peers/) and some other useful information, the peer being identified includes the set of addresses that it has observed for the peer asking the question. - -This external discovery mechanism serves the same role as STUN, but without the need for a set of "STUN servers". - -The identify protocol allows some peers to communicate across NATs that would otherwise be impenetrable. - -### AutoNAT - -While the [identify protocol][spec_identify] described above lets peers inform each other about their observed network addresses, not all networks will allow incoming connections on the same port used for dialing out. - -Once again, other peers can help us observe our situation, this time by attempting to dial us at our observed addresses. If this succeeds, we can rely on other peers being able to dial us as well and we can start advertising our listen address. - -A libp2p protocol called AutoNAT lets peers request dial-backs from peers providing the AutoNAT service. - -{{% notice "info" %}} -AutoNAT is currently implemented in go-libp2p via [go-libp2p-autonat](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/autonat). -{{% /notice %}} - - -### Circuit Relay (TURN) - -In some cases, peers will be unable to traverse their NAT in a way that makes them publicly accessible. - -libp2p provides a [Circuit Relay protocol](/concepts/circuit-relay/) that allows peers to communicate indirectly via a helpful intermediary peer. - -This serves a similar function to the [TURN protocol](https://tools.ietf.org/html/rfc5766) in other systems. - -[wiki_upnp]: https://en.wikipedia.org/wiki/Universal_Plug_and_Play -[wiki_nat-pmp]: https://en.wikipedia.org/wiki/NAT_Port_Mapping_Protocol -[wiki_stun]: https://en.wikipedia.org/wiki/STUN -[rfc_stun]: https://tools.ietf.org/html/rfc3489 -[lwn_reuseport]: https://lwn.net/Articles/542629/ -[spec_identify]: https://github.com/libp2p/specs/tree/master/identify diff --git a/content/concepts/nat/_index.md b/content/concepts/nat/_index.md new file mode 100644 index 00000000..261634a4 --- /dev/null +++ b/content/concepts/nat/_index.md @@ -0,0 +1,42 @@ +--- +title: "NAT Traversal" +weight: 5 +pre: ' ' +chapter: true +summary: We want libp2p applications to run everywhere, not just in data centers or on machines with stable public IP addresses. Learn about the main approaches to NAT traversal available in libp2p. +--- + +# Traverse Bytes + +The internet is composed of countless networks, bound together into shared address spaces by foundational [transport protocols](/concepts/transport/). + +As traffic moves between network boundaries, it's very common for a process called Network Address Translation to occur. Network Address Translation (NAT) maps an address from one address space to another. + +NAT allows many machines to share a single public address, and it is essential for the continued functioning of the IPv4 protocol, which would otherwise be unable to serve the needs of the modern networked population with its 32-bit address space. + +For example, when I connect to my home wifi, my computer gets an IPv4 address of `10.0.1.15`. This is part of a range of IP addresses reserved for internal use by private networks. When I make an outgoing connection to a public IP address, the router replaces my internal IP with its own public IP address. When data comes back from the other side, the router will translate back to the internal address. + +While NAT is usually transparent for outgoing connections, listening for incoming connections requires some configuration. The router listens on a single public IP address, but any number of machines on the internal network could handle the request. To serve requests, your router must be configured to send certain traffic to a specific machine, usually by mapping one or more TCP or UDP ports from the public IP to an internal one. + +While it's usually possible to manually configure routers, not everyone that wants to run a peer-to-peer application or other network service will have the ability to do so. + +We want libp2p applications to run everywhere, not just in data centers or on machines with stable public IP addresses. To enable this, here are the main approaches to NAT traversal available in libp2p today. + +### Automatic router configuration + +Many routers support automatic configuration protocols for port forwarding, most commonly [UPnP][wiki_upnp] or [nat-pmp.][wiki_nat-pmp] + +If your router supports one of those protocols, libp2p will attempt to automatically configure a port mapping that will allow it to listen for incoming traffic. This is usually the simplest option if supported by the network and libp2p implementation. + +{{% notice "info" %}} +Support for automatic NAT configuration varies by libp2p implementation. +Check the [current implementation status](https://libp2p.io/implementations/#nat-traversal) for details. +{{% /notice %}} + +{{% children description="true"%}} + +[wiki_upnp]: https://en.wikipedia.org/wiki/Universal_Plug_and_Play +[wiki_nat-pmp]: https://en.wikipedia.org/wiki/NAT_Port_Mapping_Protocol +[wiki_stun]: https://en.wikipedia.org/wiki/STUN +[rfc_stun]: https://tools.ietf.org/html/rfc3489 +[lwn_reuseport]: https://lwn.net/Articles/542629/ diff --git a/content/concepts/nat/autonat.md b/content/concepts/nat/autonat.md new file mode 100644 index 00000000..3696f95f --- /dev/null +++ b/content/concepts/nat/autonat.md @@ -0,0 +1,21 @@ +--- +title: "AutoNAT" +weight: 2 +pre: ' ' +chapter: true +summary: AutoNAT lets peers request dial-backs from peers providing the AutoNAT service. +--- + +# AutoNAT + +While the [identify protocol][spec_identify] described above lets peers inform each other about their observed network addresses, not all networks will allow incoming connections on the same port used for dialing out. + +Once again, other peers can help us observe our situation, this time by attempting to dial us at our observed addresses. If this succeeds, we can rely on other peers being able to dial us as well and we can start advertising our listen address. + +A libp2p protocol called AutoNAT lets peers request dial-backs from peers providing the AutoNAT service. + +{{% notice "info" %}} +AutoNAT is currently implemented in go-libp2p via [go-libp2p-autonat](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/autonat). +{{% /notice %}} + +[spec_identify]: https://github.com/libp2p/specs/tree/master/identify diff --git a/content/concepts/circuit-relay.md b/content/concepts/nat/circuit-relay.md similarity index 96% rename from content/concepts/circuit-relay.md rename to content/concepts/nat/circuit-relay.md index ffd44889..a5683c2e 100644 --- a/content/concepts/circuit-relay.md +++ b/content/concepts/nat/circuit-relay.md @@ -1,8 +1,13 @@ --- -title: Circuit Relay +title: "Circuit Relay" weight: 3 +pre: ' ' +chapter: true +summary: Circuit relay is a transport protocol that routes traffic between two peers over a third-party "relay" peer. --- +# Circuit Relay + Circuit relay is a [transport protocol](/concepts/transport/) that routes traffic between two peers over a third-party "relay" peer. In many cases, peers will be unable to [traverse their NAT and/or firewall](/concepts/nat/) in a way that makes them publicly accessible. Or they may not share common [transport protocols](/concepts/transport/) that would allow them to communicate directly. diff --git a/content/concepts/nat/hole-punching.md b/content/concepts/nat/hole-punching.md new file mode 100644 index 00000000..fdc49c20 --- /dev/null +++ b/content/concepts/nat/hole-punching.md @@ -0,0 +1,25 @@ +--- +title: "Hole Punching" +weight: 1 +pre: ' ' +chapter: true +summary: The internet is composed of countless networks, bound together into shared address spaces by foundational transport protocols. As traffic moves between network boundaries, it's very common for a process called Network Address Translation to occur. Network Address Translation (NAT) maps an address from one address space to another. +--- + +# Hole Punching + + + +When an internal machine "dials out" and makes a connection to a public address, the router will map a public port to the internal IP address to use for the connection. In some cases, the router will also accept *incoming* connections on that port and route them to the same internal IP. + +libp2p will try to take advantage of this behavior when using IP-backed transports by using the same port for both dialing and listening, using a socket option called [`SO_REUSEPORT`](https://lwn.net/Articles/542629/). + +If our peer is in a favorable network environment, they will be able to make an outgoing connection and get a publicly-reachable listening port "for free," but they might never know it. Unfortunately, there's no way for the dialing program to discover what port was assigned to the connection on its own. +However, an external peer can tell us what address they observed us on. We can then take that address and advertise it to other peers in our [peer routing network](/concepts/peer-routing/) to let them know where to find us. + +This basic premise of peers informing each other of their observed addresses is the foundation of [STUN][wiki_stun] (Session Traversal Utilities for NAT), which [describes][rfc_stun] a client / server protocol for discovering publicly reachable IP address and port combinations. + +One of libp2p's core protocols is the [identify protocol][spec_identify], which allows one peer to ask another for some identifying information. When sending over their [public key](/concepts/peers/) and some other useful information, the peer being identified includes the set of addresses that it has observed for the peer asking the question. + +This external discovery mechanism serves the same role as STUN, but without the need for a set of "STUN servers". +The identify protocol allows some peers to communicate across NATs that would otherwise be impenetrable. diff --git a/content/concepts/peer-routing.md b/content/concepts/peer-routing.md deleted file mode 100644 index 4a406b46..00000000 --- a/content/concepts/peer-routing.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: Peer Routing -weight: 5 ---- - -This article is coming soon! - -Please [refer to this issue](https://github.com/libp2p/docs/issues/13) to track the progress and make suggestions. diff --git a/content/concepts/publish-subscribe.md b/content/concepts/publish-subscribe.md index d8b2a08d..897039bf 100644 --- a/content/concepts/publish-subscribe.md +++ b/content/concepts/publish-subscribe.md @@ -1,14 +1,20 @@ --- -title: Publish/Subscribe +title: "Publish/Subscribe" +weight: 8 +pre: ' ' +chapter: true +summary: Publish/Subscribe is a system where peers congregate around topics they are interested in. Peers interested in a topic are said to be subscribed to that topic. Learn about how peers can message data in libp2p. --- +# Message Data + Publish/Subscribe is a system where peers congregate around topics they are interested in. Peers interested in a topic are said to be subscribed to that topic: ![Diagram showing a shaded area with curved outline representing a topic. Scattered within the shaded area are twelve dots representing peers. A label -points to the dots which reads “Peers subscribed to topic”.](subscribed_peers.png) +points to the dots which reads “Peers subscribed to topic”.](../assets/publish-subscribe/subscribed_peers.png) Peers can send messages to topics. Each message gets delivered to all peers subscribed to the topic: @@ -18,7 +24,7 @@ panel are scattered dots within a shaded area representing peers subscribed to a topic. From one of the dots comes a speech bubble labeled with “Message”. In the second panel all dots now have a copy of the speech bubble above them representing that the message has been transmitted to all peers subscribed to -the topic.](message_delivered_to_all.png) +the topic.](../assets/publish-subscribe/message_delivered_to_all.png) Example uses of pub/sub: @@ -84,7 +90,7 @@ labelled “Full-message peering” connect all the dots in a loose mesh, formin many triangles and polygons. Between these lines runs a dense mesh of thinner, lighter lines labelled “Metadata-only peering”. These lines run from each dot to almost every other dot around it, criss-crossing over each other -frequently.](types_of_peering.png) +frequently.]../assets/publish-subscribe/types_of_peering.png) ### Full-message @@ -110,7 +116,7 @@ different dot has only two lines running from it and is labelled “Peer reached lower bound”. Beneath the diagram is a legend reading “Network peering degree = 3; Upper bound = 4; Lower bound = 2“ accompanied with small symbols showing dots with three, four and two lines running from them -respectively.](full_message_network.png) +respectively.](../assets/publish-subscribe/full_message_network.png)
Throughout this guide, numbers
highlighted in purple can be configured
@@ -142,7 +148,7 @@ performs functions to help maintain the network of full-message peerings.
![Diagram showing a large shaded area with scattered dots inside connected by
many thin, light lines representing metadata-only peerings between peers. The
lines between the dots are labelled “Each peering is a network connection
-between two peers”.](metadata_only_network.png)
+between two peers”.](../assets/publish-subscribe/metadata_only_network.png)
## Grafting and pruning
@@ -163,7 +169,7 @@ on the right. In the following step, the line becomes thick, dark and is now
labelled “Full-message peering”. The second process is the reverse of the first
process; two dots are connected by a thick, dark line which becomes a thin,
light line labelled “Metadata-only peering”. The speech bubble reads “I’m
-pruning our connection back to a metadata-only peering.”](graft_prune.png)
+pruning our connection back to a metadata-only peering.”](../assets/publish-subscribe/graft_prune.png)
When a peer has too few full-message peerings it will randomly graft some of its
metadata-only peerings to become full-message peerings:
@@ -182,7 +188,7 @@ now highlighted lines. Four circles in the series are also highlighted green, up
to the circle labelled “Ideal”. The panel is titled “Select more peers to graft
to get to the ideal number”. In the final panel the highlighted green lines and
dots have become dark to indicate they have become full-content peerings. The
-title reads “Grafting complete”.](maintain_graft.png)
+title reads “Grafting complete”.](../assets/publish-subscribe/maintain_graft.png)
Conversely, when a peer has too many full-message peerings it will randomly
prune some of them back to metadata-only:
@@ -203,7 +209,7 @@ series are also highlighted pink, from the end down to the circle labelled
“Ideal”. The panel is titled “Select peers to prune to get the ideal number”. In
the final panel the highlighted pink lines and dots have become light to
indicate they have become metadata-only peerings. The title reads “Pruning
-complete”.](maintain_prune.png)
+complete”.](../assets/publish-subscribe/maintain_prune.png)
In libp2p’s implementation, each peer performs a series of checks every
1 second. These checks are called the
@@ -224,7 +230,7 @@ from “Topic 1” to “Topic 5”. The shaded areas fade out with distance fro
large, central dot indicating that the peer’s perspective has limited range.
One of the smaller dots that does not share a shaded area with the large dot is
labelled “Peers keep track of other’s subscriptions even if not subscribed to
-the same topics as them”.](subscriptions_local_view.png)
+the same topics as them”.](../assets/publish-subscribe/subscriptions_local_view.png)
Keeping track of subscriptions happens by sending **subscribe** and
**unsubscribe** messages. When a new connection is established between two peers
@@ -234,7 +240,7 @@ they start by sending each other the list of topics they are subscribed to:
speech bubble emanating from it with an arrow showing it moving along the
connecting line towards the other dot. The left dot’s speech bubble says “I am
subscribed to: Topic 1, Topic 2, Topic 3”. The right dot’s speech bubble says “I
-am subscribed to: Topic 1, Topic 5”.](subscription_list_first_connect.png)
+am subscribed to: Topic 1, Topic 5”.](../assets/publish-subscribe/subscription_list_first_connect.png)
Then over time, whenever a peer subscribes or unsubscribes from a topic, it will
send each of its peers a subscribe or unsubscribe message. These messages are
@@ -244,7 +250,7 @@ subscribed to the topic in question:
![Diagram showing two dots connected by a thin, light line. The left dot has a
speech bubble emanating from it with an arrow showing it moving along the
connecting line towards the right dot. The speech bubble says “I am
-subscribed to: Topic 5. I am unsubscribed from: Topic 2, Topic 3.”](subscription_list_change.png)
+subscribed to: Topic 5. I am unsubscribed from: Topic 2, Topic 3.”](../assets/publish-subscribe/subscription_list_change.png)
Subscribe and unsubscribe messages go hand-in-hand with graft and prune
messages. When a peer subscribes to a topic it will pick some peers that will
@@ -261,7 +267,7 @@ subscribed to Topic 3. Also, I’m grafting our connection into a full-message
peering.” The next panel shows the same three dots, however the central dot is
now inside the shaded area labelled “Topic 3” and the line connecting the
central and right dots has become thick and dark indicating a full-message
-peering.](subscribe_graft.png)
+peering.](../assets/publish-subscribe/subscribe_graft.png)
When a peer unsubscribes from a topic it will notify its full-message peers that
their connection has been pruned at the same time as sending their unsubscribe
@@ -280,7 +286,7 @@ pruning our connection back to a metadata-only peering.” The next panel shows
the same three dots, however the central dot is no longer inside the area
labelled “Topic 3” and the line connecting the central and right dots has become
thin and light like the left line to indicate a metadata-only
-peering.](unsubscribe_prune.png)
+peering.](../assets/publish-subscribe/unsubscribe_prune.png)
## Sending messages
@@ -292,7 +298,7 @@ panel is titled “Peer creates new message of its own”. A small, unlabelled
speech bubble emanates from a dot. The dot has four thick, dark lines radiating
outward from it. The second panel is titled “Message sent to all other
full-message peers”. It shows four copies of the speech bubble now moving away
-from the dot along each of the four lines.](full_message_send.png)
+from the dot along each of the four lines.](../assets/publish-subscribe/full_message_send.png)
Similarly, when a peer receives a new message from another peer, it stores the
message and forwards a copy to all other full-message peers it is connected to:
@@ -305,7 +311,7 @@ thick, dark lines radiating outward from the dot. The second panel is titled
outward. The speech bubble is now centred above the dot. The final panel is
titled “Message forwarded to all other full-message peers”. It shows three
copies of the speech bubble now moving away from the dot along the three lines
-that the speech bubble has not appeared on yet.](full_message_forward.png)
+that the speech bubble has not appeared on yet.](../assets/publish-subscribe/full_message_forward.png)
In the [gossipsub specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md#controlling-the-flood),
peers are also known as *routers* because of this function they have in routing
@@ -340,7 +346,7 @@ from the central dot. The six speech bubbles are identical and read “I have
seen:” followed by three small speech bubble symbols inside the larger speech
bubble, in different shades of purple to indicate three different messages.
One of the small purple speech bubbles is labelled “Seen messages specify the
-sender and sequence number, but not the full message contents”.](gossip_deliver.png)
+sender and sequence number, but not the full message contents”.](../assets/publish-subscribe/gossip_deliver.png)
Gossiping gives peers a chance to notice in case they missed a message on the
full-message network. If a peer notices it is repeatedly missing messages then
@@ -377,7 +383,7 @@ now a purple speech bubble travelling along the line connecting the two dots
from left to right. The final panel is titled “Newly received message is
broadcast to full-content peers”. There are now three copies of the purple
speech bubble travelling outwards from the right dot along the three thick, dark
-lines connected to it.](request_gossiped_message.png)
+lines connected to it.](../assets/publish-subscribe/request_gossiped_message.png)
In the [gossipsub specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md#control-messages),
gossip announcing recently seen messages are called *IHAVE* messages and
@@ -407,7 +413,7 @@ connecting the outside dot to dots inside the shaded area have become blue and
now have arrowheads at the end pointing towards the peer inside the shaded area
they are connected to. These lines represent fan-out peerings. A dice symbol is
present indicating random selection of which metadata-only peerings became
-fan-out peerings.](fanout_initial_pick.png)
+fan-out peerings.](../assets/publish-subscribe/fanout_initial_pick.png)
Unlike the other types of peering, fan-out peerings are unidirectional; they
always point from the peer outside the topic to a peer subscribed to the topic.
@@ -427,7 +433,7 @@ peers subscribed to the topic. The second panel is titled “Once inside the
topic, the message is forwarded to all other subscribers as usual”. The purple
speech bubbles have moved from outside the shaded area to inside out. Now there
is a copy of the speech bubble above every dot in the shaded
-area.](fanout_message_send.png)
+area.](../assets/publish-subscribe/fanout_message_send.png)
If the sender goes to send a message but notices some of their fan-out peers
went away since last time, they will randomly select additional fan-out peers
@@ -450,7 +456,7 @@ peer that was previously outside the shaded area is now inside it, representing
that it is now subscribed to the topic. The three previously-blue arrows have
become thick, dark lines representing former fan-out peerings becoming
full-message peerings. The other lines from the dot are still thin and light as
-before.](fanout_grafting_preference.png)
+before.](../assets/publish-subscribe/fanout_grafting_preference.png)
After 2 minutes of not sending any messages to
a topic, all the fan-out peers for that topic are forgotten:
@@ -463,7 +469,7 @@ minutes…” There is a stopwatch above the single dot indicating the passage o
time. The second panel is titled “All fan-out peerings revert to metadata-only
peerings”. In this panel the three previously blue, arrowed lines connecting
the single dot to dots inside the shaded area have become thin and light,
-representing the peer’s fan-out peerings becoming metadata-only peerings.](fanout_forget.png)
+representing the peer’s fan-out peerings becoming metadata-only peerings.](../assets/publish-subscribe/fanout_forget.png)
## Network packets
@@ -507,7 +513,7 @@ columns, similar to the previous table. The first column is titled “For this
topic…”, however the second column is titled “You have been…” with the choice of
grafted or pruned. There are three rows in this table. Two of the topics
have been grafted and one has been pruned (no particular connection to the
-previous table).](network_packet_structure.png)
+previous table).](../assets/publish-subscribe/network_packet_structure.png)
See the [specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md#protobuf) for the exact [Protocol Buffers](https://developers.google.com/protocol-buffers)
schema used to encode network packets.
@@ -560,7 +566,7 @@ the full message column is empty. These rows only have the sender (Peer ID),
sequence number and time first seen columns populated. The table rows are
listed in order of time first seen, from 1 second ago in the top row to 90
seconds ago in the bottom row. Some of the sequence numbers are shared between
-messages, but only where the sender is different.](state.png)
+messages, but only where the sender is different.](../assets/publish-subscribe/state.png)
## More information
diff --git a/content/concepts/secure-comm/_index.md b/content/concepts/secure-comm/_index.md
new file mode 100644
index 00000000..c9a1d105
--- /dev/null
+++ b/content/concepts/secure-comm/_index.md
@@ -0,0 +1,17 @@
+---
+title: "Secure Communication"
+weight: 3
+pre: ' '
+chapter: true
+summary: Before two peers can transmit data, the communication channel they established with a transport protocol should be secure. Learn about secure channels in libp2p.
+---
+
+# Secure bytes
+
+Before two peers can transmit data, the communication channel they established
+with a transport protocol should be secure. A transport protocol like QUIC provides
+security guarantees out-of-the-box, but other transports in libp2p do not provide the
+logic to secure their channel. This requires an upgrade to the transport using an upgrader.
+Security is always established first over the raw connection.
+
+{{% children description="true"%}}
diff --git a/content/concepts/secure-comm/noise.md b/content/concepts/secure-comm/noise.md
new file mode 100644
index 00000000..ab126a6f
--- /dev/null
+++ b/content/concepts/secure-comm/noise.md
@@ -0,0 +1,13 @@
+---
+title: "Noise"
+weight: 1
+pre: ' '
+chapter: true
+summary: Learn about Noise in libp2p.
+---
+
+# Noise
+
+{{% notice "note" %}}
+Coming soon!
+{{% /notice %}}
diff --git a/content/concepts/secure-comm/tls.md b/content/concepts/secure-comm/tls.md
new file mode 100644
index 00000000..5aa05ff7
--- /dev/null
+++ b/content/concepts/secure-comm/tls.md
@@ -0,0 +1,13 @@
+---
+title: "TLS"
+weight: 2
+pre: ' '
+chapter: true
+summary: Learn about TLS 1.3 in libp2p.
+---
+
+# TLS
+
+{{% notice "note" %}}
+Coming soon!
+{{% /notice %}}
diff --git a/content/concepts/secure-comms.md b/content/concepts/secure-comms.md
deleted file mode 100644
index c6512a46..00000000
--- a/content/concepts/secure-comms.md
+++ /dev/null
@@ -1,8 +0,0 @@
----
-title: Secure Communication
-weight: 2
----
-
-This article is coming soon!
-
-Please [refer to this issue](https://github.com/libp2p/docs/issues/19) to track the progress and make suggestions.
diff --git a/content/concepts/security-considerations.md b/content/concepts/security/security-considerations.md
similarity index 94%
rename from content/concepts/security-considerations.md
rename to content/concepts/security/security-considerations.md
index ce845d9b..5b2944a1 100644
--- a/content/concepts/security-considerations.md
+++ b/content/concepts/security/security-considerations.md
@@ -1,10 +1,15 @@
---
-title: 'Security Considerations'
-weight: 12
+title: "Security Considerations"
+weight: 10
+pre: ' '
+chapter: true
+summary: libp2p makes it simple to establish encrypted, authenticated communication channels between two peers, but there are other important security issues to consider when building robust peer-to-peer systems.
---
-libp2p makes it simple to establish [encrypted, authenticated communication
-channels](../secure-comms/) between two peers, but there are other important
+# Secure Data
+
+libp2p makes it simple to establish encrypted, authenticated communication
+channels between two peers, but there are other important
security issues to consider when building robust peer-to-peer systems.
Many of the issues described here have no known "perfect solution," and the
diff --git a/content/concepts/stream-multiplexing.md b/content/concepts/stream-multiplexing.md
deleted file mode 100644
index d467286d..00000000
--- a/content/concepts/stream-multiplexing.md
+++ /dev/null
@@ -1,68 +0,0 @@
----
-title: Stream Multiplexing
----
-
-Stream Multiplexing (_stream muxing_) is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed (_demuxed_) so it can be output and used by separate applications.
-
-## Multiplexing
-
-Multiplexing is by no means unique to libp2p. Most communication networks involve some kind of multiplexing, as the transport medium is generally scarce and needs to be shared by many participants.
-
-This is done to share a single TCP connection using unique port numbers to distinguish streams, between the multiple proceeses (such as kademlia and gossipsub) used by applications (such as ipfs) to make connection and transmission more efficient. With muxing, libp2p applications may have many separate streams of communication between peers, as well as have multiple concurrent streams open at the same time with a peer.
-
-Stream multiplexing allows us to initialize and use the same [transport](/concepts/transport/) connection across the lifetime of our interaction with a peer. With muxing, we also only need to deal with [NAT traversal](nat) once to be able to open as many streams as we need, since they will all share the same underlying transport connection.
-
-libp2p provides a common [interface](#interface) for stream multiplexers with several [implementations](#implementations) available. Applications can enable support for multiple multiplexers, which will allow you to fall back to a widely-supported multiplexer if a preferred choice is not supported by a remote peer.
-
-## Where it fits in the libp2p stack
-
-libp2p's multiplexing happens at the "application layer", meaning it's not provided by the operating system's network stack. However, developers writing libp2p applications rarely need to interact with stream multiplexers directly, except during initial configuration to control which modules are enabled.
-
-### Switch / swarm
-
-libp2p maintains some state about known peers and existing connections in a component known as the switch (or "swarm", depending on the implementation). The switch provides a dialing and listening interface that abstracts the details of which stream multiplexer is used for a given connection.
-
-When configuring libp2p, applications enable stream muxing modules, which the switch will use when dialing peers and listening for connections. If the remote peers support any of the same stream muxing implementations, the switch will select and use it when establishing the connection. If you dial a peer that the switch already has an open connection to, the new stream will automatically be multiplexed over the existing connection.
-
-Reaching agreement on which stream multiplexer to use happens early in the connection establishment process. Peers use [protocol negotiation](/concepts/protocols/#protocol-negotiation/) to agree on a commonly supported multiplexer, which upgrades a "raw" transport connection into a muxed connection capable of opening new streams.
-
-## Interface and Implementations
-
-### Interface
-The [stream multiplexing interface][interface-stream-muxing] defines how a stream muxing module can be applied to a connection and what operations are supported by a multiplexed connection.
-
-### Implementations
-
-There are several stream multiplexing modules available in libp2p. Please note that not all stream muxers are supported by every libp2p language implementation.
-
-#### mplex
-
-mplex is a protocol developed for libp2p. The [spec](https://github.com/libp2p/specs/tree/master/mplex) defines a simple protocol for multiplexing that is widely supported across libp2p language implementations:
-
-- Go: [go-mplex](https://github.com/libp2p/go-mplex)
-- Javascript: [js-mplex](https://github.com/libp2p/js-libp2p-mplex)
-- Rust: [rust-libp2p mplex module](https://github.com/libp2p/rust-libp2p/tree/master/muxers/mplex)
-
-#### yamux
-
-[yamux](https://github.com/hashicorp/yamux) is a multiplexing protocol designed by [Hashicorp](https://www.hashicorp.com/).
-
-yamux offers more sophisticated flow control than mplex, and can scale to thousands of multiplexed streams over a single connection.
-
-yamux is currently supported in go and rust:
-
-- Go: [go-smux-yamux](https://github.com/whyrusleeping/go-smux-yamux)
-- Rust: [rust-libp2p yamux module](https://github.com/libp2p/rust-libp2p/tree/master/muxers/yamux).
-
-#### quic
-
-[QUIC][wiki-quic] is a [transport](/concepts/transport/) protocol that contains a "native" stream multiplexer. libp2p will automatically use the native multiplexer for streams using a quic transport.
-
-quic is currently supported in go via [go-libp2p-quic-transport](https://github.com/libp2p/go-libp2p/tree/master/p2p/transport/quic).
-
-
-[interface-stream-muxing]: https://github.com/libp2p/js-libp2p-interfaces/tree/master/packages/interface-stream-muxer
-
-[repo-multistream-select]: https://github.com/multiformats/multistream-select
-
-[wiki-quic]: https://en.wikipedia.org/wiki/QUIC
diff --git a/content/concepts/transport.md b/content/concepts/transport.md
deleted file mode 100644
index 6a9b967b..00000000
--- a/content/concepts/transport.md
+++ /dev/null
@@ -1,307 +0,0 @@
----
-title: Transport
-weight: 1
----
-
-When you make a connection from your computer to a machine on the internet,
-chances are pretty good you're sending your bits and bytes using TCP/IP, the
-wildly successful combination of the Internet Protocol, which handles addressing
-and delivery of data packets, and the Transmission Control Protocol, which
-ensures that the data that gets sent over the wire is received completely and in
-the right order.
-
-Because TCP/IP is so ubiquitous and well-supported, it's often the default
-choice for networked applications. In some cases, TCP adds too much overhead,
-so applications might use [UDP](https://en.wikipedia.org/wiki/User_Datagram_Protocol),
-a much simpler protocol with no guarantees about reliability or ordering.
-
-While TCP and UDP (together with IP) are the most common protocols in use today,
-they are by no means the only options. Alternatives exist at lower levels
-(e.g. sending raw ethernet packets or bluetooth frames), and higher levels
-(e.g. QUIC, which is layered over UDP).
-
-In libp2p, we call these foundational protocols that move bits around
-**transports**, and one of libp2p's core requirements is to be
-*transport agnostic*. This means that the decision of what transport protocol
-to use is up to the developer, and in fact one application can support many
-different transports at the same time.
-
-## Listening and Dialing
-Transports are defined in terms of two core operations, **listening** and
-**dialing**.
-
-Listening means that you can accept incoming connections from other peers,
-using whatever facility is provided by the
-transport implementation. For example, a TCP transport on a unix platform could
-use the `bind` and `listen` system calls to have the operating system route
-traffic on a given TCP port to the application.
-
-Dialing is the process of opening an outgoing connection to a listening peer.
-Like listening, the specifics are determined by the implementation, but every
-transport in a libp2p implementation will share the same programmatic interface.
-
-## Addresses
-
-Before you can dial up a peer and open a connection, you need to know how to
-reach them. Because each transport will likely require its own address scheme,
-libp2p uses a convention called a "multiaddress" or `multiaddr` to encode
-many different addressing schemes.
-
-The [addressing doc](/concepts/addressing/) goes into more detail, but an overview of
-how multiaddresses work is helpful for understanding the dial and listen
-interfaces.
-
-Here's an example of a multiaddr for a TCP/IP transport:
-
-```
-/ip4/7.7.7.7/tcp/6543
-```
-
-This is equivalent to the more familiar `7.7.7.7:6543` construction, but it
-has the advantage of being explicit about the protocols that are being
-described. With the multiaddr, you can see at a glance that the `7.7.7.7`
-address belongs to the IPv4 protocol, and the `6543` belongs to TCP.
-
-For more complex examples, see [Addressing](/concepts/addressing/).
-
-Both dial and listen deal with multiaddresses. When listening, you give the
-transport the address you'd like to listen on, and when dialing you provide the
-address to dial to.
-
-When dialing a remote peer, the multiaddress should include the
-[PeerId](/concepts/peers/) of the peer you're trying to reach.
-This lets libp2p establish a [secure communication channel](/concepts/secure-comms/)
-and prevents impersonation.
-
-An example multiaddress that includes a `PeerId`:
-
-```
-/ip4/1.2.3.4/tcp/4321/p2p/QmcEPrat8ShnCph8WjkREzt5CPXF2RwhYxYBALDcLC1iV6
-```
-
-The `/p2p/QmcEPrat8ShnCph8WjkREzt5CPXF2RwhYxYBALDcLC1iV6` component uniquely
-identifies the remote peer using the hash of its public key.
-For more, see [Peer Identity](/concepts/peers/).
-
-{{% notice "tip" %}}
-
-When [peer routing](/concepts/peer-routing/) is enabled, you can dial peers
-using just their PeerId, without needing to know their transport addresses
-before hand.
-
-{{% /notice %}}
-
-## Supporting multiple transports
-
-libp2p applications often need to support multiple transports at once. For
-example, you might want your services to be usable from long-running daemon
-processes via TCP, while also accepting websocket connections from peers running
-in a web browser.
-
-The libp2p component responsible for managing the transports is called the
-[switch][definition_switch], which also coordinates
-[protocol negotiation](/concepts/protocols/#protocol-negotiation),
-[stream multiplexing](/concepts/stream-multiplexing),
-[establishing secure communication](/concepts/secure-comms/) and other forms of
-"connection upgrading".
-
-The switch provides a single "entry point" for dialing and listening, and frees
-up your application code from having to worry about the specific transports
-and other pieces of the "connection stack" that are used under the hood.
-
-{{% notice "note" %}}
-The term "swarm" was previously used to refer to what is now called the "switch",
-and some places in the codebase still use the "swarm" terminology.
-{{% /notice %}}
-
-[definition_switch]: /reference/glossary/#switch
-
-## QUIC
-
-QUIC is a new transport protocol that provides an always-encrypted, stream-multiplexed
-connection built on top of UDP. It started as an experiment by Google between Google
-services and Chrome in 2014, and was later standardized by the IETF in
-[RFC 9000](https://datatracker.ietf.org/doc/html/rfc9000),
-[RFC 9001](https://datatracker.ietf.org/doc/html/rfc9001), and
-[RFC 9002](https://datatracker.ietf.org/doc/html/rfc9002).
-
-### Key challenges with TCP
-
-1. Head-of-line blocking (HoL blocking): TCP is a single byte stream exposed by the
- kernel, so streams layered on top of TCP experience head-of-line (HoL) blocking.
-
- {{% notice "info" %}}
- In TCP, head-of-line blocking occurs when a single packet is lost, and packets delivered
- after that need to wait in the kernel buffer until a retransmission for the lost packet
- is received.
- {{% /notice %}}
-
-2. Ossification: Because the header of TCP packet is not encrypted, middleboxes can
- inspect and modify TCP header fields and may break unexpectedly when they encounter
- anything they don’t understand. This makes it practically impossible to deploy any
- changes to the TCP protocol that change the wire format.
-
-3. Handshake inefficiency: TCP spends one network round-trip (RTT) on verifying the
- client's address. Only after this can TLS start the cryptographic handshake, consuming
- another RTT. Setting up an encrypted connection therefore always takes 2 RTTs.
-
-QUIC was designed with the following goals in mind:
-
-- Making the transport layer aware of streams, so that packet loss doesn't cause HoL blocking
- between streams.
-- Reducing the latency of connection establishment to a single RTT for new connections, and to
- allow sending of 0 RTT application data for resumed connections.
-- Encrypting as much as possible. This eliminates the ossification risk, as middleboxes aren't
- able to read any encrypted fields. This allows future evolution of the protocol.
-
-### Comparing HTTP/2 and HTTP/3
-
-In addition to defining the QUIC transport, the IETF also standardized a new version of HTTP that runs on top of QUIC: HTTP/3 (
-[RFC 9114](https://datatracker.ietf.org/doc/html/rfc9114)). HTTP/3 combines the advantages
-of the existing transfer protocols HTTP/2 and HTTP over QUIC in one standard for faster and
-more stable data transmission.
-
-The following diagram illustrates the OSI model for HTTP/2 and HTTP/3 [1]:
-
-![HTTP/2 & HTTP/3 OSI model](https://cloudspoint.xyz/wp-content/uploads/2022/03/http3.png)
-
-A web browser connection typically entails the following **(TCP+TLS+HTTP/2)**:
-
-1. Transport layer: TCP runs on top of the IP layer to provide a reliable
- byte stream.
- - TCP provides a reliable, bidirectional connection between two end systems.
-2. Security layer: A TLS handshake runs on top of TCP to
- establish an encrypted and authenticated connection.
- - Standard TLS over TCP requires 3 RTT. A typical TLS 1.3 handshake takes 1 RTT.
-3. Application layer: HTTP runs on a secure transport connection to transfer
- information and applies a stream muxer to serve multiple requests.
- - Application data starts to flow.
-
-In contrast, HTTP/3 runs over [QUIC](##what-is-quic), where QUIC is similar to
-TCP+TLS+HTTP/2 and runs over UDP. Building on UDP allows HTTP/3 to bypass the challenges
-found in TCP and use all the advantages of HTTP/2 and HTTP over QUIC.
-
-### What is QUIC?
-
-QUIC combines the functionality of these layers. Instead of TCP, it builds on UDP.
-When a UDP datagram is lost, it is not automatically retransmitted by the kernel.
-QUIC therefore takes responsibility for loss detection and repair itself. By using
-encryption, QUIC avoids ossified middleboxes. The TLS 1.3 handshake is performed in
-the first flight, removing the cost of verifying the client’s address and saving an
-RTT. QUIC also exposes multiple streams (and not just a single byte stream), so
-no stream multiplexer is needed at the application layer. Part of the application
-layer is also built directly into QUIC.
-
-In addition, a client can make use of QUIC's 0 RTT feature for subsequent connections
-when it has already communicated with a certain server. The client can then send
-(encrypted) application data even before the QUIC handshake has finished.
-
-### QUIC in libp2p
-
-libp2p only supports bidirectional streams and uses TLS 1.3 by default.
-Since QUIC already provides an encrypted, stream-multiplexed connection,
-libp2p directly uses QUIC streams, without any additional framing.
-
-To authenticate each others' peer IDs, peers encode their peer ID into a self-signed
-certificate, which they sign using their host's private key. This is the same way peer
-IDs are authenticated in the
-[libp2p TLS handshake](https://github.com/libp2p/specs/blob/master/tls/tls.md).
-
-{{% notice "note" %}}
-
-To be clear, there is no additional security handshake and stream muxer needed as QUIC
-provides all of this by default. This also means that establishing a libp2p connection between
-two nodes using QUIC only takes a single RTT.
-
-{{% /notice %}}
-
-Following the multiaddress format described earlier, a standard QUIC connection will
-look like: `/ip4/127.0.0.1/udp/65432/quic/`.
-
-## WebTransport
-
-While browsers perform HTTP request using HTTP/3, so far they don't offer an API to allow applications
-to gain access to the underlying QUIC stream.
-WebTransport is a new specification that uses QUIC to offer an alternative to
-WebSocket. Conceptually, it can be considered WebSocket over QUIC.
-It allows browsers to establish a stream-multiplexed and bidirectional connection
-to servers, and use streams to send and receive application data.
-
-While WebSocket provides a single bidirectional, full-duplex communication between a
-browser and a server over a TCP connection, WebTransport exposes allows the endpoints to use multiple
-streams in parallel.
-
-When connecting to a WebSocket server, browsers require the server to present a
-TLS certificate signed by a trusted CA (certificate authority). Few nodes have such
-a certificate, which is the reason that WebSocket never saw widespread adoption in the
-libp2p network. libp2p WebTransport offers a browser API that includes a way to
-accept the server's certificate by checking the (SHA-256) hash of the certificate
-(using the
-[`serverCertificateHashes` option](https://www.w3.org/TR/webtransport/#dom-webtransportoptions-servercertificatehashes)),
-even if the certificate is "just" a self-signed certificate. This allows us to connect
-any browser node to any server node, as long as the browser knows the certificate hash in
-advance (see [WebTransport in libp2p](#webtransport-in-libp2p) for how WebTransport addresses
-achieve this).
-
-Therefore, WebTransport exhibits all the advantages of QUIC over TCP, that being
-faster handshakes, no HoL blocking, and being future-proof.
-
-{{% notice "caution" %}}
-
-There is an experimental WebTransport transport in go-libp2p that is part
-of the [v0.23 release](https://github.com/libp2p/go-libp2p/releases/tag/v0.23.0).
-The implementation should be used experimentally and is not recommended for production
-environments.
-
-js-libp2p also plans to release
-[WebTransport support](https://github.com/libp2p/js-libp2p-webtransport) very soon.
-
-There are currently no concrete plans to support WebTransport beyond the Go and JS
-implementations.
-
-{{% /notice %}}
-
-For network stacks like libp2p, WebTransport is a pluggable
-protocol that fits well with a modular network design.
-
-For a standard WebSocket connection, the roundtrips required are as follows:
-
-- 1 RTT for TCP handshake
-- 1 RTT for TLS 1.3 handshake
-- 1 RTT for WebSocket upgrade
-- 1 RTT for multistream security negotiation (Noise or TLS 1.3)
-- 1 RTT for security handshake (Noise or TLS 1.3)
-- 1 RTT for multistream muxer negotiation (mplex or yamux)
-
-In total, 6 RTTs.
-
-WebTransport running over QUIC only requires 3 RTTs, as:
-
-- 1 RTT for QUIC handshake
-- 1 RTT for WebTransport handshake
-- 1 RTT for libp2p handshake; one for multistream and one for authentication
- (with a Noise handshake)
-
-In principle, the WebTransport protocol would even allow running the WebTransport
-handshake and the Noise handshake in parallel. However, this is currently not
-possible since the [browser API doesn't allow that yet](https://github.com/w3c/webtransport/issues/409).
-
-### WebTransport in libp2p
-
-WebTransport multiaddresses are composed of a QUIC multiaddr, followed
-by `/webtransport` and a list of multihashes of the node certificates that the server uses.
-
-For instance, for multiaddress `/ip4/127.0.0.1/udp/123/quic/webtransport/certhash/