libp2p-v0.53.0 #4797

thomaseizinger · 2023-11-05T21:18:03Z

thomaseizinger
Nov 5, 2023
Collaborator

The most ergonomic version of rust-libp2p yet!

We've been busy again, with over 250 PRs being merged into master since v0.52.0 (excluding dependency updates).

Backwards-compatible features

Numerous improvements landed as patch releases since the v0.52.0 release, for example a new, type-safe SwarmBuilder that also encompasses the most common transport protocols:

let mut swarm = libp2p::SwarmBuilder::with_new_identity()
    .with_tokio()
    .with_tcp(
        tcp::Config::default().port_reuse(true).nodelay(true),
        noise::Config::new,
        yamux::Config::default,
    )?
    .with_quic()
    .with_dns()?
    .with_relay_client(noise::Config::new, yamux::Config::default)?
    .with_behaviour(|keypair, relay_client| Behaviour {
        relay_client,
        ping: ping::Behaviour::default(),
        dcutr: dcutr::Behaviour::new(keypair.public().to_peer_id()),
    })?
    .build();

The new builder makes heavy use of the type-system to guide you towards a correct composition of all transports.
For example, it is important to compose the DNS transport as a wrapper around all other transports but before the relay transport.
Luckily, you no longer need to worry about these details as the builder takes care of that for you!
Have a look yourself if you dare here but be warned, the internals are a bit wild :)

Some more features that we were able to ship in v0.52.X patch-releases include:

stable QUIC implementation
for rust-libp2p compiled to WASM running in the browser
- WebTransport support
- WebRTC support
UPnP implementation to automatically configure port-forwarding with ones gateway
option to limit connections based on available memory

We always try to ship as many features as possible in a backwards-compatible way to get them to you faster.
Often times, these come with deprecations to give you a heads-up about what will change in a future version.
We advise updating to each intermediate version rather than skipping directly to the most recent one, to avoid missing any crucial deprecation warnings.
We highly recommend you stay up-to-date with the latest version to make upgrades as smooth as possible.

Some improvments we unfortunately cannot ship in a way that Rust considers a non-breaking change but with every release, we attempt to smoothen the way for future upgrades.

`#[non_exhaustive]` on key enums

We've identified that adding a new "message" to the ToSwarm enum is a common cause for breaking changes.
This enum is used by plugins (i.e. NetworkBehaviours) to communicate with the Swarm.
Similarly, the FromSwarm enum is used to inform plugins about state changes.
By adding #[non_exhaustive] to these and other enums we enable future additions to be non-breaking changes.

For example, we plan to allow NetworkBehaviours to share addresses of peers they've discovered with each other.
Previously, we had to queue this feature until the next breaking change whereas now, we can simply ship it as soon as it is ready!

Thanks to @dhuseby for getting the ball rolling on this one.
See PR 4581 for details.

Smarter keep-alive algorithm

Prior to this release, a protocol (i.e. ConnectionHandler) had to diligently track streams and other state to correctly compute the KeepAlive for the connection.
A bug in this computation sometimes caused the connection to be shut down, despite it still being in use by the protocol.
Earlier this year, I thought "there is gotta be a better way" and drafted up some ideas.
Eventually @mxinden and myself hashed out the key-points around being smart in regards to keeping connections alive:

By default, any existing stream will keep the connection alive.
Protocols like ping can opt out of this with Stream::ignore_for_keep_alive().
ConnectionHandler::connection_keep_alive only comes into effect once a connection would be shut down.
This allows protocols like Circuit Relay V2 to keep the connection alive despite there not being any streams.

The nice benefit for you as a user is that connections are shut down more aggressivley, minimizing system resource use.
Should you want to keep idle connections alive nonetheless, you can configure an additional delay using libp2p::swarm::Config::with_idle_connection_timeout.

This also makes wiritng your own protocols simpler.

The tracking issue for this effort was #4306 if you wanted to have a closer look.
Special thanks to @leonzchang for landing many small PRs that made this possible!

Be gone `ConnectionHandler::Error`!

Talking about closing of connections, say goodbye to ConnectionHandler::Error and its related components.
It took many months of careful planning, numerous discussions, failed attempts and s-e-v-e-r-a-l refactorings but we finally delete ConnectionHandler::Error with this release.

There's lots to be excited about with this change; let me explain.
Deleting ConnectionHandler::Error also means:

Removing a type-parameter from SwarmEvent.
No more

  SwarmEvent<MyBehaviourEvent, Either<Either<Either<Either<Either<io::Error, Void>, io::Error>, io::Error>, Void>, io::Error>, StreamUpgradeError<Void>>, Void>

Just:

SwarmEvent<MyBehaviourEvent>

Isn't that neat? 🎉

ConnectionHandlerEvent::Close is gone too and together with it, a type parameter on its definition!
FromSwarm also as a type-parameter removed which makes it easier and more concise to reference!
Fewer force-closing of connections.
Prior to this change, a ConnectionHandler could simply close an entire connection at any point, despite the connection still being in use by other protocols.
(Remember that protocols are multiplexed over a single connection.)
For example, an error while making a reservation with a relay would lead to the entire relayed connection being closed, even though other protocols might still have been using it.

This means more reliable connections and protocol executions as no one is pulling the rug out from underneath your protocol's legs as they run across the wire 🏃

`tracing`

Protocols don't actually have legs, Thomas.

That is true.
But if we imagine for a moment that they do, using the latest migration to tracing we can now watch them!

I am not sure if @eserilev knew what they were signing up for when they suggested to tackle #1533 back in August.
A lot of impressive work and pretty much exactly three months later though, we merged #4282 which:

migrates the entire workspace from log to tracing
adds several spans to interesting functions
showcases how to ingest this data into Jaeger

Enough talking, here is a screenshot:

Here we can see what one particular call to Connection::poll actually did.
We can see how much time is spent in the respective identify and ping handlers as well as interaction with the stream multiplexer.

We put some consideration in to which spans might be useful but we are excited to hear from users how useful they are and what else might need instrumentation.

`PollParameters` has been removed

A relict from earlier designs of rust-libp2p is finally gone too: PollParameters.
The NetworkBehaviour::poll function now just takes a &mut Context as you'd expect it to, without any additional clutter.
All features of PollParameters have previously already been deprecated and alternatives were offered in terms of events.

If you enforce no warnings in your codebase, you should be able to just delete that (unused) parameter from the function.

Credits

This release would not have been possible without the contributions of many people.
Thank you to (in alphabetical order):

Closing

As always, a detailed log of changes for every release can be found in our repository: CHANGELOG.md. If you are struggling with an upgrade, feel free to tag Max (@mxinden) or myself (@thomaseizinger) in a PR and we'll try to help.

Happy coding!

This discussion was created from the release libp2p-v0.53.0.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

libp2p-v0.53.0 #4797

{{title}}

Replies: 0 comments

Select a reply

libp2p-v0.53.0 #4797

thomaseizinger Nov 5, 2023 Collaborator

Backwards-compatible features

#[non_exhaustive] on key enums

Smarter keep-alive algorithm

Be gone ConnectionHandler::Error!

tracing

PollParameters has been removed

Credits

Closing

Replies: 0 comments

thomaseizinger
Nov 5, 2023
Collaborator

`#[non_exhaustive]` on key enums

Be gone `ConnectionHandler::Error`!

`tracing`

`PollParameters` has been removed