Roadmap.md: Discuss the near term future of rust-libp2p

[mxinden]

Here is my (@mxinden) proposal for a roadmap for the next 6 months for the rust-libp2p maintainers (@elenaf9, @thomaseizinger, @jxs and @mxinden) working on the rust-libp2p project.

(rust-)libp2p is an open source and open community project. Thus:

• I want to make sure we listen to the community and align our work accordingly. I invite everyone to leave feedback here.
• The below is only planning work for the maintainers, not the community. Folks from the community may suggest and pick up work not prioritized (e.g. Kademlia efficient querying) nor mentioned on this roadmap and we (maintainers) should be supporting them as best as we can.

I consider this more of a short-lived document, enabling everyone working and using rust-libp2p to discuss and eventually align on a common mission, making prioritization explicit. Once done, it will likely be outdated quickly and thus I suggest never actualy merge it. In my eyes, continued project management is better done via GitHub issues and thus all the information here should eventually make it into the corresponding GitHub issues if not already the case.

Also see:

• libp2p roadmap
• go-libp2p roadmap

The items in this roadmap are ordered by importance where I define importance as:

• Enable others to build with rust-libp2p before building components ourself.
  • e.g. cross behaviour communication over Kademlia client mode where the former enables the latter
• Invest into tools that enable us to work more efficiently
  • e.g. Testground and release process]
• Better do one thing well than many things close to right.
  • e.g. deprioritize BitSwap implementation and continue on[QUIC even once first ieration is merged. That does not imply that we won't support community members implementing it.
• Improve existing components before introducing new ones.
  • e.g. Kademlia client mode before WebTransport

I am curious what folks think and again encourage everyone to share their thoughts.

[p-shahi]

This is awesome, I just gave some drive by comments.

[thomaseizinger]

All of these are exciting!

[thomaseizinger]

I consider this more of a short-lived document, enabling everyone working and using rust-libp2p to discuss and eventually align on a common mission, making prioritization explicit. Once done, it will likely be outdated quickly and thus I suggest never actualy merge it.

Happy with that!

[BigLep]

Thanks for putting this together @mxinden . I know @p-shahi will engage more here, but wanted to give a couple of comments.

I consider this more of a short-lived document, enabling everyone working and using rust-libp2p to discuss and eventually align on a common mission, making prioritization explicit. Once done, it will likely be outdated quickly and thus I suggest never actualy merge it.

It's ok if this document specifically is short-lived, but I think we need a document/table somewhere that communicates to stakeholders the major initiatives of the project and when we estimate completing them as described in https://docs.google.com/document/d/1XJiFG9tTCDi4RsKEAivHgcXueIqqwhkJDE-OKvy8yec/edit# . This "stakeholder roamdap" will be higher-level than the week-to-week plans that the engineering team is using to drive their execution. We see this with projects like FVM where they have https://fvm.filecoin.io/#roadmap-4 but then also fine grained detailed planning to hit those milestones.

The hat I would wear is: imagine you're someone like Juan or Molly who has been told they have to share tomorrow about what rust-libp2p is up to and what users can expect in the next 6 months or if they are trying to summarize the efforts across multiple EngRes teams. You'd want the higher-level items that really matter to collaborators and users.

If this document is the best we have currently (which is totally fine), I do think we should get it checked in but:

1. We can communicate the document's status (actively being reviewed/updated in October)
2. We make it clear how someone leaves comments or tracks the status of additional updates. I presume we have a tracking "2023 roadmap issue" that we link to from this document.

In my eyes, continued project management is better done via GitHub issues and thus all the information here should eventually make it into the corresponding GitHub issues if not already the case.

I agree, but have a few comments:

1. Agreed GitHub issues are far superior for linking between other issues/prs, checklists, labeling, commenting etc. I think each milestone item below having a tracking makes sens.e. I see you have that in some cases. (That said, I would make sure to link to rust-libp2p items which can then link out to spec issues).
2. We shouldn't require someone to go spelunking through github issues to get a sense of the high-level roadmap. I do think it's possible to have a world where you have all the upcoming epics on a clean board for someone to get the stakeholder view, but that will take setup. Until that happens, I think going with a document like you have now is probably the easiest way to go. Given the "stakeholder" roadmap is only a handful of milestones, I don't think it's a lot of duplicated work to have that be it's own document/presentation which is separate for where all the engineering work is happening day-to-day in GitHub issues.

In summary, until we have something alternative/better that communicates the direction of the project, I do think we should be driving to get this merged in.

[MarcoPolo]

I'd love to see benchmarking show up here because:

1. The sooner we do this, the sooner we can quantify the performance and make regressions harder.
2. Perception (even if unfounded) of a slow networking library pose an existential risk to libp2p.
3. It lets us highlight QUIC improvements in a concrete manner.
4. Signal to users that we are committed to a performant library. This is the "zero cost abstraction" of networking libraries (alright maybe not zero cost, but close enough).

[mxinden]

Thanks for the input everyone.

• Make this a high level document for stakeholders.
• Track detailed work in project board https://github.com/orgs/libp2p/projects/4/views/1
• Fine with it being merged into master.
• Still need to address benchmark item raised by @MarcoPolo.

[mxinden]

@dignifiedquire, @melekes, @kpp, @AgeManning, @divagant-martian, @ackintosh, @ everyone:

Want to give this a review? Anything you think is missing? Anything you would like to see prioritized?

[melekes]

👍

[thomaseizinger]

I think this is a good roadmap!

My suggestion would be to present it slightly differently. For some features (TLS, QUIC and WebRTC) it is pretty clear to outside users and stakeholders, what the outcome is.

For some of the other items, that intention is missing. I would suggest to introduce the following high-level items:

• Performance optimisations
• Developer ergonomics

I think those are goals that are easier to relate to. It is also easier to say: "In Q4, rust-libp2p will focus on TLS, QUIC, WebRTC and developer ergonomics".

On issue with this approach is that "developer ergonomics" is never complete but I don't think it needs to be. It can be on the roadmap for every quarter and just the concrete goals underneath get completed.
I think this also ties in well with @MarcoPolo's comment about benchmarks. Those can go under "Performance optimisations" as an initial starting point.

Last but not least, what is missing for me here is a concrete call-out to spec-compliance. It is easy to say that this should be implicit (i.e. all implementations always being compliant). However, if we don't give it any attention, it is easy to overlook. I'd love to have an epic for spec-compliance where we can f.e. group all the testground work in but also things like libp2p/specs#466.

[thomaseizinger]

I think this is an important item that we should work on because it will be a nice ergonomic improvement for our users. At the same time though, it is only just that: An ergonomic improvement. A user can integrate identify and kademlia today, they just need to write a bit of code for it.

With that in mind, I think we should remove this item from this roadmap.

[mxinden]

I would like to keep the paragraph here to stress that generic cross-behaviour-communication is not on the roadmap. Otherwise folks might confuse the scope of this roadmap item.

[thomaseizinger]

I am talking about the entire roadmap item, not just the generic approach :)

[mxinden]

I think we should prioritize it. In my eyes having this feature significantly reduces the amount of complexity forced onto new users.

[thomaseizinger]

I think these could go under a heading "Improving runtime efficiency".

[mxinden]

I am confused. Why is Kademlia a runtime or part of a runtime? Or as in "at run time"?

[thomaseizinger]

The latter. A better term is probably "Performance improvements".

[thomaseizinger]

My thoughts on this one are similar to cross-behaviour communication but it is a little less clear because you need to work around the fact that connections get established first and a behaviour would have to instantly kill it again. This might get the job done but is a bit hacky.

I am inclined to say that this should either also be removed or grouped under a "Developer ergonomics" heading.

[mxinden]

This might get the job done but is a bit hacky.

As an aside, that is how https://github.com/sigp/lighthouse and https://github.com/paritytech/substrate do it today.

I am inclined to say that this should either also be removed or grouped under a "Developer ergonomics" heading.

In my eyes, this goes beyond ergonomics. I.e. it enables users to implement a clean eclipse defense.

[thomaseizinger]

Perhaps it should go under a "Security" heading then?

[thomaseizinger]

My way of looking at this is that we should make it easier for users to write NetworkBehaviours instead of giving them an escape hatch, hence I think we should remove this.

What we could do is add an item to the "Developer Ergonomics" heading (if you end up introducing it) that briefly mentions "Implementations of custom protocols".

[mxinden]

My way of looking at this is that we should make it easier for users to write NetworkBehaviours instead of giving them an escape hatch, hence I think we should remove this.

I am still undecided here. I think we need to do both. I fear that we will never be able to make NetworkBehaviour (a) easy enough for a newcomer to implement a proof-of-concept, while (b) still supporting advanced protocols like Kademlia, Gossipsub and BitSwap.

I suggest we continue experimenting with both. I don't have an opinion on which one to prioritize.

[thomaseizinger]

This could be grouped into "performance optimisations" as well.

[mxinden]

Thanks for the input @thomaseizinger!

On the matter of grouping items, I decided against it, but instead introduced categories for each item.

Reason for not grouping items is that thus the prioritization across groups gets lost. E.g. while WebRTC and WebTransport are in the Connectivity category, one is high priority, the other is low priority, with many items from other groups in between.

Let me know if that works for you.

Last but not least, what is missing for me here is a concrete call-out to spec-compliance. It is easy to say that this should be implicit (i.e. all implementations always being compliant). However, if we don't give it any attention, it is easy to overlook. I'd love to have an epic for spec-compliance where we can f.e. group all the testground work in but also things like libp2p/specs#466.

I don't have a strong opinion. We removed the Testground item from this roadmap as it is not user facing. In the same spirit I suggest not tracking specification compliance here either. How about tracking it on https://github.com/orgs/libp2p/projects/4 instead?

That is not to say that specification compliance is not a high priority.

On issue with this approach is that "developer ergonomics" is never complete but I don't think it needs to be. It can be on the roadmap for every quarter and just the concrete goals underneath get completed.
I think this also ties in well with @MarcoPolo's comment about benchmarks. Those can go under "Performance optimisations" as an initial starting point.

As above, I suggest not tracking benchmarking on this roadmap given that it is (arguably) not user facing.

On this matter, I would prioritize it below the testing effort along the lines of: "make it work, make it correct, make it fast".

[efrisch]

@mxinden There's a few typos in this sentence. Is this saying QUIC will continue to have higher priority than the BitSwap implementation once QUIC is merged?

• e.g. deprioritize BitSwap implementation and continue on[QUIC even once first ieration is merged. That does not imply that we won't support community members implementing it.

[divagant-martian]

While this looks fairly ambitious to me, it's really great! Looking forward to it

[thomaseizinger]

Thanks for the input @thomaseizinger!

On the matter of grouping items, I decided against it, but instead introduced categories for each item.

Reason for not grouping items is that thus the prioritization across groups gets lost. E.g. while WebRTC and WebTransport are in the Connectivity category, one is high priority, the other is low priority, with many items from other groups in between.

Let me know if that works for you.

Yeah the priority is a bit tricky. I think categories work. I guess it depends on what you think is more important to present to stake-holders:

• Priority of concrete items: Use that as the sort-order and have categories as tags.
• Broad categorization of work: List the categories and stick a priority label on each item.

I don't mind either way. Without asking some of the non-technical stakeholders, we are unlikely to know what they'd prefer.

Last but not least, what is missing for me here is a concrete call-out to spec-compliance. It is easy to say that this should be implicit (i.e. all implementations always being compliant). However, if we don't give it any attention, it is easy to overlook. I'd love to have an epic for spec-compliance where we can f.e. group all the testground work in but also things like libp2p/specs#466.

I don't have a strong opinion. We removed the Testground item from this roadmap as it is not user facing. In the same spirit I suggest not tracking specification compliance here either. How about tracking it on github.com/orgs/libp2p/projects/4 instead?

I mostly mentioned it because it may be worth giving visibility to this to other stakeholders. I am certain that we will be giving it an appropriate amount of attention :)

On issue with this approach is that "developer ergonomics" is never complete but I don't think it needs to be. It can be on the roadmap for every quarter and just the concrete goals underneath get completed.
I think this also ties in well with @MarcoPolo's comment about benchmarks. Those can go under "Performance optimisations" as an initial starting point.

As above, I suggest not tracking benchmarking on this roadmap given that it is (arguably) not user facing.

On this matter, I would prioritize it below the testing effort along the lines of: "make it work, make it correct, make it fast".

👍

[mxinden]

@mxinden There's a few typos in this sentence. Is this saying QUIC will continue to have higher priority than the BitSwap implementation once QUIC is merged?

• e.g. deprioritize BitSwap implementation and continue on[QUIC even once first ieration is merged. That does not imply that we won't support community members implementing it.

Yes @efrisch. In my eyes we should prioritize fixing and improving existing functionality (QUIC once merged) before new functionality (BitSwap).

[mxinden]

Thanks for all the input here.

Looking forward to tackling these with all of you. Exciting times ahead.

I am merging here.

Feel free to post additional comments here or create pull requests against the file. This is a living document owned by all of us.

[mxinden]

Semver check failure is discussed in #2647, Testground failure is tracked in libp2p/test-plans#59.

[BigLep]

@mxinden : arg! I see that I never submitted a review I did on Tuesday. Arg - my bad. I'm submitting it now.

[BigLep]

A couple of things:

1. That project board is private
2. Maybe rename it to "rust-libp2p Detailed Roadmap"
3. Add a readme https://github.com/orgs/libp2p/projects/4/settings ? It can have a note pointing back to this document.

[BigLep]

@mxinden : I took care of 2 and 3. I can't do 3 (I think because I don't have sufficient permissions.)

[BigLep]

Additional things to potentially cover:

1. Link to the specs vision roadmap (which should ideally cover core tenets of the project like security, stability, performance, and specs first). I suggested to @p-shahi in roadmap: reorder priority, reorganize sections go-libp2p#1831 (comment) to factor some of the tenets there so we can link to them from the various libp2p implementation roadmaps.
2. How does someone leave feedback or comments. go-libp2p and testground went with a pinned tracking issue with a comment at the top: https://github.com/libp2p/go-libp2p/blob/master/ROADMAP.md#go-libp2p-roadmap-q422q123
3. I think we need to add notes about how interoperability and performance testing are critical initiatives for the project and in many cases take higher priority than other initiatives listed here. go-libp2p has some notes on this here: roadmap: reorder priority, reorganize sections go-libp2p#1831

[BigLep]

@mxinden : I'd like to get a callout about us focusing on interoperability testing right now. go-libp2p has this here: https://github.com/libp2p/go-libp2p/blob/master/ROADMAP.md#benchmarking-and-testing

[BigLep]

We can do the work here: #3091

[BigLep]

Suggested change

	communicate with each other, i.e. can not make use of information known by another
	communicate with each other, i.e. cannot make use of information known by another

[mxinden]

For the record, follow ups to @BigLep's review above is happening in #3062.