The Deep Dives format is designed to give all participants a taste of what it feels like to work on the big and meaty Open Problems by contributing directly to them in an intense design session - evaluating all the possible solutions, their advantages and shortcomings, and documenting the results.
This type of discussion is core to the genesis of IPFS. We’ve done multiple over the years which have yielded many valuable results such as IPLD, PubSub, GraphSync and other advancements to the original IPFS protocol.
These design sessions are not about explaining how things work, they were focused on finding the solution with the knowledge and resources we had available.
- Objective: Deep knowledge transfer that only happens in synchronous interactions.
- Activity: In a group of max 4 people, dive deep on a complex technical design topic.
- Outcome: The output of each of these sessions would be a RFC like document (can be very raw) that describes the proposed solution (even if with known shortcomings), or it could be a map of all the important concepts, obstacles, and related topics you covered in the session.
- Duration: 1:45h
On the day, this is how y'all will Deep Dive 🐋:
- Large cards with Deep Dive topics will be displayed
- You choose a deep dive to participate in and take a slip from the card
- Groups assemble and the topic owner explains the problem
- Existing solutions should be briefly enumerated!
- Groups discuss their topic
- New ideas are suggested 💭
- Groups aim to come up with one good solution
- It can be a completely new solution or a change/fix to an existing one
- After 1 hour, groups take a 10 minute break (phew) ☕️
- Groups reconvene and send a PR to ipfs/camp with their proposal
- Didn't come up with any solutions?
- List your ideas, and why they weren't suitable so we can bootstrap future discussions
- Didn't come up with any solutions?
- Groups customise their slide for presenting their solution at the graduation ceremony
- Groups are free to iterate on their PR or their slide overnight
The next day (graduation day), this is what happens:
- Each group gets 2 minutes to present a summary of your solution
- You get one slide in the shared slide deck
- The best solution(s) win an award 🏆
| Topic | Owner | Description | |
|---|---|---|---|
| Scaling up the DHT 📦 |  |
@Kubuxu | What can we do to optimise the performance of our DHT? Are there alternatives? |
| IPID - Master key security and user-experience 📦 |  |
@satazor | IDM (Identity Manager) aims to support several DIDs methods and IPID was chosen as the first DID method to be integrated. |
| Package Managers: What Now, What Next? 📦 |  |
@jessicaschilling | Community evaluation and ideation session to recap the current (as of late June) state of the IPFS-enabled package manager landscape, discuss happiness and pain points, and do some ideation/co-creation around the direction of our next rounds of package manager efforts. |
| IPFS on Mobile |  |
@carsonfarmer | Increasingly, everyday computing is happening in mobile environments. At the same time, many features of IPFS and related technologies work best on an 'always-connected' network that simply doesn't exist in the mobile world. In this deep dive, we will explore the issues inherent to p2p systems operating in mobile environments, discuss some existing solutions, and hopefully outline a research/development agenda that will help catalyze a wave of new IPFS-based mobile solutions. |
| Private content on IPFS 📦 |  |
@ianopolous | Can we have a private group chat implemented on IPFS? |
| Building the IPFS Community |  |
@stefanhans | Community, evangelism, engagement, on ramp. |
| IPFS for websites (static, dynamic, search, etc) |  |
@autonome | What are the best practices and open questions for using IPFS for static / dynamic websites? What about indexing and searching? |
| Revocation/Rotating of IPNS Keys |  |
@aschmahmann | The idea of using a public key to identify mutable data is cool, but what happens when that key becomes compromised? |
| How can IPFS support sharding 📦 |  |
@hsanjuan | How to best pin an arbitrary part of a DAG. |
| Connection management |  |
@jacobheun | How can we better manage connections in libp2p to create stable overlays and reduce the impact on connection resource consumption? |
| Replication protocol 📦 |  |
@hannahhoward | How can we more quickly sync a DAG or part of a DAG between two nodes? A compliment to bitswap, affectionately overloaded as GraphSync |
| Do not replicate lists - DNR |  |
@olizilla | How to stop your node from replicating content due to take down notices or other reasons |
| Fast IPNS 📦 |  |
@hugomrdias | How to increase the IPNS resolution speed |
| IPFS Education and Documentation |  |
@meiqimichelle | What are the biggest improvements we can make to documentation for IPFS? |
| Mounting an IPFS filesystem 📦 |  |
@djdv | Is there a way to efficiently mount IPFS as a filesystem? FUSE support is here. What are current limitations, challenges? What needs to happen to enable mounting of MFS? WebDAV is often used in network storage drives, and mainstream operating systems come with native support for mounting WebDAV as drives. It could supersede Writable Gateway and create a surface for interop with apps that provide built-in client-side support for WebDAV. |
| IPNS link rot |  |
@alanshaw | Right now, IPNS needs to be refreshed by original publisher every day, otherwise it stops working. There should be a way to pin IPNS address so that other peers keep refreshing the last valid entry when original node goes offline, keeping the last version alive even if the original publisher disappears forever. |
| Deterministic CIDs! Reproducible File Imports! Verifable HTTP Gateways! |  |
@lidel | How to deterministically get the same CID for the same data, even if default parameters of ipfs.add change? How to validate data read via HTTP Gateway without running full IPFS node? |
| IPFS in Web Workers |  |
@gozala | Where we are, limitations, known problems. |
| UnixFS v2 📦 |  |
@mikeal | Latest spec iteration and implementation work |
| Using CRDTs to build real-time collaborative DApps |  |
@jimpick | What are the hurdles to using CRDTs in dapps with IPFS? How can we make peer-base easier to understand and use? |
| Better NAT traversal so that Relay servers are a last (not first) resort |  |
@raulk | The current cost of running relay servers for IPFS is unsustainable. We need to implement better NAT hole punching mechanisms in libp2p to reduce the usage of relay servers by NAT’d hosts. TCP hole punching has a success rate of ~60%, so we should be preferentially hole punching. TCP simultaneous open is one example. STUN servers are another example; yet another is making every non-NATed host in a libp2p network a relay node. |
| libp2p message orientation |  |
@bigs | libp2p is currently stream-oriented. We want to enable message-oriented communication (e.g. UDP). Help us envision what the APIs would look like, and how an application could simultaneously hold stream-based and message-based conversations with peers in the world via libp2p. |
| IPFS metrics and usage statistics 📦 |  |
@lanzafame | How to collect telemetry like application metrics and usage in a distributed world |
| Decentralized Data Processing |  |
@alari | How could we process large amounts of data in permissionless networks? Use cases, verifiable state transitions, aggregations and bulk updates, etc. |
| Building Dynamic P2P Applications |  |
@jkarni | IPFS and its associated technologies are mostly static. Radicle is an attempt to build a dynamic P2P web on top of IPFS; one which can host applications with the same abilities of the centralized servers of today to receive inputs from various people or sources. |
| Better file chunking 📦 |  |
@mib-kd743naq | Within the IPFS stack/ecosystem, just as within computing as a whole, an uncompressed stream of untagged octets is a fundamental unit of exchange. As a general-purpose data storage system IPFS needs to handle an unbounded variety of content represented by such streams. Handling the maximum amount of this variety efficiently ( ideally by default ) would likely have an outsized impact on the future adoption of IPFS as a long-term data interchange medium/format. |
| Dynamic DAG traversal |  |
@maparent | Extend IPLD to cross between immutable and mutable references |
| Design Patterns for DWeb Apps |  |
@leshokunin | Most IPFS apps have UX issues. We will pick a feature of an existing app, and improve its UX by simplifying how to interact with it, how it works. Textile.io lowered the permission and roles system from ~15 methods to 2 this way! |
| IPFS and Guix 📦 |  |
@ambrevar | Exploration into existing or potential issues with integrating IPFS with Guix |
Note: topics relevant to package managers have a 📦 suffix.
Use this template to create your one-pager description about your topic. Send a PR to https://github.com/ipfs/camp/tree/master/DEEP_DIVES with your topic description page e.g. FAST_IPNS.md
# <Deep Dive Topic NAME>
<Short summary of topic including Open Problem(s) to solve>
## Goal
<Define what we would get if the Open Problem(s) are solved for this Deep Dive>
## Requirements to consider
<Do we have requirements (i.e. work with 1MM nodes)?>
## Where to learn about it
<List of talks, links to github issues, code, notes and so on>