Skip to content

Commit

Permalink
formatting
Browse files Browse the repository at this point in the history
  • Loading branch information
daviddias committed Dec 17, 2018
1 parent 5936940 commit bc48a4b
Showing 1 changed file with 46 additions and 38 deletions.
84 changes: 46 additions & 38 deletions WG_GO_CORE.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# IPFS Go Core Dev Team - Roadmap 2019

The Go IPFS Working Group is responsible for the development of the Go IPFS implementation and participating in the planning and development of IPFS in general.

### Responsibilities include:
Expand Down Expand Up @@ -28,133 +29,141 @@ The Go IPFS Working Group is responsible for the development of the Go IPFS impl
## Milestones

### 📦 Package managers
* M (P0): The IPFS core team understands the needs of package management communities and engages their interest

* `M (P0)`: The IPFS core team understands the needs of package management communities and engages their interest
* Meet people from various package management communities -- both distros and language tooling -- and learn about their needs
* Support interested package manager communities with DRIs
* F-droid
* Guix
* Snapshots.debian.org ?
* M (P0): We have understanding and documentation of what kind of scale (in terms of size at one time; and change in size over time) mirroring a package manager archive requires.
* `M (P0)`: We have understanding and documentation of what kind of scale (in terms of size at one time; and change in size over time) mirroring a package manager archive requires.
* Ex: document how long mirroring npm into IPFS takes
* Ex: VictorBjelkholm/arch-mirror also exists, let’s make sure we gather stats
* M (P1): Develop documentation for what Package Distribution with IPFS should look like.
* `M (P1)`: Develop documentation for what Package Distribution with IPFS should look like.
* There is documentation and guidelines to help maintainers from existing package management communities learn about IPFS, explore common alternatives, and figure out what would work for their system
* Document and highlight best practices for dealing with the common hurdles in moving to immutable-first: e.g. “save packages in a merkle tree; build a package index merkle tree; use IPNS to handle updates of the tip of package index tree”.
* M (P1): There is a way to prioritize different peers to optimize network utilization
* `M (P1)`: There is a way to prioritize different peers to optimize network utilization
* Bitswap peer prioritization supports pinning local / responsive peers


### 🗂 Large Files

Go-IPFS is fast, efficient and stable when handling and transferring large amounts of data. (“large files” = 10GB - 1TB, “many small files” = 10K-1M 1MB files). The goals in this section are exclusive of finding the data (DHT/providers etc.). This is just the speed of transferring data when already connected to IPFS nodes that have that data.
* M (P0): Awesome go-ipfs benchmark test suite exists comparing ipfs performance and transfer size relative to bittorrent, rsync, http, cp, dd
* M (P0): Go-IPFS bitswap (or equivalent) can transfer large files from many peers at 0.8 times the speed of BitTorrent.

* `M (P0)`: Awesome go-ipfs benchmark test suite exists comparing ipfs performance and transfer size relative to bittorrent, rsync, http, cp, dd
* `M (P0)`: Go-IPFS bitswap (or equivalent) can transfer large files from many peers at 0.8 times the speed of BitTorrent.
* Bitswap session improvements that improve transfer efficiency by reducing duplicate blocks and efficiently fetching related data from many peers simultaneously.
* Graphsync selector implementation that can understand non-overlapping subsets of data to select from multiple hosts and the driver that can build up graphsync requests for many peers to maximize efficiency and throughput.
* M (P0): Go-IPFS can transfer many small files at 0.8 times the speed of rsync (for single peers) or BitTorrent (for many peers)
* `M (P0)`: Go-IPFS can transfer many small files at 0.8 times the speed of rsync (for single peers) or BitTorrent (for many peers)
* Bitswap session improvements
* Graph Exchange that can route requests efficiently
* Graphsync
* Flexible provider strategies
* M (P1): Go-IPFS can transfer large files from a single peer at 0.8 times the speed of HTTP.
* `M (P1)`: Go-IPFS can transfer large files from a single peer at 0.8 times the speed of HTTP.
* Flexible provider strategies to use the DHT efficiently (and therefore not slow IPFS down while adding/fetching files) while still preserving the reachability of data.
* M (P1): Go-IPFS can add a directory of many small files at 50% the speed of a simple local hard drive copy (or dd) (10k x 1MB files)
* `M (P1)`: Go-IPFS can add a directory of many small files at 50% the speed of a simple local hard drive copy (or dd) (10k x 1MB files)
* Flexible provider strategies to limit the number of DHT provider advertisements we make
* Faster datastore (Badger or equivalent) that doesn’t slow down as the number of blocks stored grows large
* GC improvements (transactions, speed)
* M (P2): Data transferred over the network is no more than 1.2x the size of the data being transferred
* `M (P2)`: Data transferred over the network is no more than 1.2x the size of the data being transferred
* Bitswap session improvements to reduce duplicates
* Graphsync


### 🔄 Decentralized Web

go-ipfs is the go-to backend for building DApps.
* M (P0): Provider lookups are efficient (<1 sec) in real-world network conditions (ex between physically distant peers)
* * M (P0): ipfs://, ipns:// work in web browsers

* `M (P0)`: Provider lookups are efficient (<1 sec) in real-world network conditions (ex between physically distant peers)
* `M (P0)`: ipfs://, ipns:// work in web browsers
* Base32 CIDs by default
* Base32 IPNS keys (or make IPNS keys CIDs)
* M (PX): <1 sec IPNS resolution for any IPNS record
* `M (PX)`: <1 sec IPNS resolution for any IPNS record
* QUIC
* NAT detection/traversal
* Only "reachable" nodes join the DHT
* M (P1): IPFS provides a clean way to persist data (dapps need to store stuff)
* M (PX): IPFS supports basic dapp requirements for security and private content
* `M (P1)`: IPFS provides a clean way to persist data (dapps need to store stuff)
* `M (PX)`: IPFS supports basic dapp requirements for security and private content
* dapps are secure (api security, multi tenancy). Apps can't read each other's state
* There are examples and documentation for ipfs users to create private-content dapps using IPFS (aka handling encryption on the client side)
* M (PX): IPFS is usable for building wordpress-level decentralized sites
* `M (PX)`: IPFS is usable for building wordpress-level decentralized sites
* API doesn't suck (and is fast)
* We can build an interactive blogging engine with a built-in editor/publisher entirely in a webapp (backed by a go-ipfs instance). And it's *easy* to do this.

### 🤝 IPFS Contributors and Developers
Developers using IPFS can rely on go-ipfs as a platform on which to build their product, app, or ecosystem, and new contributors are well supported
* M (P0): (all) Go-IPFS is approachable as a new contributor

Developers using IPFS can rely on go-ipfs as a platform on which to build their product, app, or ecosystem, and new contributors are well supported.

* `M (P0)`: (all) Go-IPFS is approachable as a new contributor
* Go-IPFS internals are well documented
* Technical debt is paid off, all "indefinitely-in-progress" refactors are completed.
* Cleanup/rethink abstraction layers with 20-20 hindsight.
* Re-structure repos into reusable but not fragmented components.
* M (P0): (all) Go-IPFS is usable without gx
* `M (P0)`: (all) Go-IPFS is usable without gx
* Fix the versions (make them all sub-0)
* Add go.mod files to every package
* Merge #5435.
* M (P0): (all) Users can reliably transfer data between any two nodes.
* `M (P0)`: (all) Users can reliably transfer data between any two nodes.
* Reliable NAT traversal (AutoNAT, relay, TURN, etc).
* Connection manager doesn't kill useful connections.
* Scalable content routing
* Reliable DHT: unreliable nodes don't join the DHT.
* M (P0): (dapps) ipfs:// and ipns:// work in web browsers
* `M (P0)`: (dapps) ipfs:// and ipns:// work in web browsers
* Base32 CIDs
* Base32 IPNS or IPNS uses CIDs
* M (P1): (all) Go-IPFS is approachable as a user/app developer
* `M (P1)`: (all) Go-IPFS is approachable as a user/app developer
* User documentation.
* Well designed API interface
* Well designed API transport
* M (P1): (Dapps) Fast (< 3s), mutable name resolution (IPNS)
* `M (P1)`: (Dapps) Fast (< 3s), mutable name resolution (IPNS)
* Reliable DHT.
* QUIC (for fast connection establishment).
* Better protocol negotiation (multistream-2.0)
* Delegated Routing
* M (P1): (Dapps) IPFS Realtime Story is as good as using a centralized service (e.g. Socket.io, Pusher, etc)
* `M (P1)`: (Dapps) IPFS Realtime Story is as good as using a centralized service (e.g. Socket.io, Pusher, etc)
* The API for doing secure (authenticated) broadcast updates exists
* M (P2): Go-ipfs can handle large datasets (>1TiB, >1M nodes)
* `M (P2)`: Go-ipfs can handle large datasets (>1TiB, >1M nodes)
* Scalable content routing (providing)
* DagSync (or at least better bitswap)
* Reliable, performant datastore
* M (P2): (all, Dapps) IPFS is pluggable and extensible (9/10 users don't need custom features)
* `M (P2)`: (all, Dapps) IPFS is pluggable and extensible (9/10 users don't need custom features)
* The go-ipfs plugins system is expanded to support new datastores, exchanges, etc
* Exchanges support multiple protocols (that diagram from hack week)
* Multi-DHT (https://github.com/ipfs/notes/issues/291#issuecomment-414495124)
* UnixFS-V2 is implemented
* M (P2): (machine-learning/package managers) IPFS can handle (and transfer) large (>1M entries) sharded indexes (objects, directories)
* `M (P2)`: (machine-learning/package managers) IPFS can handle (and transfer) large (>1M entries) sharded indexes (objects, directories)
* Bitswap Improvements (sessions, prediction, etc)
* UnixFS-V2 (better directory structure)
* M (P3): (Dapps) IPFS can locally share data without a shared network
* `M (P3)`: (Dapps) IPFS can locally share data without a shared network
* A bluetooth (or like) transport
* M (P4): (anti-censorship) Our IPFS implementations are secure and don't leak sensitive information
* `M (P4)`: (anti-censorship) Our IPFS implementations are secure and don't leak sensitive information
* Get a security audit
* Implement the privacy preserving DHT (https://github.com/ipfs/notes/issues/291#issuecomment-396003860)
* Add a download-only mode (avoid serving local data)
* Investigate a privacy-preserving transport

### 🧠 Strategic goals
* M (P0): We have a stable core that people can add things to without breaking changes

* `M (P0)`: We have a stable core that people can add things to without breaking changes
* UnixFS is stable and extensible
* UnixFS v2 (with a good HAMT)
* The DHT is extensible
* Multi-DHT: https://github.com/ipfs/notes/issues/291#issuecomment-414495124
* M (P0): Have a solution for testing/benchmarking
* `M (P0)`: Have a solution for testing/benchmarking
* Available for offline testing.
* Reproducibility (across different platforms).
* Provide realistic Internet/WAN simulations (with delays and dropped packets).
* M (P0): Support allow- and deny-lists for content
* `M (P0)`: Support allow- and deny-lists for content
* Required for exposing data by libraries and archives (allowlist)
* Required for ease of Copyright compliance (denylist)
* M (P1): Support read only gateways (not-offline)
* `M (P1)`: Support read only gateways (not-offline)
* Might be more fitting for partners
* M (P1): Interfaces are future-proof enough for 1.0
* `M (P1)`: Interfaces are future-proof enough for 1.0
* CoreAPI
* Plugins
* M (P1): Grow our own team processes / active contributors to add more project coordination support
* M (P2): Investigate IPNS that would continue working offline/in small network segment (for future goals)
* M (P2): Certificate based private networks
* `M (P1)`: Grow our own team processes / active contributors to add more project coordination support
* `M (P2)`: Investigate IPNS that would continue working offline/in small network segment (for future goals)
* `M (P2)`: Certificate based private networks

## ⏳ Timeline

Expand All @@ -163,7 +172,6 @@ Developers using IPFS can rely on go-ipfs as a platform on which to build their
- Q3
- Q4


## ⁉️ Want to get involved?
- File an issue here with questions, or make a PR to suggest an alternate path or addition
- Want to help make this happen? [Open an issue on the Go IPFS Working Group repo!](https://github.com/ipfs/go-ipfs/issues)

0 comments on commit bc48a4b

Please sign in to comment.