OKRs - 2019 Q4 - Package Managers #1041
Conversation
Stebalien
requested review from
achingbrain,
andrew,
aschmahmann,
dirkmc,
djdv,
meiqimichelle and
momack2
|
Ref #1033 |
OKR/PACKAGE-MANAGERS.md
Outdated
|
|
||
| **Owner** @aschmahmann/@achingbrain | ||
|
|
||
| Ship a demo of adding NPM to IPFS. |
There was a problem hiding this comment.
other ideas: time to mirror ipfs-wikipedia (a data package made up of many small files), apt-on-ipfs and clojars-on-ipfs will really focus the work here
There was a problem hiding this comment.
I'm fine with whatever demo we decide is most effective. As I mentioned in my IPNS comment above we should figure out what the main "this is cool" factors are aside from it just working on IPFS.
As some of our research docs https://github.com/ipfs/package-managers/tree/master/docs (including https://app.mural.co/t/protocollabs6957/m/protocollabs6957/1557168696127/577c9453a3c51199c8163cf0fe5701294e55f99b) indicate there are a number of things we could focus on. They might be things like faster downloads of packages, or ensuring that specific package versions cannot be deleted. Adding in offline operation might be an interesting thing to demonstrate as well.
There was a problem hiding this comment.
The goal of this initiative is to demo the performance improvements to ipfs add.
OKR/PACKAGE-MANAGERS.md
Outdated
|
|
||
| TODO: define "quickly and efficiently" in terms of beating HTTP. | ||
|
|
||
| TODO: consider breaking this into two KRs? In the past, we had one for efficiency and one for throughput. |
There was a problem hiding this comment.
- IPFS uses at most 10% more bandwidth than HTTP.
- IPFS maxes out a 1GiB link on the internet (100ms latencies).
There was a problem hiding this comment.
I assume we already know this, but was at the ICN conference this past week and a paper was presented (https://conferences.sigcomm.org/acm-icn/2019/proceedings/icn19-34.pdf) that shows (in go-ipfs 0.4.20) a lot of over requesting of data from Bitswap on a small closed network of 10 nodes.
Is the 10% you mention between 2 peers, or even in a multi-peer environment?
There was a problem hiding this comment.
I think steven's comment is the KR goal - not what we currently hit...
OKR/PACKAGE-MANAGERS.md
Outdated
|
|
||
| IPFS can quickly update a large package repository. | ||
|
|
||
| TODO: "quickly" should be defined in terms of normal package update procedures used by package maintainers for some package manager. This could be: |
There was a problem hiding this comment.
OKR/PACKAGE-MANAGERS.md
Outdated
|
|
||
| IPFS can quickly add a large package repository. | ||
|
|
||
| TODO: "quickly" should be defined in terms of adding some well-known package repo or test dataset. |
There was a problem hiding this comment.
|
|
||
| **Not Included** | ||
|
|
||
| * IPNS: To build a fully decentralized package manager, IPNS is a must. However, to get in the door with existing package managers, IPFS just needs to be a viable _transport_. |
There was a problem hiding this comment.
I don't think I buy this. DNSLink is insufficient if users care about 1) Update speed 2) Offline operation or 3) Third party persistence (i.e. not relying on the publisher to continue to maintain their package)
Ditching IPNS here means that any demo we do has to focus on IPFS being more performant/efficient then the current system which means putting all our eggs in the Bitswap sessions basket. Going with the IPNS basics (which at this point is really features like recording saved IPNS records and allowing them to be republished, all API things that do not effect the network) allows us to demo offline usage which is currently not really an option for existing package managers.
While things like Bitswap performance are almost certainly more important for package managers than IPNS support, I think offline usage makes a compelling user story even if we end up having trouble optimizing Bitswap sessions.
There was a problem hiding this comment.
I don't think I buy this. DNSLink is insufficient if users care about 1) Update speed 2) Offline operation or 3) Third party persistence (i.e. not relying on the publisher to continue to maintain their package)
The goal is table-stakes.
- System package managers often have multi-hour-to-day delays between releasing a package and mirrors updating. Nobody's going to notice the DNS propagation time (which is often on the order of minutes).
- Table stakes. Offline operation is great, but we're just trying to be viable.
- Again, table-stakes.
At this point, we're not shooting for a compelling user story. We're trying to make IPFS viable. At the moment.
- Adding can take days:
- Updating isn't possible (without a bunch of custom tooling or re-adding).
- Downloading is extremely slow (it rarely gates faster than a few megabytes per second) and has an insane overhead (at least +50%, usually more like 100%).
Given that and our resource constraints, we have to make a choice. We can't choose implementing a compelling user story over fixing an issue that prevents users from even considering IPFS.
OKR/PACKAGE-MANAGERS.md
Outdated
|
|
||
| **Owner** @aschmahmann/@achingbrain | ||
|
|
||
| Ship a demo of adding NPM to IPFS. |
There was a problem hiding this comment.
I'm fine with whatever demo we decide is most effective. As I mentioned in my IPNS comment above we should figure out what the main "this is cool" factors are aside from it just working on IPFS.
As some of our research docs https://github.com/ipfs/package-managers/tree/master/docs (including https://app.mural.co/t/protocollabs6957/m/protocollabs6957/1557168696127/577c9453a3c51199c8163cf0fe5701294e55f99b) indicate there are a number of things we could focus on. They might be things like faster downloads of packages, or ensuring that specific package versions cannot be deleted. Adding in offline operation might be an interesting thing to demonstrate as well.
|
I'm going to merge this as is because it has a ton of really useful rhetoric outside the OKR sheet |
Rough draft of some OKRs to get us started. I focused on motivation/discussion instead of trying to nail down the exact KRs.
Resourcing: