OKRs - 2019 Q2 JS Core

[alanshaw]

refs #902

• Score 2019 Q1 OKRs
• Async Sync Retrospective
• 2019 Q2 OKRs Open Planning (this thread)
• Move 2019 Q2 OKRs to 2019 Q2 OKRs Spreadsheet

cc @achingbrain @dirkmc @hugomrdias @vasco-santos @jacobheun @daviddias

[olizilla]

@alanshaw It'd be rad to be able to get per CID bandwidth stats so we could show users info on share ratios for things like modules they are co-hosting on npm-on-ipfs... as per: ipfs/kubo#4740

[alanshaw]

Candidate Q2 OKRs and justifications, based on our Q1 review:

• P0: The DHT is landed in JS IPFS and enabled by default in the browser and Node.js

  Owner: @alanshaw

  Without the DHT we have limited ability to find content in the p2p network so any hopes of saving bandwidth or getting content quickly from peers close to us is severely hampered.

---

• P0: Vital missing APIs are implemented /repo/gc, /refs

  Owner: @dirkmc

  If we're going to store data for package managers, it needs to be pinned so that it doesn't get deleted and garbage needs to be removed at appropriate times so heavy MFS usage doesn't eventually fill up the disk.

  The /refs endpoint needs to exist so that JS IPFS can be a preload node for itself. See KR below...

---

• P2: A GC strategy exists to serve the needs of preload nodes

  Owner: @dirkmc

  Currently preload nodes have their content GC'd every hour. This is bad news if you preload your content 1s before GC. A more appropriate GC strategy might be to instead remove CIDs that were added over an hour ago.

---

• P3: JS IPFS replaces the current go-ipfs preload nodes

  Owner: @alanshaw

  As a stepping stone to getting JS IPFS working as a gateway node, it should serve itself in production and be the public preload nodes.

---

• P2: Existing pinning code is refactored to not be a bottleneck when adding files

  Owner: @dirkmc

  This has been proven to be a bottleneck and needs to be resolved for package managers use cases.

---

• P1: ipfs.resolve can resolve IPNS names

  Owner: @hugomrdias

  Currently ipfs.resolve cannot resolve IPNS names - this is required for package managers to figure out the latest version of a registry.

---

• P0: IPNS resolution is assisted by DNS for speed

  Owner: @hugomrdias

  IPNS needs to be fast. The DNS "hack" for IPNS resolution would provide a fast reliable centralized method with a fallback to a decentralized method.

---

• P0: Base32 encoded Peer IDs can be used with IPNS

  Owner: @alanshaw

  Needed to allow mutable content on gateway without security issues.

---

• P0: rsync for IPFS exists to make mirroring package manager data with IPFS work

  Owner: @achingbrain

  Mirroring package manager data to IPFS is impossible right now. FUSE is too slow, filestore cannot be updated, re-import everything takes too long and requires double the disk space.

---

• P0: An experimental UnixFS v2 implementation exists in JS IPFS to allow permissions/metadata be stored with files

  Owner: @achingbrain

  Last modified time and file permissions (executable or not) are needed for mirroring package manager repo data on IPFS.

  Notes: Involves working with @mikeal on Interface design (for js-ipfs) ipld/js-unixfsv2#2.

---

• P1: Core APIs are refactored to use async await/iterators; at least 3/4 of repos in scope have a proposal/PR

  Owner: @alanshaw

  Ongoing endeavour effecting performance, maintainability and DX. Involves the whole JS Core team.

---

• P1: Performance profiles exist for desktop, browser, server, mobile and test

  Owner: @dirkmc

  If we're going to run JS IPFS nodes in production, we need a server profile that configures JS IPFS to operate appropriately in a data centre.

---

• P2: Perf benchmarking site is hooked up to CI

  Owner: @alanshaw

  We need visibility on code changes that have an adverse effect on performance. Similarly when making changes to improve performance we need to be able to measure the gains and ensure performance gains in one area does not effect performance elsewhere.

---

• P0: IPFS Camp: Core course: "Understanding how the InterPlanetary File System deals with Files"

  Owner: @alanshaw

---

• P0: IPFS Camp: Elective course: "TBC"

  Owner: @achingbrain

---

• P3: A repo migration tool exists and JS IPFS can automatically upgrade older repos

  Owner: @dirkmc

  This is necessary to get JS IPFS into production and will be required when migrating the block store to use multihashes instead of CIDs for keys (a component of the CIDv1 base32 work).

  Notes: This KR has already been picked up by a community member so will mostly involve guiding the process. See Create an utility for IPFS Repo migrations js-ipfs#1115 (comment)

---

• P3: Research and create PoC for collecting improved network introspection metrics around size and activity

  Owner: @achingbrain

  Package manager maintainers will still want metrics even when switching to a distributed system.

  Notes: https://github.com/protocol/metrics/issues/8

---

• P3: Port Rabin to JS (or use JS version if now available)

  Owner: @achingbrain

  Advanced chunking capabilities should also be available to browser nodes.

---

I ran out of time on the justifications! Will finish asap.

Would love your thoughts on these - what we should drop, what we've missed, what needs to be re-worded, what needs to be re-prioritised as well as volunteers to champion each KR.

@dirkmc @achingbrain @hugomrdias @vasco-santos @jacobheun @vmx

[hugomrdias]

P1: ipfs.resolve can resolve IPNS names
P0: IPNS resolution is assisted by DNS for speed
These two I'll be doing as part of the WB working group OKRs.

[alanshaw]

I've put your name next to the KRs I think you'd like to own 😜

Remember, owning a KR does not necessarily mean you do all the work!

Can I please get an ACK or a NACK @achingbrain @dirkmc?

[dirkmc]

ACK 👍

[achingbrain]

P0: An experimental UnixFS v2 implementation exists in JS IPFS to allow permissions/metadata be stored with files

I think a quicker way to deliver this would be to implement & extend the v1 metadata construct as v1 it won't be dependent on the UnixFS v2 spec either being finished or stable enough to implement.

Otherwise this all LGTM.

[jimpick]

After talking with @alanshaw, I'm willing to take:

• P0: The DHT is landed in JS IPFS and enabled by default in the browser and Node.js
• P3: JS IPFS replaces the current go-ipfs preload nodes

And I can help out with:

• P0: IPFS Camp: Core course: "Understanding how the InterPlanetary File System deals with Files"

I'm already working on Course C with @lidel and @hugomrdias - "Core Course C - Developing Apps with IPFS. Pick implementation and way of interacting with it"

Perhaps that should be an OKR as well.

Here are the Q2 OKRs I made for DDC (which will be scrapped):

https://docs.google.com/spreadsheets/d/1YSeyWqXh3ImanRrTkYQHHkCofiORn68bYqM_KTLBlsA/edit#gid=27675624

I'd probably keep this one:

• Launch new PeerPad (1 week max)

I think I can do some of the Testlab work as part of the above OKRs. I'll ping @parkan to see what he needs still for collaboration work.

[alanshaw]

Apologies to @jimpick - I misunderstood what was happening with team changes and thought it was happening sooner than it is. OKRs for this quarter persist and we'll be welcoming @jimpick into JS Core with open (OK)aRms next quarter.

[alanshaw]

P0: An experimental UnixFS v2 implementation exists in JS IPFS to allow permissions/metadata be stored with files

I think a quicker way to deliver this would be to implement & extend the v1 metadata construct as v1 it won't be dependent on the UnixFS v2 spec either being finished or stable enough to implement.

I removed "v2" from the wording so you can select the best way to do this ;)

[alanshaw]

@momack2 good to merge?

[momack2]

Seems reasonable. to document current course - Jim is keeping his Q2 OKRs from the DDC sheet re launching peerbase and prototyping integrations (https://docs.google.com/spreadsheets/d/1YSeyWqXh3ImanRrTkYQHHkCofiORn68bYqM_KTLBlsA/edit#gid=27675624), but also going to pitch in on the DHT and preload node KRs. =]