[MOSS] Reporting

[joehand]

Tracking milestones and other deliverables to report for MOSS 2018 grant.

[joehand]

• Updated Dat Project site based on initial community feedback and onboarding of our Mozilla fellow (launch pending)
• Launched early version of dat.land site based on community feedback.
• Started work on Dat Protocol book.
• Starting new developer on-boarding documentation next week

[yoshuawuyts]

Status update on the implementation book:

• The general outline has been set: we start of with a general introduction to concepts, and then a per-module guide with pointers on how to create them.
• The general introduction to concepts is mostly there. We'll need to do a few edit passes, and need review, but otherwise most of the key concepts have been touched upon.
• We started work on the implementation guide, and are going module-by-module. Covering all methods, and ordering in how to approach things. So far only the flat-tree chapter has been merged.

We've also conducted a general survey of existing DEPs. This was useful to use as a reference when writing the docs, but also to point out some of the gaps in the existing deps. We're currently considering a few possible DEPs, but are very much prioritizing the guide first.

[okdistribute]

I wonder if the MOSS funding of the documentation could also cover some user guides that document all of the available options and their implications as part of the existing hypercore node library?

[okdistribute]

@cinnamon-bun started a hackmd for this: https://hackmd.io/AFhXnZ-3Q3KyAWp0ZbYgUg?both#

[joehand]

I wonder if the MOSS funding of the documentation could also cover some user guides that document all of the available options and their implications as part of the existing hypercore node library?

Ya that could be a part of milestone 3!

[ghost]

Here is an update on the documentation I am working on:

• Described how mDNS and centralized DNS discovery work with examples showing what bytes to send over the network.
• Documented how to encode messages used in the wire protocol.
• Documented how the encryption scheme works.

Next I am going to look into replication strategy and show examples of peers exchanging data about a hypercore feed. I am working towards showing how to download or share files using hyperdrive.

So far I have been trying to cover some breadth and am planning to revisit for depth, polish and consistency.

[ghost]

I’m almost halfway through the time I have dedicated to work on documentation. Here’s an update on where I’m heading.

Scope

So far I have covered:

• DNS-based discovery
• Handshakes and the wire protocol
• Encryption
• How data and signatures are structured within a Dat
• How peers exchange data

The two big topics I plan to add in the following weeks are:

• How to verify data integrity
• How files and folders are represented (i.e. Hyperdrive)

And then spend the rest of the time on polish, checking correctness, improvements in response to feedback and consistency.

Currently I don’t think these topics will make the cut, but this will depend on timing:

• On-disk storage format
• Hyperswarm and Bittorrent DHT discovery

Combining with other docs

I’m just starting to reach what is already covered in the Dat Protocol Book, so I’ve been thinking about which of these docs it will make sense to merge into and how it will integrate with datproject.org / datprotocol.com.

I think that there is enough difference between the styles and target readers of these different docs that they will each be able to stand on their own. For example, the docs I am working on have a strong focus on explanation without offering much guidance on implementation and the Dat Protocol Book has a strong focus on implementation and offers guidance on ways to structure code.

I’m still open to later merging my work into another place if people think that will help keep Dat’s docs cohesive, but I’m going to keep working just in an HTML file for now and we can figure out where a good place to host it will be.

Naming

Finally on naming, I saw that @joehand described what I am working on as “How Dat works”. I really like the name and think it lines up with what I’m trying to do so I’ll rename it to that soon!

(edit: I’ve done the rename now and updated the links above to point to the new location)

[ghost]

Progress update on How Dat Works

Since last time I’ve added the two sections on data integrity and files/folders, as well as started working on consistency and polish.

I’ve got two weeks left of the time I have dedicated to working on these docs, so I’m planning and working towards a v1 Release. The idea is to generate a bit of excitement and enthusiasm about releasing these new docs. The date I have planned is 22 Jan 2019, two weeks away.

Here is my list of things I’ll be working on to get ready for the release. Mostly it’s polish and improving the quality of the docs with a few gaps to fill in about the protocol.

My plan for the release itself is:

• Publish a post on the Dat Project Blog (if that’s good with @joehand and @daniellecrobinson).
• Get help from the Dat community to spread the word, read the guide and send feedback. I’ll make some cover art for those who want to post with the link.

One goal I have for the release is to gather feedback from new readers and find some people willing to do a readthrough with me. When I’ve done this kind of user testing in the past it has been very time intensive but also hugely improved the quality. So I think it’s worth it but not something that I’ll be able to make happen before v1.

There’s still lots that could be improved with the guide beyond the fixes that I’ll be spending the next two weeks on, but I think this will be a good start.

[joehand]

@yoshuawuyts can you give a quick update on the DEPs you reviewed, authored, or contributed to in some way for this work? Want to make sure to get that in our report.

[yoshuawuyts]

@joehand Sure thing!

I've read and reviewed all DEPs, and pre-draft DEPS. Not all (pre-)drafts seemed to be in a state where feedback would be useful, so I refrained from providing feedback on some of them. The following changes were landed on the DEPs repository:

• Patched DEP 000 "Template" - fix typo in dep 000 DEPs#55
• Patched DEP 001 "Process" - fix typos in 0001-dep-process DEPs#47
• Patched DEP 004 "HyperDB" - Fix typos in 004-hyperdb DEPs#48
• Reviewed DEP 004 "HyperDB" patch - Update hyperdb (and multiwriter) for deleted flag and HypercoreProtocol header DEPs#50
• Authored DEP xxx "SLEEP Headers" (approval pending) - Sleep Headers DEP DEPs#56

[ghost]

How Dat Works published!

Blog post here: How Dat Works: New documentation for the Dat protocol

My plans for the next while are mostly listening to feedback and working in any easy/high impact changes and fixing any mistakes that I find.

Special thanks to @joehand for organizing this, kicking it off, helping me out and making it a fun process!

[todrobbins]

@vtduncan it turned out so well! Thank you!

[joehand]

I've published an updated docs.datproject.org that includes:

• Internationalization support
• Easier contributions
• Better mobile support
• Well-supported documentation website software

This closes out milestone 3!

[okdistribute]

@whoisgina is creating surveys and doing research on what the next iteration of our public materials should be