Skip to content
This repository has been archived by the owner on Dec 27, 2022. It is now read-only.

Commit

Permalink
Dat networking: Enable read-verification during replication.
Browse files Browse the repository at this point in the history
We uncovered a bad assumption during debugging. Hyperdrive allows
configurable external storage, and its up to the storage to
decide what data to keep.

In Beaker, currently, no historical versions of files are retained.
This is fine, except that the internal tracking in Hyperdrive was
not aware that old versions of files were lost. As a result, the
browser would wrongly tell peers that it had old versions of files,
then (even worse) send the new data when old versions were
requested, causing the replication stream to fail.

The solution here is to run checksums on all data which is read
from external storage during replication. If verification fails,
then hyperdrive will update its internal tracking and notify the
peer that, whoops, we dont have that chunk, using the 'unhave'
message.

In the future, we'll want to actively update the local tracking
during writes, to avoid this problem. However, since it's not yet
clear what the historical data storage patterns will be in the
future, we've put that off for now.
  • Loading branch information
pfrazee committed Jan 17, 2017
1 parent 6bcd28c commit 1a8b345
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions app/background-process/networks/dat/dat.js
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,7 @@ export function loadArchive (key, { noSwarm } = {}) {
var archive = drive.createArchive(key, {
live: true,
sparse: true,
verifyReplicationReads: true,
file: name => raf(path.join(archivesDb.getArchiveFilesPath(archive), name))
})
archive.userSettings = null // will be set by `configureArchive` if at all
Expand Down

0 comments on commit 1a8b345

Please sign in to comment.