Epic: Concurrent updates may overwrite each other

[expede]

Reproduced By

• Brooke
• Boris
• Philipp
• James
• Brian
• Daniel
• Steven

Environment

All of them

Ruled Out

• IPFS Cluster
• DNS
• Application code
• Web API
• SDK

Root Cause

Updates may not know about changes from concurrent updates.

Solution

We need both of the following, though the SDK is prior in the critical path:

Web API

Web API should perform structural validation on updates, and reject updates that drop files. This will then be passed back as a 422 to the client along with the current CID.

SDK

SDK needs to be able to do a merge operation on its changes versus a new WNFS HEAD. When the server rejects an update, get the new CID and rebase the local changes on this. This will change the content of each diff, since at minimum you must bring any new files along. Atomic actions (coming) improve on this further by allowing arbitrary sub-file reconciliation.

[expede]

SDK: oddsdk/ts-odd#163

[matheus23]

Web API should perform structural validation on updates, and reject updates that drop files. This will then be passed back as a 422 to the client along with the current CID.

Small question: How would the API for a destructive update (i.e. "OVERWRITE" capability) then look like?
Will the server check whether the UCAN has overwrite capabilities? I think the client should be able to choose whether the server should reject changes that overwrite.

[matheus23]

Atomic actions (coming) improve on this further by allowing arbitrary sub-file reconciliation.

🤩 🤩 🤩 🤩 🤩

sorry :D

[expede]

Small question: How would the API for a destructive update (i.e. "OVERWRITE" capability) then look like?

It depends on what you mean by "destructive". Since WNFS is nondestructive by default, removing a file doesn't actually get rid of it, since it's still in history and can be recovered. So a history-preserving deletion is an APPEND. To actually delete something, you need to rewrite history, so OVERWRITE is higher permission required.

If we're finding these specific words are confusing now that we have actual experience with this system, we could move to more WNFS-specific terms like REWRITE_HISTORY (or RETCON 😉).