P2P data sync may deadlock on error #470
Assignees
Milestone
Comments
jsimnz
added
bug
|
@jsimnz is this something we need to fix ourselves or something that ipfs should take care of? |
Unfortunately it's on us :'(. Technically there's a protocol that might help called GraphSync being worked on so Protocol Labs, but it's still a ways from being ready for prime time. |
4 tasks
shahzadlone
pushed a commit
that referenced
this issue
Apr 13, 2023
Relevant issue(s) Resolves #1028 Resolves #470 Resolves #1053 Description The main purpose of this PR is to resolve the potential deadlock. The deadlock can happen if a PushLog cycle doesn't receive the whole missing block history before an error occurs. This leaves the DAG incomplete but there is no clear way, at the moment, for Defra to be aware of that and for it to try to fill in the gap at some point. To solve the deadlock situation, John and I had discussed implementing a transaction that would cover the whole PushLog cycle. This means that any error occurring during the cycle will discard the transaction and thus leave the DAG unaffected. As side effects, we add thread safety to the badger transactions and we manage DAG workers on a per PushLog cycle basis.
shahzadlone
pushed a commit
to shahzadlone/defradb
that referenced
this issue
Feb 23, 2024
Relevant issue(s) Resolves sourcenetwork#1028 Resolves sourcenetwork#470 Resolves sourcenetwork#1053 Description The main purpose of this PR is to resolve the potential deadlock. The deadlock can happen if a PushLog cycle doesn't receive the whole missing block history before an error occurs. This leaves the DAG incomplete but there is no clear way, at the moment, for Defra to be aware of that and for it to try to fill in the gap at some point. To solve the deadlock situation, John and I had discussed implementing a transaction that would cover the whole PushLog cycle. This means that any error occurring during the cycle will discard the transaction and thus leave the DAG unaffected. As side effects, we add thread safety to the badger transactions and we manage DAG workers on a per PushLog cycle basis.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When peers are synchronizing a document graph. It will initially recieve update log L which will be n >= 1 updates ahead of the current head of the document graph H.
The data sync protocol will traverse backwards from the newly recieved log L until we get back our current head H.
If there is an error during sync, the Document will be left in an unstable state, and may not be able to recover.
This is related to this issue from IPFS: ipfs/go-ds-crdt#23
The text was updated successfully, but these errors were encountered: