Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: only in neighborhood replication #2237

Merged
merged 3 commits into from
Jul 6, 2021
Merged

fix: only in neighborhood replication #2237

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
04fed0e
fix: only in neighborhood replication
metacertain Jun 29, 2021
703f349
fix: explicit within depth func
istae Jul 6, 2021
b1d4236
fix: shortened mock func
istae Jul 6, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions pkg/pushsync/pushsync.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,10 @@ func (ps *PushSync) pushToClosest(ctx context.Context, ch swarm.Chunk, retryAllo
return nil, ErrWarmup
}

if !ps.topologyDriver.IsWithinDepth(ch.Address()) {
return nil, ErrNoPush
}

count := 0
// Push the chunk to some peers in the neighborhood in parallel for replication.
// Any errors here should NOT impact the rest of the handler.
Expand Down
6 changes: 1 addition & 5 deletions pkg/pushsync/pushsync_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,13 +129,9 @@ func TestReplicateBeforeReceipt(t *testing.T) {
defer storerEmpty.Close()
emptyRecorder := streamtest.New(streamtest.WithProtocols(psEmpty.Protocol()), streamtest.WithBaseAddr(secondPeer))

wFunc := func(addr swarm.Address) bool {
return true
}

// node that is connected to closestPeer
// will receieve chunk from closestPeer
psSecond, storerSecond, _, secondAccounting := createPushSyncNode(t, secondPeer, defaultPrices, emptyRecorder, nil, defaultSigner, mock.WithPeers(emptyPeer), mock.WithIsWithinFunc(wFunc))
psSecond, storerSecond, _, secondAccounting := createPushSyncNode(t, secondPeer, defaultPrices, emptyRecorder, nil, defaultSigner, mock.WithPeers(emptyPeer))
defer storerSecond.Close()
secondRecorder := streamtest.New(streamtest.WithProtocols(psSecond.Protocol()), streamtest.WithBaseAddr(closestPeer))

Expand Down
2 changes: 1 addition & 1 deletion pkg/topology/mock/mock.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ func (m *mock) IsWithinDepth(addr swarm.Address) bool {
if m.isWithinFunc != nil {
return m.isWithinFunc(addr)
}
return false
return true
Copy link
Member

@istae istae Jul 6, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@acud The reason why the default is now true is because the chunk must be in the neighborhood of the storer peer with this change. The majority of the tests expect a valid receipt so the closest peers in the tests must be in the neighborhood. As of writing this, there is only one test that expects a peer to be out of depth.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In short, the previous default false causes every test to fail.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@istae then can we have an explicit change to the tests, in which a custom mocked IsWithinFunc is injected with a true? my concern is that with this change, also forwarder nodes (and there are forwarding tests) will see the chunk as within depth, leading to a change in the test cases' final state

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is also counter-intuitive that we would now test everything with the following assumptions by default:

  • originator node sees all chunks as within neighborhood
  • fowarder node sees all chunks as within neighborhood
  • storer node sees all chunks as within neighborhood
  • replicating node sees all chunks as within neighborhood

a bit too permissive IMO. so I would vote for reverting this back to false and have an explicit true injected with a mock wherever needed

}

func (m *mock) EachNeighbor(f topology.EachPeerFunc) error {
Expand Down