Skip to content
This repository has been archived by the owner on Aug 2, 2021. It is now read-only.

swarm/network/stream: refactor bootstrapping of tests #1176

Closed
wants to merge 1 commit into from

Conversation

nonsense
Copy link
Contributor

@nonsense nonsense commented Jan 30, 2019

After reviewing @holisticode 's PR ethereum/go-ethereum#18972 and discussing my opinion on the test it was adding, I looked into refactoring the service bootstrap that is replicated over 10 simulation tests.

I came up with 3 constructors that pretty much abstract the functionality we need:

// newNetStoreAndDelivery is a default constructor for BzzAddr, NetStore and Delivery, used in Simulations

// newNetStoreAndDeliveryWithBzzAddr is a constructor for NetStore and Delivery, used in Simulations, accepting any BzzAddr

// newNetStoreAndDeliveryWithRequestFunc is a constructor for NetStore and Delivery, used in Simulations, accepting any NetStore.RequestFunc

These reduce the copy-pasted boilerplate that we have in the simulation tests.

I didn't get as far as reviewing the constructor for NewRegistry, but maybe there are things we could simplify there as well.

Let me know what you think.

@nonsense nonsense added ready for review cleanup code completion, add comments and more labels Jan 30, 2019
@nonsense
Copy link
Contributor Author

Note that there were number of places where we were not calling r.Close() in the clean up. Not sure if this is an issue or is fine, @janos probably knows best.

@janos
Copy link
Member

janos commented Jan 30, 2019

Note that there were number of places where we were not calling r.Close() in the clean up. Not sure if this is an issue or is fine, @janos probably knows best.

Good catch @nonsense. I thought that network shutdown will do that, but actually it will just call Stop method on Service (in this case Registry). Registry closing is required for closing intervals database.

n := ctx.Config.Node()

store, datadir, err := createTestLocalStorageForID(n.ID(), addr)
if *useMockStore {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@janos I am pretty sure this doesn't work, but haven't tried it just yet. Not sure how we cast a mock store to *storage.LocalStore further down, but this is how it is in the current code as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have found that mock store in stream tests were broken after some rewrite. I am not sure if they were fixed. Maybe @holisticode have more details, but I will have a look also.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it is important that mock store is not casted to LocalStore, there were numerous discussion on this, but injected down to ldbstore.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least now we have to fix it at one place, and not at multiple :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MockStore is working correctly here, but naming is confusing. createMockStore does not create mock store, it creates new localstore with injected mockstore. It is similar to createTestLocalStorageForID, so maybe to rename this function to createTestLocalStoreWithMockStore, or newLocalStoreWithMock, and to move it from syncer_test.go to common_test.go. BTW, the description of this flag is completely wrong, it is false by default.

@nonsense
Copy link
Contributor Author

Rebased on master after merging ethereum#18972 - review should be easier now.

Copy link
Member

@janos janos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great generalization. LGTM.

n := ctx.Config.Node()

store, datadir, err := createTestLocalStorageForID(n.ID(), addr)
if *useMockStore {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MockStore is working correctly here, but naming is confusing. createMockStore does not create mock store, it creates new localstore with injected mockstore. It is similar to createTestLocalStorageForID, so maybe to rename this function to createTestLocalStoreWithMockStore, or newLocalStoreWithMock, and to move it from syncer_test.go to common_test.go. BTW, the description of this flag is completely wrong, it is false by default.

Copy link
Contributor

@holisticode holisticode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great PR, simplifies simulation creation a lot.

@janos @nonsense would it actually make sense to move these helper functions into the simulation package itself instead of here? In other words, could they be helpful outside the stream package?


fileStore := storage.NewFileStore(netStore, storage.NewFileStoreParams())
bucket.Store(bucketKeyFileStore, fileStore)
r := NewRegistry(addr.ID(), delivery, netStore, state.NewInmemoryStore(), &RegistryOptions{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we'd just add a RegistryOptions object to the newNetStoreAndDeliveryWithRequestFunc, which then also would contain this NewRegistry code? I know it starts to look ugly with so many params...but it could further simplify simulation creation

Copy link
Contributor Author

@nonsense nonsense Jan 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@holisticode we override the registry in some of the tests, this is why I didn't do it - its not just the RegistryOptions that change.

Personally I think even the current constructors are a bit too complicated, but this is because of the high coupling between the interfaces...

The current solution is far from ideal, I just hope it bring slightly better defaults.

@zelig
Copy link
Member

zelig commented Feb 1, 2019

merged as ethereum/go-ethereum#18975

@zelig zelig closed this Feb 1, 2019
@acud acud deleted the sync-monitor-anton branch February 11, 2019 02:34
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cleanup code completion, add comments and more ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants