Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring of initial sync #5096

Merged
merged 60 commits into from
Mar 17, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
3cd4159
implements blocks queue
farazdagi Mar 11, 2020
78bf9fa
refactors updateCounter method
farazdagi Mar 12, 2020
cf4bbd6
fixes deadlock on stop w/o start
farazdagi Mar 12, 2020
986a673
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 12, 2020
b70a4a5
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 12, 2020
ffff6a3
refactors updateSchedulerState
farazdagi Mar 12, 2020
42db5d2
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 12, 2020
9083202
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 12, 2020
363270f
more tests on schduler
farazdagi Mar 12, 2020
4c39bdd
parseFetchResponse tests
farazdagi Mar 12, 2020
ae5fca5
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 12, 2020
f611c39
wraps up tests for blocks queue
farazdagi Mar 12, 2020
df3499c
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 12, 2020
1857bb8
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 12, 2020
b4e129d
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 12, 2020
ab2d199
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 12, 2020
596e193
eod commit
farazdagi Mar 13, 2020
2444ff5
Merge branch 'init-sync-blocks-queue' of github.com:prysmaticlabs/pry…
farazdagi Mar 13, 2020
2bbef6e
Merge branch 'master' of github.com:prysmaticlabs/prysm
farazdagi Mar 14, 2020
a10e317
fixes data race in round robin
farazdagi Mar 14, 2020
19cbc1a
revamps fetcher
farazdagi Mar 14, 2020
df3b327
fixes race conditions + livelocks + deadlocks
farazdagi Mar 14, 2020
58e7c3f
Merge branch 'master' into init-sync-refactoring
farazdagi Mar 14, 2020
136d2cb
less verbose output
farazdagi Mar 14, 2020
2c6a1ab
fixes data race, by isolating critical sections
farazdagi Mar 14, 2020
5366144
minor refactoring: resolves blocking calls
farazdagi Mar 14, 2020
c4a439c
implements init-sync queue
farazdagi Mar 14, 2020
471c997
udpate fetch/send buffers in blocks fetcher
farazdagi Mar 14, 2020
5b01b73
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 14, 2020
3f55eeb
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 14, 2020
6d47dcf
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 14, 2020
8e29290
Merge branch 'master' into init-sync-refactoring
rauljordan Mar 14, 2020
1aecbfb
Merge branch 'master' into init-sync-blocks-queue
farazdagi Mar 14, 2020
c900544
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 15, 2020
107da46
blockState enum-like type alias
farazdagi Mar 15, 2020
87c6abd
Merge branch 'init-sync-blocks-queue' of github.com:prysmaticlabs/pry…
farazdagi Mar 15, 2020
6a2811f
refactors common code into releaseTicket()
farazdagi Mar 15, 2020
f7f7ee0
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 15, 2020
8f71aab
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 15, 2020
f6d9369
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 15, 2020
c92be88
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 15, 2020
242de64
Merge refs/heads/master into init-sync-blocks-queue
prylabs-bulldozer[bot] Mar 15, 2020
de1eae7
better gc
farazdagi Mar 16, 2020
dd8b47d
Merge branch 'init-sync-blocks-queue' of github.com:prysmaticlabs/pry…
farazdagi Mar 16, 2020
905621b
linter
farazdagi Mar 16, 2020
462e50d
Merge branch 'master' into init-sync-refactoring
farazdagi Mar 16, 2020
bd8151f
Merge branch 'init-sync-refactoring' of github.com:prysmaticlabs/prys…
farazdagi Mar 16, 2020
c7f166f
fix conficts
farazdagi Mar 16, 2020
13abf04
minor fix to round robin
farazdagi Mar 16, 2020
6073d29
Merge branch 'master' into init-sync-refactoring
farazdagi Mar 16, 2020
ce971d2
Merge refs/heads/master into init-sync-refactoring
prylabs-bulldozer[bot] Mar 16, 2020
33402a1
moves original round robin into its own package
farazdagi Mar 17, 2020
c33b81e
Merge branch 'init-sync-refactoring' of github.com:prysmaticlabs/prys…
farazdagi Mar 17, 2020
e2e3f9e
adds enableInitSyncQueue flag
farazdagi Mar 17, 2020
3ca94a1
fixes issue with init-sync service selection
farazdagi Mar 17, 2020
f0f7f21
Merge refs/heads/master into init-sync-refactoring
prylabs-bulldozer[bot] Mar 17, 2020
4c3f402
Update beacon-chain/sync/initial-sync/round_robin.go
farazdagi Mar 17, 2020
4423fce
initsyncv1 -> initsyncold
farazdagi Mar 17, 2020
5f57ab2
adds span
farazdagi Mar 17, 2020
4e1e5ec
Merge branch 'init-sync-refactoring' of github.com:prysmaticlabs/prys…
farazdagi Mar 17, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions beacon-chain/node/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ go_library(
"//beacon-chain/state/stategen:go_default_library",
"//beacon-chain/sync:go_default_library",
"//beacon-chain/sync/initial-sync:go_default_library",
"//beacon-chain/sync/initial-sync-old:go_default_library",
"//shared:go_default_library",
"//shared/cmd:go_default_library",
"//shared/debug:go_default_library",
Expand Down
48 changes: 39 additions & 9 deletions beacon-chain/node/node.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ import (
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
prysmsync "github.com/prysmaticlabs/prysm/beacon-chain/sync"
initialsync "github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync"
initialsyncold "github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync-old"
"github.com/prysmaticlabs/prysm/shared"
"github.com/prysmaticlabs/prysm/shared/cmd"
"github.com/prysmaticlabs/prysm/shared/debug"
Expand Down Expand Up @@ -404,9 +405,19 @@ func (b *BeaconNode) registerSyncService(ctx *cli.Context) error {
return err
}

var initSync *initialsync.Service
if err := b.services.FetchService(&initSync); err != nil {
return err
var initSync prysmsync.Checker
if cfg := featureconfig.Get(); cfg.EnableInitSyncQueue {
var initSyncTmp *initialsync.Service
if err := b.services.FetchService(&initSyncTmp); err != nil {
return err
}
initSync = initSyncTmp
} else {
var initSyncTmp *initialsyncold.Service
if err := b.services.FetchService(&initSyncTmp); err != nil {
return err
}
initSync = initSyncTmp
}

rs := prysmsync.NewRegularSync(&prysmsync.Config{
Expand All @@ -431,16 +442,25 @@ func (b *BeaconNode) registerInitialSyncService(ctx *cli.Context) error {
return err
}

is := initialsync.NewInitialSync(&initialsync.Config{
if cfg := featureconfig.Get(); cfg.EnableInitSyncQueue {
is := initialsync.NewInitialSync(&initialsync.Config{
DB: b.db,
Chain: chainService,
P2P: b.fetchP2P(ctx),
StateNotifier: b,
BlockNotifier: b,
})
return b.services.RegisterService(is)
}

is := initialsyncold.NewInitialSync(&initialsyncold.Config{
DB: b.db,
Chain: chainService,
P2P: b.fetchP2P(ctx),
StateNotifier: b,
BlockNotifier: b,
})

return b.services.RegisterService(is)

}

func (b *BeaconNode) registerRPCService(ctx *cli.Context) error {
Expand All @@ -454,9 +474,19 @@ func (b *BeaconNode) registerRPCService(ctx *cli.Context) error {
return err
}

var syncService *initialsync.Service
if err := b.services.FetchService(&syncService); err != nil {
return err
var syncService prysmsync.Checker
if cfg := featureconfig.Get(); cfg.EnableInitSyncQueue {
var initSyncTmp *initialsync.Service
if err := b.services.FetchService(&initSyncTmp); err != nil {
return err
}
syncService = initSyncTmp
} else {
var initSyncTmp *initialsyncold.Service
if err := b.services.FetchService(&initSyncTmp); err != nil {
return err
}
syncService = initSyncTmp
}

genesisValidators := ctx.GlobalUint64(flags.InteropNumValidatorsFlag.Name)
Expand Down
63 changes: 63 additions & 0 deletions beacon-chain/sync/initial-sync-old/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test")

go_library(
name = "go_default_library",
srcs = [
"log.go",
"round_robin.go",
"service.go",
],
importpath = "github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync-old",
visibility = ["//beacon-chain:__subpackages__"],
deps = [
"//beacon-chain/blockchain:go_default_library",
"//beacon-chain/core/feed:go_default_library",
"//beacon-chain/core/feed/block:go_default_library",
"//beacon-chain/core/feed/state:go_default_library",
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/db:go_default_library",
"//beacon-chain/flags:go_default_library",
"//beacon-chain/p2p:go_default_library",
"//beacon-chain/sync:go_default_library",
"//proto/beacon/p2p/v1:go_default_library",
"//shared:go_default_library",
"//shared/bytesutil:go_default_library",
"//shared/featureconfig:go_default_library",
"//shared/mathutil:go_default_library",
"//shared/params:go_default_library",
"//shared/roughtime:go_default_library",
"@com_github_kevinms_leakybucket_go//:go_default_library",
"@com_github_libp2p_go_libp2p_core//peer:go_default_library",
"@com_github_paulbellamy_ratecounter//:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
],
)

go_test(
name = "go_default_test",
srcs = ["round_robin_test.go"],
embed = [":go_default_library"],
race = "on",
tags = ["race_on"],
deps = [
"//beacon-chain/blockchain/testing:go_default_library",
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/db/testing:go_default_library",
"//beacon-chain/p2p/peers:go_default_library",
"//beacon-chain/p2p/testing:go_default_library",
"//beacon-chain/state:go_default_library",
"//beacon-chain/sync:go_default_library",
"//proto/beacon/p2p/v1:go_default_library",
"//shared/hashutil:go_default_library",
"//shared/params:go_default_library",
"//shared/roughtime:go_default_library",
"//shared/sliceutil:go_default_library",
"@com_github_kevinms_leakybucket_go//:go_default_library",
"@com_github_libp2p_go_libp2p_core//network:go_default_library",
"@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
"@com_github_prysmaticlabs_go_ssz//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
],
)
7 changes: 7 additions & 0 deletions beacon-chain/sync/initial-sync-old/log.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
package initialsyncold

import (
"github.com/sirupsen/logrus"
)

var log = logrus.WithField("prefix", "initial-sync")
Loading