Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[storage] Small Nits #255

Merged
merged 2 commits into from
Dec 1, 2020
Merged

[storage] Small Nits #255

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
297d70e
limit errgroup to number of CPUs
patrick-ogrady Dec 1, 2020
deb00b5
pre-allocate allChanges
patrick-ogrady Dec 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ require (
github.com/kr/pretty v0.2.0 // indirect
github.com/lucasjones/reggen v0.0.0-20180717132126-cdb49ff09d77
github.com/mitchellh/mapstructure v1.3.3
github.com/neilotoole/errgroup v0.1.5
github.com/pkg/errors v0.9.1 // indirect
github.com/segmentio/fasthash v1.0.3
github.com/stretchr/objx v0.1.1 // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,8 @@ github.com/mitchellh/mapstructure v1.3.3 h1:SzB1nHZ2Xi+17FP0zVQBHIZqvwRN9408fJO8
github.com/mitchellh/mapstructure v1.3.3/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/naoina/go-stringutil v0.1.0/go.mod h1:XJ2SJL9jCtBh+P9q5btrd/Ylo8XwT/h1USek5+NqSA0=
github.com/naoina/toml v0.1.2-0.20170918210437-9fafd6967416/go.mod h1:NBIhNtsFMo3G2szEBne+bO4gS192HuIYRqfvOWb4i1E=
github.com/neilotoole/errgroup v0.1.5 h1:DxEGoIfFm5ooGicidR+okiHjoOaGRKFaSxDPVZuuu2I=
github.com/neilotoole/errgroup v0.1.5/go.mod h1:Q2nLGf+594h0CLBs/Mbg6qOr7GtqDK7C2S41udRnToE=
github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
github.com/olekukonko/tablewriter v0.0.1/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo=
Expand Down
6 changes: 4 additions & 2 deletions parser/balance_changes.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,9 +128,11 @@ func (p *Parser) BalanceChanges(
}
}

allChanges := []*BalanceChange{}
i := 0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh I'm not a go expert but I should have seen this that the key is string. ACK this thanks!

allChanges := make([]*BalanceChange, len(balanceChanges))
for _, change := range balanceChanges {
allChanges = append(allChanges, change)
allChanges[i] = change
i++
}

return allChanges, nil
Expand Down
8 changes: 5 additions & 3 deletions storage/modules/balance_storage.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ import (
"math/big"
"sync"

"golang.org/x/sync/errgroup"
"github.com/neilotoole/errgroup"

"github.com/coinbase/rosetta-sdk-go/asserter"
"github.com/coinbase/rosetta-sdk-go/parser"
Expand Down Expand Up @@ -194,7 +194,8 @@ func (b *BalanceStorage) AddingBlock(
var newAccounts int
var newAccountsLock sync.Mutex

g, gctx := errgroup.WithContext(ctx)
// Concurrent execution limited to runtime.NumCPU
g, gctx := errgroup.WithContextN(ctx, 0, 0)
for i := range changes {
// We need to set variable before calling goroutine
// to avoid getting an updated pointer as loop iteration
Expand Down Expand Up @@ -268,7 +269,8 @@ func (b *BalanceStorage) RemovingBlock(
staleAccounts := []*types.AccountCurrency{}
var staleAccountsMutex sync.Mutex

g, gctx := errgroup.WithContext(ctx)
// Concurrent execution limited to runtime.NumCPU
g, gctx := errgroup.WithContextN(ctx, 0, 0)
for i := range changes {
// We need to set variable before calling goroutine
// to avoid getting an updated pointer as loop iteration
Expand Down