Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix: metrics client is initialized 2 times per shard #484

Merged
merged 4 commits into from
Jan 2, 2018
Merged

Conversation

wxing1292
Copy link
Contributor

solves #477

@coveralls
Copy link

Coverage Status

Coverage increased (+0.5%) to 67.357% when pulling 50c311f on bugfix into 36c6f54 on master.

@vancexu
Copy link
Contributor

vancexu commented Dec 27, 2017

Where is the 2 times initialization in old implementation?

@wxing1292
Copy link
Contributor Author

@vancexu cassandraPersistenceClientFactory.go and shardContext.go

Copy link

@yiminc-zz yiminc-zz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you verify your fix would solve #477?
It seems we could simply tag the reporter in newShardController() and remove the other 2 places without these refactoring change?
BTW, why do we need the tag? How is that used?

@wxing1292
Copy link
Contributor Author

@yiminc
#484 (review)
A1: I change the # of shards from 4 to 16384, and launch the service, check the memory usage using pprof, which is ~7G (used by this tagged metrics client), I change to this bugfix, and redo the check, the memory usage is ~ 3-4 G. so issue fixed.

A2 & A3: since the tagged metrics client is per shard (from the original code), i assume that we need to continue to do that.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.04%) to 66.795% when pulling b41eff2 on bugfix into 36c6f54 on master.

@@ -94,7 +94,7 @@ func (s *shardControllerSuite) TestAcquireShardSuccess() {
if hostID == 0 {
myShards = append(myShards, shardID)
mockExecutionMgr := &mmocks.ExecutionManager{}
s.mockExecutionMgrFactory.On("CreateExecutionManager", mock.Anything).Return(mockExecutionMgr, nil).Once()
s.mockExecutionMgrFactory.On("CreateExecutionManager", mock.Anything, mock.Anything).Return(mockExecutionMgr, nil).Once()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you still need this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch

logger bark.Logger
metricsClient metrics.Client

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revert this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need this field, so the factory can use the metrics client if provided

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@@ -214,7 +214,8 @@ func (s *Service) Start() {
p.CassandraConfig.Keyspace,
s.config.ExecutionMgrNumConns,
p.Logger,
base.GetMetricsClient())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this method do? Should it be removed if no longer used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -107,7 +107,7 @@ func newShardController(host *membership.HostInfo, resolver membership.ServiceRe

func newHistoryShardsItem(shardID int, shardMgr persistence.ShardManager, historyMgr persistence.HistoryManager,
metadataMgr persistence.MetadataManager, executionMgrFactory persistence.ExecutionManagerFactory, factory EngineFactory,
host *membership.HostInfo, config *Config, logger bark.Logger, reporter metrics.Client) (*historyShardsItem, error) {
host *membership.HostInfo, config *Config, logger bark.Logger, metricsClient metrics.Client) (*historyShardsItem, error) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: revert this as well.

@@ -37,7 +37,7 @@ type (

// NewCassandraPersistenceClientFactory is used to create an instance of ExecutionManagerFactory implementation
func NewCassandraPersistenceClientFactory(hosts string, port int, user, password, dc string, keyspace string,
numConns int, logger bark.Logger, mClient metrics.Client) (ExecutionManagerFactory, error) {
numConns int, logger bark.Logger, metricsClient metrics.Client) (ExecutionManagerFactory, error) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: revert this

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.04%) to 66.795% when pulling d744f7f on bugfix into 36c6f54 on master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.03%) to 66.812% when pulling 6c20856 on bugfix into 36c6f54 on master.

@wxing1292 wxing1292 merged commit 8a89e91 into master Jan 2, 2018
@wxing1292 wxing1292 deleted the bugfix branch January 2, 2018 22:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants