Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server/embed: fix data race when start insecure grpc #15509

Merged
merged 1 commit into from
Mar 19, 2023

Conversation

fuweid
Copy link
Member

@fuweid fuweid commented Mar 18, 2023

There are two goroutines accessing the gs grpc server var. Before insecure gs server start, the gs can be changed to secure server and then the client will fail to connect to etcd with insecure request. It is data-race. We should use argument for reference in the new goroutine.

fix: #15495

Please read https://github.com/etcd-io/etcd/blob/main/CONTRIBUTING.md#contribution-flow.

There are two goroutines accessing the `gs` grpc server var. Before
insecure `gs` server start, the `gs` can be changed to secure server and
then the client will fail to connect to etcd with insecure request. It
is data-race. We should use argument for reference in the new goroutine.

fix: etcd-io#15495

Signed-off-by: Wei Fu <fuweid89@gmail.com>
@fuweid fuweid marked this pull request as ready for review March 18, 2023 14:57
Copy link
Member

@ahrtr ahrtr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Thank you @fuweid

Comment on lines +186 to +192
defer func(gs *grpc.Server) {
if err != nil {
sctx.lg.Warn("stopping secure grpc server due to error", zap.Error(err))
gs.Stop()
sctx.lg.Warn("stopped secure grpc server due to error", zap.Error(err))
}
}(gs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comment: the code block is almost identical to the code block for insecure. I am thinking to define a separate function something like below, then call it in both secure and insecure code block.

func closegRPCServer(gs *grpc.Server) {
  ...
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it needs return error to the log, I'm not sure how to handle this with your idea. In here, I follow the defer pattern~

@serathius serathius merged commit 30abf17 into etcd-io:main Mar 19, 2023
@serathius
Copy link
Member

@fuweid Could you backport it to release-3.5 and release-3.4?

@fuweid
Copy link
Member Author

fuweid commented Mar 20, 2023

@fuweid Could you backport it to release-3.5 and release-3.4?

Sure! Will send prs later.

@fuweid fuweid deleted the fix-15495 branch March 20, 2023 12:38
fuweid added a commit to fuweid/etcd that referenced this pull request Mar 22, 2023
Add item **server/embed: fix data race when start insecure grpc** to
3.4/3.5 changelog.

REF: etcd-io#15509

Signed-off-by: Wei Fu <fuweid89@gmail.com>
fuweid added a commit to fuweid/etcd that referenced this pull request Mar 22, 2023
Add item [server/embed: fix data race when starting both secure & insecure
gRPC servers on the same address] to 3.4/3.5 changelog.

REF: etcd-io#15509

Signed-off-by: Wei Fu <fuweid89@gmail.com>
@jmhbnz jmhbnz mentioned this pull request Sep 25, 2023
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Flaking TestConnectionMultiplexing
3 participants