Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix: avoid data race in set exec exit status #2152

Merged
merged 1 commit into from
Aug 29, 2018
Merged

bugfix: avoid data race in set exec exit status #2152

merged 1 commit into from
Aug 29, 2018

Conversation

Ace-Tang
Copy link
Contributor

there are two goroutines response for changed exec config information,
if exec process got error, data race appears, exec exit hook runs in a
goroutine, it modify exec exit status, StartExec also modify exec exit
status in another goroutine since error happend. Fix it by only one
place can modify exec exit status.

Signed-off-by: Ace-Tang aceapril@126.com

Ⅰ. Describe what this PR did

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗?)

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

@pouchrobot pouchrobot added kind/bug This is bug report for project size/S labels Aug 23, 2018
@Ace-Tang
Copy link
Contributor Author

Ace-Tang commented Aug 23, 2018

The result may error depend on which goroutine run last

#./pouch exec 30f44535e526947d9e1cb41974f4fb944c596c385938d353002f0e44646ea7ea aaaaa
failed to exec process: OCI runtime exec failed: exec failed: container_linux.go:265: starting container process caused "exec: \"aaaaa\": executable file not found in $PATH": unknown

#echo $?
0
#./pouch exec 30f44535e526947d9e1cb41974f4fb944c596c385938d353002f0e44646ea7ea dsad
failed to exec process: OCI runtime exec failed: exec failed: container_linux.go:265: starting container process caused "exec: \"dsad\": executable file not found in $PATH": unknown

#echo $?
126

@@ -77,7 +77,6 @@ func (mgr *ContainerManager) StartExec(ctx context.Context, execid string, attac
exitCode := 126
execConfig.ExitCode = int64(exitCode)
}
mgr.ExecProcesses.Put(execid, execConfig)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

execConfig is a pointer, no need to use mgr.ExecProcesses.Put again.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this part, I want to raise proposal that we should deep copy data from manager data store to avoid the data race. But it will introduce the complicated.

@codecov-io
Copy link

codecov-io commented Aug 23, 2018

Codecov Report

Merging #2152 into master will decrease coverage by 24.95%.
The diff coverage is 40%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #2152       +/-   ##
==========================================
- Coverage   64.46%   39.5%   -24.96%     
==========================================
  Files         209     209               
  Lines       16680   16678        -2     
==========================================
- Hits        10752    6589     -4163     
- Misses       4599    9053     +4454     
+ Partials     1329    1036      -293
Flag Coverage Δ
#criv1alpha1test ?
#criv1alpha2test ?
#integrationtest 39.5% <40%> (+0.19%) ⬆️
#unittest ?
Impacted Files Coverage Δ
daemon/mgr/container_exec.go 77.66% <ø> (+9.4%) ⬆️
ctrd/container.go 43.3% <40%> (+0.1%) ⬆️
apis/opts/sysctl.go 0% <0%> (-100%) ⬇️
cri/v1alpha1/cri_types.go 0% <0%> (-100%) ⬇️
apis/opts/shm_size.go 0% <0%> (-100%) ⬇️
apis/opts/memory.go 0% <0%> (-100%) ⬇️
apis/opts/memory_swap.go 0% <0%> (-100%) ⬇️
cri/stream/errors.go 0% <0%> (-100%) ⬇️
cri/v1alpha2/cri_types.go 0% <0%> (-100%) ⬇️
cri/stream/httpstream/httpstream.go 0% <0%> (-96.3%) ⬇️
... and 105 more

logrus.Errorf("failed to execute the exec exit hooks: %v", err)
break
// run hook if not got fail here
if err := <-fail; err == nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we get the real exit code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When fails here, we got a empty(meaningless) exit status, that means exit code is zero.


// test a 'executable file not found' fail should get exit code 126
name = "TestExecFailCode"
command.PouchRun("run", "-d", "--name", name, busyboxImage, "top").Assert(c, icmd.Success)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you update the case description, since there are two cases here?

there are two goroutines response for changed exec config information,
if exec process got error, data race appears, exec exit hook runs in a
goroutine, it modify exec exit status, StartExec also modify exec exit
status in another goroutine since error happend. Fix it by only one
place can modify exec exit status.

Signed-off-by: Ace-Tang <aceapril@126.com>
@Ace-Tang
Copy link
Contributor Author

fail with

" exited with 0.
$ bash <(curl -s https://codecov.io/bash) -cF ${TEST_SUITE} -y .codecov.yml
/dev/fd/63: line 1: html: No such file or directory
/dev/fd/63: line 2: syntax error near unexpected token `<'

func (suite *PouchExecSuite) TestExecFail(c *check.C) {
name := "TestExecFail"
res := command.PouchRun("run", "-d", "--name", name, "-u", name, busyboxImage, "top")
defer DelContainerForceMultyTime(c, name)
c.Assert(res.Stderr(), check.NotNil)

// test a 'executable file not found' fail should get exit code 126.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we split it into two cases?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just keep this ?

@@ -77,7 +77,6 @@ func (mgr *ContainerManager) StartExec(ctx context.Context, execid string, attac
exitCode := 126
execConfig.ExitCode = int64(exitCode)
}
mgr.ExecProcesses.Put(execid, execConfig)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this part, I want to raise proposal that we should deep copy data from manager data store to avoid the data race. But it will introduce the complicated.

@Ace-Tang
Copy link
Contributor Author

@fuweid, deep copy may need, but for this case, we should keep the data race from the logic, deep copy won't avoid this.

@yyb196
Copy link
Collaborator

yyb196 commented Aug 29, 2018

LGTM

@pouchrobot pouchrobot added the LGTM one maintainer or community participant agrees to merge the pull reuqest. label Aug 29, 2018
@yyb196 yyb196 merged commit 3311b05 into AliyunContainerService:master Aug 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug This is bug report for project LGTM one maintainer or community participant agrees to merge the pull reuqest. size/S
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants