Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: use ringBuffer to make log not blocking and make it configurable #2428

Merged
merged 1 commit into from
Nov 22, 2018

Conversation

starnop
Copy link
Contributor

@starnop starnop commented Nov 5, 2018

Signed-off-by: Starnop starnop@163.com

Ⅰ. Describe what this PR did

As the title describes.

Ⅱ. Does this pull request fix one issue?

None.

Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗?)

Add it Later.

Ⅳ. Describe how to verify it

Add it Later.

Ⅴ. Special notes for reviews

@codecov
Copy link

codecov bot commented Nov 5, 2018

Codecov Report

Merging #2428 into master will decrease coverage by 0.08%.
The diff coverage is 31.16%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2428      +/-   ##
==========================================
- Coverage    69.2%   69.11%   -0.09%     
==========================================
  Files         277      278       +1     
  Lines       18330    18387      +57     
==========================================
+ Hits        12685    12709      +24     
- Misses       4206     4242      +36     
+ Partials     1439     1436       -3
Flag Coverage Δ
#criv1alpha1test 31.35% <7.79%> (-0.07%) ⬇️
#criv1alpha2test 35.54% <7.79%> (-0.07%) ⬇️
#integrationtest 40.51% <11.68%> (-0.26%) ⬇️
#nodee2etest 32.81% <7.79%> (-0.17%) ⬇️
#unittest 26.6% <19.48%> (-0.1%) ⬇️
Impacted Files Coverage Δ
daemon/containerio/io.go 72.81% <0%> (-6.98%) ⬇️
daemon/logger/logbuffer/logbuff.go 0% <0%> (ø)
daemon/logger/logbuffer/list.go 93.75% <100%> (ø)
daemon/mgr/container.go 58.6% <22.22%> (+0.24%) ⬆️
daemon/mgr/container_validation.go 43.67% <46.66%> (-0.08%) ⬇️
daemon/logger/logbuffer/ringbuff.go 80.76% <68.42%> (ø)
apis/server/utils.go 71.15% <0%> (-3.85%) ⬇️
daemon/mgr/spec_linux.go 78.3% <0%> (-1.02%) ⬇️
cri/v1alpha2/cri_wrapper.go 63.6% <0%> (ø) ⬆️
... and 5 more

}

if info.LogConfig["mode"] == LogNonBlocking {
return bufferlog.NewBufferLog(ld, info)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we postpone to start the goroutine after startLogging?

daemon/logger/bufferlog/bufferlog.go Outdated Show resolved Hide resolved
@starnop starnop force-pushed the not-blocking branch 3 times, most recently from d7e2c97 to c6e6f14 Compare November 7, 2018 05:17
@pouchrobot pouchrobot added size/XL and removed size/L labels Nov 7, 2018
@starnop starnop force-pushed the not-blocking branch 5 times, most recently from 70b8fe0 to 5eaadc6 Compare November 7, 2018 07:30
@starnop starnop changed the title [WIP] feature: use ringBuffer to make log not blocking and make it configurable feature: use ringBuffer to make log not blocking and make it configurable Nov 7, 2018
@starnop
Copy link
Contributor Author

starnop commented Nov 7, 2018

@HusterWan @fuweid All mentioned above has been updated. PTAL. THX.

Copy link
Contributor

@fuweid fuweid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can't use origin ringbuffer implementation. we should change it.

daemon/logger/logbuffer/logbuff.go Outdated Show resolved Hide resolved
daemon/logger/logbuffer/ringbuff.go Outdated Show resolved Hide resolved
daemon/logger/logbuffer/ringbuff.go Outdated Show resolved Hide resolved
@starnop starnop force-pushed the not-blocking branch 2 times, most recently from 8d3b1b7 to 852dc7f Compare November 20, 2018 03:38
@fuweid
Copy link
Contributor

fuweid commented Nov 20, 2018

Basically, LGTM. But, @starnop could you help to add benchmark cases for the logbuffer?

daemon/mgr/container.go Outdated Show resolved Hide resolved
@starnop
Copy link
Contributor Author

starnop commented Nov 21, 2018

Basically, LGTM. But, @starnop could you help to add benchmark cases for the logbuffer?

Maybe add benchmark cases in the next PR?

@fuweid
Copy link
Contributor

fuweid commented Nov 21, 2018

oops. @starnop could you help to take a look for this?

$ sudo pouch run --rm --log-opt mode=non-blocking busybox sh -c 'for i in $(seq 1 1000); do echo 1; done' | wc -l
Error: failed to run container: {"message":"unknown log opt 'mode' for json-file log driver"}

@fuweid
Copy link
Contributor

fuweid commented Nov 21, 2018

Maybe add benchmark cases in the next PR?

Sure.

@starnop
Copy link
Contributor Author

starnop commented Nov 21, 2018

oops. @starnop could you help to take a look for this?

$ sudo pouch run --rm --log-opt mode=non-blocking busybox sh -c 'for i in $(seq 1 1000); do echo 1; done' | wc -l
Error: failed to run container: {"message":"unknown log opt 'mode' for json-file log driver"}

Has been fixed.

@starnop starnop force-pushed the not-blocking branch 2 times, most recently from 4cdf322 to 32d02ed Compare November 22, 2018 02:10
Copy link
Contributor

@fuweid fuweid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fuweid fuweid merged commit c523380 into AliyunContainerService:master Nov 22, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants