enhance: cache image size and oci image spec in pouchd #2301
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #2301 +/- ##
==========================================
+ Coverage 66.61% 66.84% +0.22%
==========================================
Files 211 211
Lines 17195 17205 +10
==========================================
+ Hits 11455 11500 +45
+ Misses 4337 4308 -29
+ Partials 1403 1397 -6
|
allencloud
reviewed
Oct 9, 2018
daemon/mgr/image_store.go
Outdated
| @@ -62,6 +65,16 @@ type imageStore struct { | |||
|
|
|||
| // primaryRefsIndexByID stores primay references, index by image ID | |||
| primaryRefsIndexByID map[digest.Digest]referenceMap | |||
|
|
|||
| // cache ociImage to avoid open stream grpc to containerd, index by digest | |||
There was a problem hiding this comment.
Please add the consequence which will happen if not do the cache thing.
when the containerd manages many big images, the pouchd ListImage will consume a lot of CPU resources caused by containerd.Content.Read stream grpc call. In the k8s, the kubelet gc image manager will send the ListImage request every 30s. If the containerd consumes too many CPU resources, it will impact the container. Therefore, pouchd needs to cache the data. Signed-off-by: Wei Fu <fuweid89@gmail.com>
HusterWan
reviewed
Oct 10, 2018
| if err != nil { | ||
| if err == errCtrdImageInfoNotExist { |
There was a problem hiding this comment.
If the image not found in local store, should we check it from containerd?
HusterWan
reviewed
Oct 10, 2018
| // RemoveAllReferences removes all the reference by the given imageID. | ||
| func (store *imageStore) RemoveAllReferences(id digest.Digest) error { | ||
| // ListCtrdImageInfo returns all the CtrdImageInfo. | ||
| func (store *imageStore) ListCtrdImageInfo() []CtrdImageInfo { |
HusterWan
reviewed
Oct 10, 2018
| // containerd.RemoveImage must success. so if the localStore has been | ||
| // remove all the primary references, we should clear the CtrdImageInfo | ||
| // cache. | ||
| defer func() { |
There was a problem hiding this comment.
Whether we should remove the cache image info when the image has been succeeded?
|
I have discuss with @fuweid offline and think this PR is LGTM |
pouchrobot
added
the
LGTM
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Wei Fu fuweid89@gmail.com
Ⅰ. Describe what this PR did
when the containerd manages many big images, the pouchd ListImage will
consume a lot of CPU resources caused by containerd.Content.Read stream
grpc call.
In the k8s, the kubelet gc image manager will send the ListImage request
every 30s. If the containerd consumes too many CPU resources, it will
impact the container. Therefore, pouchd needs to cache the data.
Ⅱ. Does this pull request fix one issue?
NONE
Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗？)
no need
Ⅳ. Describe how to verify it
CI
Ⅴ. Special notes for reviews