This repository has been archived by the owner on Nov 1, 2022. It is now read-only.
Reduce sorts done by ListImagesWithOptions #2338
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
squaremo
force-pushed
the
do-less-sorting
branch
from
6c98cd0 to
cd37ae6
Compare
squaremo
changed the title
squaremo
force-pushed
the
do-less-sorting
branch
from
7d0729e to
d229dd3
Compare
squaremo
force-pushed
the
do-less-sorting
branch
from
d229dd3 to
1fa9c25
Compare
hiddeco
approved these changes
Aug 12, 2019
squaremo
force-pushed
the
do-less-sorting
branch
from
1fa9c25 to
d39a3a5
Compare
The ListImagesWithOptions API method lets you supply a list of the fields you want for each container, so that you can cut down on the size of the response when you don't care about some fields. But all the fields are calculated, whatever you ask for, which involves a lot of sorting and filtering. This commit makes the procedure calculating the fields _only_ calculate the fields that were actually requested. Some of the calculations depend on the same intermediate results; so, the approach is to lazily calculate and cache intermediate results.
Much of the time, the images returned from the ListImagesWithOptions API are sorted according to timestamp. As a cheap and simple optimisation for this case, when we're constructing a result, cache both the slice of image.Info records and those records sorted by timestamp. This means, for any given request, images that occur in more than one container will be sorted by timestamp only once.
squaremo
force-pushed
the
do-less-sorting
branch
from
d39a3a5 to
1578a12
Compare
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The PR as a whole is intended to address #2328 by reducing the number of sorts, since that was identified as a bottleneck there.
The API
ListImagesWithOptionsas called by Weave Cloud asks for pretty much all the fields, so the first commit here will not make a big difference -- there will still be one sort per container. It may avoid some allocation, and some calculation in other uses (i.e., when not called by Weave Cloud).The second commit caches the result of sorting by timestamp, when that is done. It will be effective to the extent that
There are some modest improvements that could be made with more invasive changes. Possibly the biggest improvement would be to sort images (actually the image tags) when they are fetched, rather than when they are used, since the latter will usually outnumber the former by orders of magnitude.