{un,re}pack: support overlayfs whiteouts #342
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
👍 on |
|
Yeah, I should have done it a while ago :) The tests are failing locally for me with: and I see they're failing with a different error in actual CI, so I'll try again tomorrow and see if they'll fail the same way for me. |
|
The tests are failing locally because you have an old version of |
Oh, I thought docker was smart enough to figure that out. But you're right, I cleared my cache and now it's working. |
tych0
force-pushed
the
add-whiteout-option
branch
6 times, most recently
from
2b9f23b to
70f0eee
Compare
cyphar
reviewed
Jul 11, 2020
tych0
force-pushed
the
add-whiteout-option
branch
from
70f0eee to
989fd97
Compare
A long time ago as a hack to avoid this very patch, I added KeepDirlinks to MapOptions. Then, I added a callback and the StartFrom argument, having to thread that through all the callsites. In the next patch, I'll add another unpack option. So, let's bite the bullet and create an UnpackOptions struct to pass to the unpacking code. Note that this doesn't have *all* of the options like bundle path and such, since those aren't necessarily relevant for all unpacking operations (UnpackRootfs(), UnpackManifest(), Unpack(), etc.). Signed-off-by: Tycho Andersen <tycho@tycho.ws>
we produce this via 'make ci', let's ignore it too. Signed-off-by: Tycho Andersen <tycho@tycho.ws>
tych0
force-pushed
the
add-whiteout-option
branch
from
989fd97 to
6b7327b
Compare
|
Ok, I rebased too because of the gitignore conflict :) |
cyphar
previously requested changes
Jul 14, 2020
While not strictly OCI standard, it can be useful to have umoci write
overlayfs stile whiteouts when unpacking instead of just manipulating the
filesystem on its own, for example for use with drivers which are going to
use overlayfs to speed up image building and extraction.
Note that writing whiteouts (i.e. mknod foo c 0 0) is supported as an
unprivileged user in the upstream kernel via a3c751a50fe6 ("vfs: allow
unprivileged whiteout creation"), which will be released in 5.8. There is
*no* support right now for writing the opaque overlay attribute as an
unprivileged user, but since that's mostly done via GenerateInsertLayer()
and not in the general case, I doubt most people will run into this. And if
they do, they can send a kernel patch :)
Note that we only run the test on linux, since osx presumably doesn't have
this hack to let us mknod c 0 0 as an unprivileged user.
Signed-off-by: Tycho Andersen <tycho@tycho.ws>
This is the compliment to the previous commit, in case people want to
re-pack layers using the OCI standard instead of the overlayfs standard.
Also add a PackOptions{} struct, similar to UnpackOptions{}, which
can control the low level behavior of the packing routines. I expect we'll
use this later for hooks, too.
Signed-off-by: Tycho Andersen <tycho@tycho.ws>
tych0
force-pushed
the
add-whiteout-option
branch
from
6b7327b to
a425f92
Compare
This way we can automatically DTRT if people set the WhiteoutMode during Unpack(). Signed-off-by: Tycho Andersen <tycho@tycho.ws>
|
Seems I can't run the CI manually? |
|
Ok, I think I addressed everything modulo the Translate bit. |
|
Sorry, I forgot that I needed to push the image. I've pushed it to Docker Hub. |
|
Ping, I think this one is good to go. |
|
Ping? |
1 similar comment
|
Ping? |
|
Ping? |
|
LGTM, sorry for letting this one sit for so long (I got half-way through re-reviewing it several times). |
|
NP, thanks :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Here's a couple of patches. The first is me cleaning up a mess I made, and the second one is adding overlay style whiteout support. See patch notes for details.