Better support for testing with different storage drivers #162
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Make the tests use the storage driver named in $STORAGE_DRIVER, if one's set, instead of hard-coding the default of "vfs". Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
In the imgtype test helper, add a -debug flag, correctly handle things on the off chance that we need to call a reexec handler, and read the manifest using the Manifest() method of an image that we're already opening, rather than creating a source image just so that we can call its GetManifest() method. Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Logging at Fatal calls os.Exit(), which keeps us from shutting down storage properly, which prevents test cleanup from succeeding. Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
|
LGTM |
|
@TomSweeneyRedHat PTAL |
| logrus.Fatalf("error parsing reference %q: %v", image, err) | ||
| } | ||
|
|
||
| src, err := ref.NewImageSource(systemContext, []string{expectedManifestType}) |
There was a problem hiding this comment.
Is this test/check no longer needed? I don't see it in the new version.
There was a problem hiding this comment.
The code for handling an error result from ParseStoreReference? The diff is assuming one if err != nil is equivalent to another, but we should still be handling that case in the proposed version.
There was a problem hiding this comment.
I wasn't referring to the Transport.ParseStoreReference(), but the call to ref.NewImageSource on line 85 in the old code. I didn't see that elsewhere.
There was a problem hiding this comment.
Ah, yes. Realized that we can grab the manifest contents from the Image object, so there's no need to create an ImageSource object to read it when we already have an Image object.
|
@TomSweeneyRedHat Can we merge? |
rh-atomic-bot
pushed a commit
that referenced
this pull request
Jun 28, 2017
In the imgtype test helper, add a -debug flag, correctly handle things on the off chance that we need to call a reexec handler, and read the manifest using the Manifest() method of an image that we're already opening, rather than creating a source image just so that we can call its GetManifest() method. Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> Closes: #162 Approved by: rhatdan
rh-atomic-bot
pushed a commit
that referenced
this pull request
Jun 28, 2017
Logging at Fatal calls os.Exit(), which keeps us from shutting down storage properly, which prevents test cleanup from succeeding. Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> Closes: #162 Approved by: rhatdan
|
☀️ Test successful - status-redhatci |
nalind
pushed a commit
that referenced
this pull request
Apr 2, 2018
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com> Closes: #162 Approved by: rhatdan
kolyshkin
added a commit
to kolyshkin/buildah
that referenced
this pull request
May 16, 2020
This subtle bug keeps lurking in because error checking for `Mkdir()` and `MkdirAll()` is slightly different wrt `EEXIST`/`IsExist`: - for `Mkdir()`, `IsExist` error should (usually) be ignored (unless you want to make sure directory was not there before) as it means "the destination directory was already there"; - for `MkdirAll()`, `IsExist` error should NEVER be ignored. This commit removes ignoring the IsExist error, as it should not be ignored. For more details, a quote from opencontainers/runc PR containers#162: -quote- TL;DR: check for IsExist(err) after a failed MkdirAll() is both redundant and wrong -- so two reasons to remove it. Quoting MkdirAll documentation: > MkdirAll creates a directory named path, along with any necessary > parents, and returns nil, or else returns an error. If path > is already a directory, MkdirAll does nothing and returns nil. This means two things: 1. If a directory to be created already exists, no error is returned. 2. If the error returned is IsExist (EEXIST), it means there exists a non-directory with the same name as MkdirAll need to use for directory. Example: we want to MkdirAll("a/b"), but file "a" (or "a/b") already exists, so MkdirAll fails. The above is a theory, based on quoted documentation and my UNIX knowledge. 3. In practice, though, current MkdirAll implementation [1] returns ENOTDIR in most of cases described in containers#2, with the exception when there is a race between MkdirAll and someone else creating the last component of MkdirAll argument as a file. In this very case MkdirAll() will indeed return EEXIST. Because of containers#1, IsExist check after MkdirAll is not needed. Because of containers#2 and containers#3, ignoring IsExist error is just plain wrong, as directory we require is not created. It's cleaner to report the error now. Note this error is all over the tree, I guess due to copy-paste, or trying to follow the same usage pattern as for Mkdir(), or some not quite correct examples on the Internet. > [1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go -end-quote- Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
kolyshkin
added a commit
to kolyshkin/buildah
that referenced
this pull request
Jun 11, 2020
This subtle bug keeps lurking in because error checking for `Mkdir()` and `MkdirAll()` is slightly different wrt `EEXIST`/`IsExist`: - for `Mkdir()`, `IsExist` error should (usually) be ignored (unless you want to make sure directory was not there before) as it means "the destination directory was already there"; - for `MkdirAll()`, `IsExist` error should NEVER be ignored. This commit removes ignoring the IsExist error, as it should not be ignored. [v2: skip patching (*Builder).Run] For more details, a quote from opencontainers/runc PR containers#162: -quote- TL;DR: check for IsExist(err) after a failed MkdirAll() is both redundant and wrong -- so two reasons to remove it. Quoting MkdirAll documentation: > MkdirAll creates a directory named path, along with any necessary > parents, and returns nil, or else returns an error. If path > is already a directory, MkdirAll does nothing and returns nil. This means two things: 1. If a directory to be created already exists, no error is returned. 2. If the error returned is IsExist (EEXIST), it means there exists a non-directory with the same name as MkdirAll need to use for directory. Example: we want to MkdirAll("a/b"), but file "a" (or "a/b") already exists, so MkdirAll fails. The above is a theory, based on quoted documentation and my UNIX knowledge. 3. In practice, though, current MkdirAll implementation [1] returns ENOTDIR in most of cases described in containers#2, with the exception when there is a race between MkdirAll and someone else creating the last component of MkdirAll argument as a file. In this very case MkdirAll() will indeed return EEXIST. Because of containers#1, IsExist check after MkdirAll is not needed. Because of containers#2 and containers#3, ignoring IsExist error is just plain wrong, as directory we require is not created. It's cleaner to report the error now. Note this error is all over the tree, I guess due to copy-paste, or trying to follow the same usage pattern as for Mkdir(), or some not quite correct examples on the Internet. > [1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go -end-quote- Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
kolyshkin
added a commit
to kolyshkin/buildah
that referenced
this pull request
Jul 1, 2020
This subtle bug keeps lurking in because error checking for `Mkdir()` and `MkdirAll()` is slightly different wrt `EEXIST`/`IsExist`: - for `Mkdir()`, `IsExist` error should (usually) be ignored (unless you want to make sure directory was not there before) as it means "the destination directory was already there"; - for `MkdirAll()`, `IsExist` error should NEVER be ignored. This commit removes ignoring the IsExist error, as it should not be ignored. [v2: skip patching (*Builder).Run] For more details, a quote from opencontainers/runc PR containers#162: -quote- TL;DR: check for IsExist(err) after a failed MkdirAll() is both redundant and wrong -- so two reasons to remove it. Quoting MkdirAll documentation: > MkdirAll creates a directory named path, along with any necessary > parents, and returns nil, or else returns an error. If path > is already a directory, MkdirAll does nothing and returns nil. This means two things: 1. If a directory to be created already exists, no error is returned. 2. If the error returned is IsExist (EEXIST), it means there exists a non-directory with the same name as MkdirAll need to use for directory. Example: we want to MkdirAll("a/b"), but file "a" (or "a/b") already exists, so MkdirAll fails. The above is a theory, based on quoted documentation and my UNIX knowledge. 3. In practice, though, current MkdirAll implementation [1] returns ENOTDIR in most of cases described in containers#2, with the exception when there is a race between MkdirAll and someone else creating the last component of MkdirAll argument as a file. In this very case MkdirAll() will indeed return EEXIST. Because of containers#1, IsExist check after MkdirAll is not needed. Because of containers#2 and containers#3, ignoring IsExist error is just plain wrong, as directory we require is not created. It's cleaner to report the error now. Note this error is all over the tree, I guess due to copy-paste, or trying to follow the same usage pattern as for Mkdir(), or some not quite correct examples on the Internet. > [1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go -end-quote- Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
rhatdan
added a commit
to rhatdan/buildah
that referenced
this pull request
Oct 31, 2020
This subtle bug keeps lurking in because error checking for `Mkdir()` and `MkdirAll()` is slightly different wrt `EEXIST`/`IsExist`: - for `Mkdir()`, `IsExist` error should (usually) be ignored (unless you want to make sure directory was not there before) as it means "the destination directory was already there"; - for `MkdirAll()`, `IsExist` error should NEVER be ignored. This commit removes ignoring the IsExist error, as it should not be ignored. [v2: skip patching (*Builder).Run] For more details, a quote from opencontainers/runc PR containers#162: -quote- TL;DR: check for IsExist(err) after a failed MkdirAll() is both redundant and wrong -- so two reasons to remove it. Quoting MkdirAll documentation: > MkdirAll creates a directory named path, along with any necessary > parents, and returns nil, or else returns an error. If path > is already a directory, MkdirAll does nothing and returns nil. This means two things: 1. If a directory to be created already exists, no error is returned. 2. If the error returned is IsExist (EEXIST), it means there exists a non-directory with the same name as MkdirAll need to use for directory. Example: we want to MkdirAll("a/b"), but file "a" (or "a/b") already exists, so MkdirAll fails. The above is a theory, based on quoted documentation and my UNIX knowledge. 3. In practice, though, current MkdirAll implementation [1] returns ENOTDIR in most of cases described in #2, with the exception when there is a race between MkdirAll and someone else creating the last component of MkdirAll argument as a file. In this very case MkdirAll() will indeed return EEXIST. Because of #1, IsExist check after MkdirAll is not needed. Because of #2 and #3, ignoring IsExist error is just plain wrong, as directory we require is not created. It's cleaner to report the error now. Note this error is all over the tree, I guess due to copy-paste, or trying to follow the same usage pattern as for Mkdir(), or some not quite correct examples on the Internet. > [1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go -end-quote- Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com> Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
rhatdan
added a commit
to rhatdan/buildah
that referenced
this pull request
Nov 3, 2020
This subtle bug keeps lurking in because error checking for `Mkdir()` and `MkdirAll()` is slightly different wrt `EEXIST`/`IsExist`: - for `Mkdir()`, `IsExist` error should (usually) be ignored (unless you want to make sure directory was not there before) as it means "the destination directory was already there"; - for `MkdirAll()`, `IsExist` error should NEVER be ignored. This commit removes ignoring the IsExist error, as it should not be ignored. [v2: skip patching (*Builder).Run] For more details, a quote from opencontainers/runc PR containers#162: -quote- TL;DR: check for IsExist(err) after a failed MkdirAll() is both redundant and wrong -- so two reasons to remove it. Quoting MkdirAll documentation: > MkdirAll creates a directory named path, along with any necessary > parents, and returns nil, or else returns an error. If path > is already a directory, MkdirAll does nothing and returns nil. This means two things: 1. If a directory to be created already exists, no error is returned. 2. If the error returned is IsExist (EEXIST), it means there exists a non-directory with the same name as MkdirAll need to use for directory. Example: we want to MkdirAll("a/b"), but file "a" (or "a/b") already exists, so MkdirAll fails. The above is a theory, based on quoted documentation and my UNIX knowledge. 3. In practice, though, current MkdirAll implementation [1] returns ENOTDIR in most of cases described in #2, with the exception when there is a race between MkdirAll and someone else creating the last component of MkdirAll argument as a file. In this very case MkdirAll() will indeed return EEXIST. Because of #1, IsExist check after MkdirAll is not needed. Because of #2 and #3, ignoring IsExist error is just plain wrong, as directory we require is not created. It's cleaner to report the error now. Note this error is all over the tree, I guess due to copy-paste, or trying to follow the same usage pattern as for Mkdir(), or some not quite correct examples on the Internet. > [1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go -end-quote- Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com> Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change updates the test helper macros to pass the right flags to tools to use the storage driver named in
$STORAGE_DRIVERinstead ofvfs, if the variable is set when the tests are run. It adds debug logging support to theimgtypetest helper, and ensures that it actually shuts down storage, even when it exits with errors.