Docker image for version 22.3.0 does not contain the scancode executable anymore

[sschuberth]

Describe the bug

The scancode executable is not available in the released Docker image anymore.

To Reproduce

Running

docker run --entrypoint scancode -it ghcr.io/oss-review-toolkit/ort:22.3.0

fails with

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "scancode": executable file not found in $PATH: unknown.

Expected behavior

The scancode executable should be contained in the Docker image and available in PATH. Locations should be

/opt/python/shims/scancode
/opt/python/versions/3.11.8/bin/scancode

Additional context

Last working version is 22.2.0:

docker run --entrypoint scancode -it ghcr.io/oss-review-toolkit/ort:22.2.0
Unable to find image 'ghcr.io/oss-review-toolkit/ort:22.2.0' locally
22.2.0: Pulling from oss-review-toolkit/ort
[...]
Digest: sha256:0ed58257387dc55e1e0c27b2314e50ee295ef6cb00192a48e1f039c804ae3feb
Status: Downloaded newer image for ghcr.io/oss-review-toolkit/ort:22.2.0
Usage: scancode [OPTIONS] <OUTPUT FORMAT OPTION(s)> <input>...
Try the 'scancode --help' option for help on options and arguments.

Error: Invalid value: At least one input path is required.

[sschuberth]

Interestingly, it works for me when building the image locally via docker build -t ort ., that image does contain scancode at /opt/python/shims/scancode.

Maybe we're just tagging the wrong image instead of something being wrong with the build?

[sschuberth]

The only possibly relevant change between 22.2.0...22.3.0 seems to be ae16d01, but I currently do not see how it could be related.

[heliocastro]

Should be, reverting it make image works again. I will try to analyze it afternoon

[heliocastro]

I think we have a bigger structural problem. Looking here, https://github.com/oss-review-toolkit/ort/actions/runs/9098861442/job/25010063746#step:11:155, is clear that our build on Github is pulling only the caches, disregarding the changes on the Dockerfile.
On my previous approach i added an invalidate switch to the older context process.
We need to make it similar to the new docker otherwise this problem will occurs when some changes happens that affects docker and we will not notice at all.
And this explains why local builds work.

[sschuberth]

is clear that our build on Github is pulling only the caches, disregarding the changes on the Dockerfile.

But why is that a problem? Just pulling / populating the cache should be no issue, because if the Dockerfile was modified, the cache entries should simply not be used, but relevant parts would be built from scratch. AT least that's how it should be.

[sschuberth]

Not sure what the issue was, but it's gone with release 22.5.0. I.e., running

docker run --entrypoint /opt/python/shims/scancode -it ghcr.io/oss-review-toolkit/ort:22.5.0

gives

Usage: scancode [OPTIONS] <OUTPUT FORMAT OPTION(s)> <input>...
Try the 'scancode --help' option for help on options and arguments.

Error: Invalid value: At least one input path is required.

as expected.

[heliocastro]

Actually there is an explanation.

If you look few commits before, @fviernau update some of python dependencies, which invalidated the cache for the "pythonbuild" layer.

Before, only the "python" layer was copied from cache, as "pythonbuild" layer was never regenerated.

"python" layer was cached before the ARG change, so when processed by the runtime layer is always a hit and miss.
After the change in some of dependencies, "pythonbuild" layer was built, "python" layer need to update itself, and then the image was fixed.

The only question remaining is why the cache missed scancode after the transition, need to keep an eye for future builds.

[sschuberth]

But the funny thing was that IIRC Python was present in the image, but just ScanCode was not...