COPY resolves symlinks it shouldn't try to resolve #1111

gilbsgilbs · 2020-03-06T14:33:48Z

Actual behavior

Since v0.18.0, COPY tries to resolve some symlinks it shouldn't try to resolve, which is a regression.

Expected behavior

I would expect kaniko to let those symlinks remain symlinks when copied in the image.

To Reproduce

For instance, we can create a symlink that points to nothing to make kaniko crash:

mkdir example
cd example/

echo "FROM alpine\nCOPY . /example" > Dockerfile
ln -s ./missing symlink

docker run --rm -it -v $(pwd):/workspace:ro gcr.io/kaniko-project/executor:debug --no-push

👇 👇 👇

[…]                           
error building image: error building stage: failed to execute command: lstat /workspace/missing: no such file or directory

If we create the symlink target, it builds successfully but the symlink is replaced with a plain file within the image:

touch missing
docker run --rm -it -v $(pwd):/workspace:ro --entrypoint='' gcr.io/kaniko-project/executor:debug /busybox/sh -c '/kaniko/executor --no-push && ls -l /example/'

👇 👇 👇

[…]
total 4
-rw-r--r--    1 1000     1000            29 Mar  6 14:28 Dockerfile
-rw-r--r--    1 1000     1000             0 Mar  6 14:28 missing
-rw-r--r--    1 1000     1000             0 Mar  6 14:28 symlink

Whereas in v0.17.1 (or using docker build) the symlink is not resolved:

$ docker run --rm -it -v $(pwd):/workspace:ro --entrypoint='' gcr.io/kaniko-project/executor:debug-v0.17.1 /busybox/sh -c '/kaniko/executor --no-push && ls -l /example/'
[…]
total 8
-rw-r--r--    1 1000     1000            29 Mar  6 14:29 Dockerfile
-rw-r--r--    1 1000     1000             0 Mar  6 14:29 missing
lrwxrwxrwx    1 root     root             9 Mar  6 14:29 symlink -> ./missing

Additional Information

This is a regression from v0.18.0. It worked properly in v0.17.1.

Note that Kaniko v0.18.0 changed the way symlinks work to better match actual Docker's behavior. That probably introduced this bug as a side-effect.

Triage Notes for the Maintainers

Description	Yes/No
Please check if this a new feature you are proposing
Please check if the build works in docker but not in kaniko
Please check if this error is seen when you use `--cache` flag
Please check if your dockerfile is a multistage dockerfile

The text was updated successfully, but these errors were encountered:

gilbsgilbs · 2020-03-06T16:37:13Z

FWIW, this was probably introduced by #943 .

tejal29 · 2020-03-07T00:05:08Z

@gilbsgilbs Thanks for the issue.

1. Copying a dead symlink throws an error.

We changed copy to be as close to docker build.
I verified your case on docker version 1.9.03

Dockerfile

tejaldesai@@tmep$ cat Dockerfile 
FROM scratch

COPY deadlink workdir/

My workdir

tejaldesai@@tmep$ ls -al 
total 24
drwxr-xr-x  5 tejaldesai primarygroup 4096 Mar  6 15:49 .
drwxr-x--- 11 tejaldesai primarygroup 4096 Mar  6 11:57 ..
lrwxrwxrwx  1 tejaldesai primarygroup    4 Mar  6 15:48 deadlink -> test

Dockerbuild log

tejaldesai@@tmep$ docker build . 
Sending build context to Docker daemon   7.68kB
Step 1/2 : FROM scratch
 ---> 
Step 2/2 : COPY deadlink workdir/
COPY failed: stat /usr/local/google/docker/tmp/docker-builder885367293/test: no such file or directory

2. Copying a single symlink creates a regular file.

We also made changes to make COPY behavior is mimic to docker build
Dockerfile

FROM scratch
COPY sym.link workdir/

My dir

tejaldesai@@tmep$ ls -al 
drwxr-xr-x  5 tejaldesai primarygroup 4096 Mar  6 15:56 .
drwxr-x--- 11 tejaldesai primarygroup 4096 Mar  6 11:57 ..
-rw-r--r--  1 tejaldesai primarygroup   37 Mar  6 15:56 Dockerfile
-rw-r--r--  1 tejaldesai primarygroup    6 Mar  6 15:56 hello
lrwxrwxrwx  1 tejaldesai primarygroup    5 Mar  6 15:56 sym.link -> hello

Run docker build

context: path ",." not found
tejaldesai@@tmep$ docker build . 
Sending build context to Docker daemon  8.704kB
Step 1/2 : FROM scratch
 ---> 
Step 2/2 : COPY sym.link workdir/
 ---> 3725f395af68
Successfully built 3725f395af68

Then, i exploded the built image.

tejaldesai@@tar$ tar -xvf 48233d291de8fc27f595bf188b1ed62de4ca3d3a562add526f302e8220e8385c/layer.tar 
workdir/
workdir/sym.link
tejaldesai@@tar$ ls -al workdir/
total 12
drwxr-xr-x 2 tejaldesai primarygroup 4096 Mar  6 15:57 .
drwxr-xr-x 4 tejaldesai primarygroup 4096 Mar  6 15:59 ..
-rw-r--r-- 1 tejaldesai primarygroup    6 Mar  6 15:56 sym.link
tejaldesai@@tar$

If you see the sym.link is a regular file.

3. Copying a directory with symlinks preserves symlinks

However, if you copy a directory, then symlinks are preserved.
Dockerfile

FROM scratch

COPY src/ workdir/

My dir

tejaldesai@@tmep$ ls -al src/
total 12
drwxr-xr-x 2 tejaldesai primarygroup 4096 Mar  6 15:19 .
drwxr-xr-x 4 tejaldesai primarygroup 4096 Mar  6 16:01 ..
lrwxrwxrwx 1 tejaldesai primarygroup    5 Mar  6 15:02 sym1.link -> test1
-rw-r--r-- 1 tejaldesai primarygroup   10 Mar  6 15:02 test1
lrwxrwxrwx 1 tejaldesai primarygroup   19 Mar  6 15:19 zSym.link -> ../anotherSrc/hello
tejaldesai@@tmep$ ls -al anotherSrc/
total 12
drwxr-xr-x 2 tejaldesai primarygroup 4096 Mar  6 15:18 .
drwxr-xr-x 4 tejaldesai primarygroup 4096 Mar  6 16:01 ..
-rw-r--r-- 1 tejaldesai primarygroup   14 Mar  6 15:18 hello
tejaldesai@@tmep$

Build Docker image

Sending build context to Docker daemon   7.68kB
Step 1/2 : FROM scratch
 ---> 
Step 2/2 : COPY src/ workdir/
 ---> Using cache
 ---> 0ab1ea7e4e3f
Successfully built 0ab1ea7e4e3f

Exploded image tar

tejaldesai@@tar$ ls -al workdir/
total 12
drwxr-xr-x 2 tejaldesai primarygroup 4096 Mar  6 15:19 .
drwxr-xr-x 4 tejaldesai primarygroup 4096 Mar  6 16:03 ..
lrwxrwxrwx 1 tejaldesai primarygroup    5 Mar  6 15:02 sym1.link -> test1
-rw-r--r-- 1 tejaldesai primarygroup   10 Mar  6 15:02 test1
lrwxrwxrwx 1 tejaldesai primarygroup   19 Mar  6 15:19 zSym.link -> ../anotherSrc/hello
tejaldesai@@tar$

In this case, zSymlink is a deadlink since anotherSrc wasn't copied over.

Copying a dir with symlink and dead symlinks
Dockerfile

FROM scratch

COPY src/ workdir/

My workdir

tejaldesai@@src$ ls -al 
total 12
drwxr-xr-x 2 tejaldesai primarygroup 4096 Mar  6 16:21 .
drwxr-xr-x 5 tejaldesai primarygroup 4096 Mar  6 16:20 ..
lrwxrwxrwx 1 tejaldesai primarygroup    1 Mar  6 16:21 deadlink -> t
lrwxrwxrwx 1 tejaldesai primarygroup    5 Mar  6 15:02 sym1.link -> test1
-rw-r--r-- 1 tejaldesai primarygroup   10 Mar  6 15:02 test1
lrwxrwxrwx 1 tejaldesai primarygroup   19 Mar  6 15:19 zSym.link -> ../anotherSrc/hello
tejaldesai@@src$

Docker build

Step 1/2 : FROM scratch
 ---> 
Step 2/2 : COPY src/ workdir/
 ---> 81a9999565db
Successfully built 81a9999565db

Exploded tar

tejaldesai@@tard$ ls -al workdir/
total 12
drwxr-xr-x 2 tejaldesai primarygroup 4096 Mar  6 16:23 .
drwxr-xr-x 4 tejaldesai primarygroup 4096 Mar  6 16:24 ..
lrwxrwxrwx 1 tejaldesai primarygroup    1 Mar  6 16:21 deadline -> t
lrwxrwxrwx 1 tejaldesai primarygroup    5 Mar  6 15:02 sym1.link -> test1
-rw-r--r-- 1 tejaldesai primarygroup   10 Mar  6 15:02 test1
lrwxrwxrwx 1 tejaldesai primarygroup   19 Mar  6 15:19 zSym.link -> ../anotherSrc/hello

WestonThayer · 2020-03-07T02:59:42Z

I'm also seeing build failures that are likely caused by this when using npm link during image build. All fine in v0.17.1.

tejal29 · 2020-03-07T05:10:48Z

@WestonThayer can you please provide your dockerfile?

I am not very familiar with npm link

gilbsgilbs · 2020-03-07T05:42:12Z

@tejal29 I agree that Kaniko is closer to Docker's behavior in the cases you described and that the title of this issue may be slightly incorrect (I'll fix it). But I insist, the case I gave in the original post still is a regression regarding docker 19.03 and kaniko 0.17.1 behavior. Kaniko v0.18.0 tries to resolve some symlinks it shouldn't resolve. The regression label should be kept. Are you unable to reproduce the two repro I gave in the original post?

Case 0: dead symlink

Setup

[nils@buks example]$ lsb_release -a
LSB Version:	1.4
Distributor ID:	Arch
Description:	Arch Linux
Release:	rolling
Codename:	n/a

[nils@buks example]$ uname -a
Linux buks 5.5.6-arch1-1 #1 SMP PREEMPT Mon, 24 Feb 2020 12:20:16 +0000 x86_64 GNU/Linux

[nils@buks example]$ docker --version
Docker version 19.03.6-ce, build 369ce74a3c

[nils@buks example]$ cat Dockerfile 
FROM alpine
COPY . /example

[nils@buks example]$ ls -la
total 4
drwxr-xr-x  2 nils nils  80 Mar  7 06:13 .
drwxrwxrwt 16 root root 360 Mar  7 06:23 ..
-rw-r--r--  1 nils nils  28 Mar  7 06:13 Dockerfile
lrwxrwxrwx  1 nils nils   9 Mar  7 06:13 symlink -> ./missing

docker build

works 👇

[nils@buks example]$ sudo docker build --no-cache -t testimage .
Sending build context to Docker daemon   2.56kB
Step 1/2 : FROM alpine
 ---> e7d92cdc71fe
Step 2/2 : COPY . /example
 ---> 881fa1ebd1ec
Successfully built 881fa1ebd1ec
Successfully tagged testimage:latest

[nils@buks example]$ sudo docker run --rm -it testimage ls -la /example
total 8
drwxr-xr-x    1 root     root            34 Mar  7 05:23 .
drwxr-xr-x    1 root     root           148 Mar  7 05:23 ..
-rw-r--r--    1 root     root            28 Mar  7 05:13 Dockerfile
lrwxrwxrwx    1 root     root             9 Mar  7 05:13 symlink -> ./missing

Kaniko v0.18.0

Crashes 👇

[nils@buks example]$ sudo docker run --rm -it -v $(pwd):/workspace:ro gcr.io/kaniko-project/executor:debug-v0.18.0 --no-push
INFO[0000] Resolved base name alpine to alpine          
INFO[0000] Resolved base name alpine to alpine          
INFO[0000] Retrieving image manifest alpine             
INFO[0001] Retrieving image manifest alpine             
INFO[0002] Built cross stage deps: map[]                
INFO[0002] Retrieving image manifest alpine             
INFO[0002] Retrieving image manifest alpine             
INFO[0003] Unpacking rootfs as cmd COPY . /example requires it. 
INFO[0004] Taking snapshot of full filesystem...        
INFO[0004] Resolving paths                              
INFO[0004] COPY . /example                              
error building image: error building stage: failed to execute command: lstat /workspace/missing: no such file or directory

Kaniko v0.17.1

works 👇

[nils@buks example]$ sudo docker run --rm -it -v $(pwd):/workspace:ro gcr.io/kaniko-project/executor:debug-v0.17.1 --no-push
INFO[0000] Resolved base name alpine to alpine          
INFO[0000] Resolved base name alpine to alpine          
INFO[0000] Retrieving image manifest alpine             
INFO[0001] Retrieving image manifest alpine             
INFO[0002] Built cross stage deps: map[]                
INFO[0002] Retrieving image manifest alpine             
INFO[0002] Retrieving image manifest alpine             
INFO[0003] Unpacking rootfs as cmd COPY . /example requires it. 
INFO[0003] Taking snapshot of full filesystem...        
INFO[0004] COPY . /example                              
INFO[0004] Taking snapshot of files...                  
INFO[0004] Skipping push to container registry due to --no-push flag

Case 1: good symlink

Environment

[nils@buks example]$ ls -la
total 4
drwxr-xr-x  2 nils nils 100 Mar  7 06:36 .
drwxrwxrwt 16 root root 360 Mar  7 06:36 ..
-rw-r--r--  1 nils nils  28 Mar  7 06:13 Dockerfile
-rw-r--r--  1 nils nils   0 Mar  7 06:36 missing
lrwxrwxrwx  1 nils nils   9 Mar  7 06:13 symlink -> ./missing

[nils@buks example]$ cat Dockerfile 
FROM alpine
COPY . /example

docker build

Doesn't crash and doesn't resolve the symlink

[nils@buks example]$ sudo docker build --no-cache -t testimage .
Sending build context to Docker daemon  3.072kB
Step 1/2 : FROM alpine
 ---> e7d92cdc71fe
Step 2/2 : COPY . /example
 ---> 89d33df5422b
Successfully built 89d33df5422b
Successfully tagged testimage:latest

[nils@buks example]$ sudo docker run --rm -it testimage ls -la /example
total 8
drwxr-xr-x    1 root     root            48 Mar  7 05:36 .
drwxr-xr-x    1 root     root           148 Mar  7 05:36 ..
-rw-r--r--    1 root     root            28 Mar  7 05:13 Dockerfile
-rw-r--r--    1 root     root             0 Mar  7 05:36 missing
lrwxrwxrwx    1 root     root             9 Mar  7 05:13 symlink -> ./missing

kaniko v0.18.0

Doesn't crash but resolves the symlink:

[nils@buks example]$ sudo docker run --rm -it -v $(pwd):/workspace:ro --entrypoint='' gcr.io/kaniko-project/executor:debug-v0.18.0 /busybox/sh -c '/kaniko/executor --no-push && ls -l /example/'
INFO[0000] Resolved base name alpine to alpine          
INFO[0000] Resolved base name alpine to alpine          
INFO[0000] Retrieving image manifest alpine             
INFO[0001] Retrieving image manifest alpine             
INFO[0002] Built cross stage deps: map[]                
INFO[0002] Retrieving image manifest alpine             
INFO[0002] Retrieving image manifest alpine             
INFO[0003] Unpacking rootfs as cmd COPY . /example requires it. 
INFO[0003] Taking snapshot of full filesystem...        
INFO[0003] Resolving paths                              
INFO[0003] COPY . /example                              
INFO[0003] Resolving paths                              
INFO[0003] Taking snapshot of files...                  
INFO[0003] Skipping push to container registry due to --no-push flag 
total 4
-rw-r--r--    1 1000     1000            28 Mar  7 05:39 Dockerfile
-rw-r--r--    1 1000     1000             0 Mar  7 05:39 missing
-rw-r--r--    1 1000     1000             0 Mar  7 05:39 symlink

kaniko v0.17.1

Doesn't crash and doesn't resolve the symlink:

[nils@buks example]$ sudo docker run --rm -it -v $(pwd):/workspace:ro --entrypoint='' gcr.io/kaniko-project/executor:debug-v0.17.1 /busybox/sh -c '/kaniko/executor --no-push && ls -l /example/'
INFO[0000] Resolved base name alpine to alpine          
INFO[0000] Resolved base name alpine to alpine          
INFO[0000] Retrieving image manifest alpine             
INFO[0001] Retrieving image manifest alpine             
INFO[0002] Built cross stage deps: map[]                
INFO[0002] Retrieving image manifest alpine             
INFO[0002] Retrieving image manifest alpine             
INFO[0003] Unpacking rootfs as cmd COPY . /example requires it. 
INFO[0003] Taking snapshot of full filesystem...        
INFO[0003] COPY . /example                              
INFO[0003] Taking snapshot of files...                  
INFO[0003] Skipping push to container registry due to --no-push flag 
total 8
-rw-r--r--    1 1000     1000            28 Mar  7 05:39 Dockerfile
-rw-r--r--    1 1000     1000             0 Mar  7 05:39 missing
lrwxrwxrwx    1 root     root             9 Mar  7 05:39 symlink -> ./missing

Let me know if I can provide any further information that may help you.

tejal29 · 2020-03-07T05:52:47Z

Are you unable to reproduce the two repro I gave in the original post?

yes i am. The reason why we made that change was to mimic what docker does and picked up the regression. I will work on fixing it and fixing the tests.

gilbsgilbs · 2020-03-07T05:57:18Z

Thanks. Sorry for the misunderstanding. Given that you removed the regression label and since your response sounded like “not a bug” to me, I wasn't sure you were actually able to reproduce 😄 .

tejal29 · 2020-03-07T07:56:03Z

No worries at all. I fixed up my PR to revert back to old behavior.
Can you try on your end to see if that fixes your issue?

gcr.io/kaniko-project/executor:debug
gcr.io/kaniko-project/executor:debug-edge

gilbsgilbs · 2020-03-07T08:10:13Z

Thank you. I can confirm this issue is fixed on debug-edge.

tejal29 · 2020-03-07T08:26:48Z

The integration test is failing probably due to incorrect permissions.
I will take a look tomorrow

WestonThayer · 2020-03-07T20:06:29Z

I can also confirm debug-edge is working for me.

I'm not having much luck getting this down to a minimal repro, but here's the general idea. Dockerfile:

FROM node:10-alpine

# Take advantage of staged Docker builds by taking care of npm installs
# separately

# This is a basic NPM package with it's own dependencies
WORKDIR /usr/src/packages/@company/backend-shared
COPY ./packages/@company/backend-shared/package*.json ./
RUN npm install

# This is a Node app that ran `npm install ../../packages/@company/backend-shared`,
# which creates a symlink from
# `/usr/src/services/backend/node_modules/@company/backend-shared`
# to
# `/usr/src/packages/@company/backend-shared`
WORKDIR /usr/src/services/backend/
COPY ./services/backend/package*.json ./
RUN npm install

# Copy dependency source and build
WORKDIR /usr/src/packages/@company/backend-shared
COPY ./packages/@company/backend-shared/ ./
RUN npm run build

# Copy source and build
# Crashes here
WORKDIR /usr/src/services/backend

Here's the error:

INFO[0024] WORKDIR /usr/src/services/backend/        
INFO[0024] cmd: workdir                                 
INFO[0024] Changed working directory to /usr/src/services/backend/ 
INFO[0024] Creating directory /usr/src/services/backend/ 
INFO[0024] Resolving paths                              
INFO[0024] Taking snapshot of files...                  
error building image: error building stage: failed to take snapshot: unable to add file /usr/src/services/backend/backend to layered map: error creating hash for /usr/src/services/backend/backend: lstat /usr/src/services/backend/backend: no such file or directory
ERROR
ERROR: build step 0 "gcr.io/kaniko-project/executor:latest" failed: step exited with non-zero status: 1

I tried reducing down to very basic Node modules and wasn't able to reproduce it. It could've been that I used a different path structure, different NPM dependencies, or that I was skipping the npm run build steps. Would be time consuming to track it down, but if the logs give you a clue I'd be happy to debug further down a specific path.

tejal29 · 2020-03-08T03:16:45Z

@WestonThayer Thanks for confirming!

tejal29 · 2020-03-09T21:17:07Z

INFO[0024] Taking snapshot of files...
error building image: error building stage: failed to take snapshot: unable to add file /usr/src/services/backend/backend to layered map: error creating hash for /usr/src/services/backend/backend: lstat /usr/src/services/backend/backend: no such file or directory
ERROR

Thanks @WestonThayer The error you are seeing is due to #1112 and not due to symlinks.

bramus · 2020-03-16T15:17:16Z

Also having symlink issues with an image containing a PHP app, where its dependency manager (Composer) will create symlinks to provided binaries.

When building using Docker itself, the files are – as expected – symlinked into vendor/bin/:

bramus in ~/repos/vbridge/pubsub-process-cctv on develop*
$ docker run -it --entrypoint /bin/bash "pubsub-process-cctv"
root@7e98449610d1:/# ls -Al /app/vendor/bin/
total 0
lrwxrwxrwx 1 www-data www-data 43 Mar 16 12:48 google-cloud-batch -> ../google/cloud/Core/bin/google-cloud-batch
lrwxrwxrwx 1 www-data www-data 50 Mar 16 12:48 google-cloud-debugger -> ../google/cloud/Debugger/bin/google-cloud-debugger
lrwxrwxrwx 1 www-data www-data 26 Mar 16 12:48 phpunit -> ../phpunit/phpunit/phpunit
lrwxrwxrwx 1 www-data www-data 30 Mar 16 12:57 pubsub-pull -> ../plonk/plonk/bin/pubsub-pull
root@7e98449610d1:/#

When building using kaniko, the files in the vendor/bin/ are no symlinks but actual copies:

bramus in ~/repos/vbridge/pubsub-process-cctv on develop*
$ docker run -it --entrypoint /bin/bash "eu.gcr.io/REDACTED/pubsub-process-cctv:develop"
root@afbe6246ea6d:/# ls -Al /app/vendor/bin/
total 16
-rwxr-xr-x 1 www-data www-data 1729 Mar 16 13:59 google-cloud-batch
-rwxr-xr-x 1 www-data www-data 2125 Mar 16 13:59 google-cloud-debugger
-rwxr-xr-x 1 www-data www-data 1443 Mar 16 13:59 phpunit
-rwxr-xr-x 1 www-data www-data 2030 Mar 16 14:51 pubsub-pull
root@afbe6246ea6d:/#

For reference, the Composer dependencies were built in a separate image, and then COPY'd into the main image (e.g. multi-stage build):

FROM composer:1.9.1 as composer
COPY composer.json composer.lock /app/
RUN composer install \
    --ignore-platform-reqs \
    --no-interaction \
    --no-plugins \
    --no-scripts \
    --prefer-dist \
    --optimize-autoloader \
    -vvv

…

FROM php:7.3-cli
COPY --chown=www-data:www-data --from=composer /app/vendor/ /app/vendor/
…

EDIT: Is this this issue, or is it #437 ?

tejal29 · 2020-03-19T20:11:54Z

@bramus Tried your issue on latest v0.19.0 build and it does not work. i will add a fix in next release.

docker run -it --entrypoint /busybox/sh -v /usr/local/google/home/tejaldesai/.config/gcloud:/root/.config/gcloud -v /usr/local/google/home/tejaldesai/workspace/kaniko/integration:/workspace gcr.io/kaniko-project/executor:debug
/ # /kaniko/executor -f dockerfiles/Dockerfile1 --context=dir://workspace --destination=gcr.io/tejal-test/test-symlink
...

sfynx · 2020-04-02T11:40:59Z

I got this minimal Dockerfile to fail on v0.18.0 and v0.19.0 when building with target two using --target two (building target one works fine though, so it seems the other target is somehow interfering). v0.17.1 is working fine so I suspect it must be related to this issue. Hope it is of any help for testing. The Dockerfile:

FROM alpine:3.11 AS base
RUN mkdir -p /dir/source /dir/target
RUN echo test > /dir/target/file.txt
WORKDIR /dir/source
RUN ln -s ../target/file.txt file.txt

FROM scratch as one
COPY --from=base /dir/ /dir/

FROM scratch as two
COPY --from=base /dir/ /dir/

Resulting log from base stage:

INFO[0001] Retrieving image manifest alpine:3.11        
INFO[0002] Built cross stage deps: map[0:[/dir/ /dir/]] 
INFO[0002] Retrieving image manifest alpine:3.11        
INFO[0003] Retrieving image manifest alpine:3.11        
INFO[0004] Unpacking rootfs as cmd RUN mkdir -p /dir/source /dir/target requires it. 
INFO[0004] Taking snapshot of full filesystem...        
INFO[0004] Resolving paths                              
INFO[0004] RUN mkdir -p /dir/source /dir/target         
INFO[0004] cmd: /bin/sh                                 
INFO[0004] args: [-c mkdir -p /dir/source /dir/target]  
INFO[0004] Taking snapshot of full filesystem...        
INFO[0005] Resolving paths                              
INFO[0005] RUN echo test > /dir/target/file.txt         
INFO[0005] cmd: /bin/sh                                 
INFO[0005] args: [-c echo test > /dir/target/file.txt]  
INFO[0005] Taking snapshot of full filesystem...        
INFO[0005] Resolving paths                              
INFO[0005] WORKDIR /dir/source                          
INFO[0005] cmd: workdir                                 
INFO[0005] Changed working directory to /dir/source     
INFO[0005] RUN ln -s ../target/file.txt file.txt        
INFO[0005] cmd: /bin/sh                                 
INFO[0005] args: [-c ln -s ../target/file.txt file.txt] 
INFO[0005] Taking snapshot of full filesystem...        
INFO[0005] Resolving paths                              
INFO[0005] Saving file /dir/ for later use              
INFO[0005] Saving file /dir/ for later use              
error building image: symlink ../target/file.txt /kaniko/0/dir/source/file.txt: file exists

tejal29 · 2020-05-07T23:41:09Z

I tried the dockefile in #1111 (comment)
This was a easy fix where we did not dedupe the files to saved
#1252
Will try to get in next release.

tejal29 · 2020-05-07T23:53:11Z

The issue in #1111 (comment) is still not fixed.

bramus · 2020-05-08T07:34:42Z

The issue in #1111 (comment) is still not fixed.

Do I create a separate issue?

It tries to resolve symlinks and copy target file. If it's a dangling symlink, no cigar, docker kaputs. So solution (workaround?), copy the parent directory. Apparently then, the symlinks are copied as is. GoogleContainerTools/kaniko#1111 (comment) moby/moby#40449 (comment)

tejal29 added the regression label Mar 6, 2020

tejal29 removed the regression label Mar 7, 2020

tejal29 mentioned this issue Mar 7, 2020

Add more tests for Copy and some fixes. #1114

Merged

tejal29 self-assigned this Mar 7, 2020

gilbsgilbs changed the title ~~COPY should not resolve symlinks~~ COPY resolves symlinks it shouldn't try to resolve Mar 7, 2020

tejal29 added the regression label Mar 7, 2020

tejal29 mentioned this issue Mar 24, 2020

Refactor Kaniko to test across multistages #1155

Merged

ewah mentioned this issue Apr 9, 2020

COPY is bombing on dangling symlinks #1186

Closed

tejal29 mentioned this issue May 7, 2020

remove duplicates save for the same dir #1252

Merged

4 tasks

tejal29 added the area/symlinks label May 7, 2020

tejal29 closed this as completed in #1252 May 8, 2020

dtrudg mentioned this issue Jul 15, 2021

copying symbolic link with absolute target path from previous stage fails sylabs/singularity#159

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

COPY resolves symlinks it shouldn't try to resolve #1111

COPY resolves symlinks it shouldn't try to resolve #1111

gilbsgilbs commented Mar 6, 2020 •

edited

Loading

gilbsgilbs commented Mar 6, 2020

tejal29 commented Mar 7, 2020 •

edited

Loading

WestonThayer commented Mar 7, 2020

tejal29 commented Mar 7, 2020 •

edited

Loading

gilbsgilbs commented Mar 7, 2020 •

edited

Loading

tejal29 commented Mar 7, 2020

gilbsgilbs commented Mar 7, 2020

tejal29 commented Mar 7, 2020

gilbsgilbs commented Mar 7, 2020

tejal29 commented Mar 7, 2020

WestonThayer commented Mar 7, 2020

tejal29 commented Mar 8, 2020

tejal29 commented Mar 9, 2020

bramus commented Mar 16, 2020 •

edited

Loading

tejal29 commented Mar 19, 2020

sfynx commented Apr 2, 2020 •

edited

Loading

tejal29 commented May 7, 2020

tejal29 commented May 7, 2020

bramus commented May 8, 2020

COPY resolves symlinks it shouldn't try to resolve #1111

COPY resolves symlinks it shouldn't try to resolve #1111

Comments

gilbsgilbs commented Mar 6, 2020 • edited Loading

gilbsgilbs commented Mar 6, 2020

tejal29 commented Mar 7, 2020 • edited Loading

1. Copying a dead symlink throws an error.

2. Copying a single symlink creates a regular file.

3. Copying a directory with symlinks preserves symlinks

WestonThayer commented Mar 7, 2020

tejal29 commented Mar 7, 2020 • edited Loading

gilbsgilbs commented Mar 7, 2020 • edited Loading

Case 0: dead symlink

Setup

docker build

Kaniko v0.18.0

Kaniko v0.17.1

Case 1: good symlink

Environment

docker build

kaniko v0.18.0

kaniko v0.17.1

tejal29 commented Mar 7, 2020

gilbsgilbs commented Mar 7, 2020

tejal29 commented Mar 7, 2020

gilbsgilbs commented Mar 7, 2020

tejal29 commented Mar 7, 2020

WestonThayer commented Mar 7, 2020

tejal29 commented Mar 8, 2020

tejal29 commented Mar 9, 2020

bramus commented Mar 16, 2020 • edited Loading

tejal29 commented Mar 19, 2020

sfynx commented Apr 2, 2020 • edited Loading

tejal29 commented May 7, 2020

tejal29 commented May 7, 2020

bramus commented May 8, 2020

gilbsgilbs commented Mar 6, 2020 •

edited

Loading

tejal29 commented Mar 7, 2020 •

edited

Loading

tejal29 commented Mar 7, 2020 •

edited

Loading

gilbsgilbs commented Mar 7, 2020 •

edited

Loading

bramus commented Mar 16, 2020 •

edited

Loading

sfynx commented Apr 2, 2020 •

edited

Loading