Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

USER command should not require user or group IDs to exist #477

Closed
hrobertson opened this issue Dec 4, 2018 · 30 comments
Closed

USER command should not require user or group IDs to exist #477

hrobertson opened this issue Dec 4, 2018 · 30 comments
Labels
area/dockerfile-command For all bugs related to dockerfile file commands area/uid categorized differs-from-docker good first issue Good for newcomers issue/unknown-user issue/USER kind/bug Something isn't working kind/enhancement New feature or request priority/p0 Highest priority. Break user flow. We are actively looking at delivering it. works-with-docker

Comments

@hrobertson
Copy link

hrobertson commented Dec 4, 2018

It is common for a Dockerfile to contain a USER directive of the form USER 1000:1000. docker build does not require a user and group with those ids to exist, nor does it create them. However, Kaniko errors.

Interestingly, while creating a minimal dockerfile to demonstrate this I discovered that Kaniko errors in two different ways:

Working example using Docker:

$ cat usertest1
FROM alpine
USER 1000:1000
CMD ["whoami"]
$ docker build --file usertest1 -t usertest:1 .
Sending build context to Docker daemon  23.46MB
Step 1/3 : FROM alpine
 ---> 196d12cf6ab1
Step 2/3 : USER 1000:1000
 ---> Using cache
 ---> 4794c572f720
Step 3/3 : CMD ["whoami"]
 ---> Using cache
 ---> 6ab640ab2641
Successfully built 6ab640ab2641
Successfully tagged usertest:1
$ docker run --rm usertest:1
whoami: unknown uid 1000

Kaniko failure mode 1

$ cat usertest1
FROM alpine
USER 1000:1000
CMD ["whoami"]

$ docker run --rm -it -v $PWD:/workspace gcr.io/kaniko-project/executor:latest --dockerfile=usertest1 --no-push --context=dir:///workspace
INFO[0000] Downloading base image alpine                
2018/12/04 13:25:52 No matching credentials were found, falling back on anonymous
INFO[0001] Taking snapshot of full filesystem...        
INFO[0001] Skipping paths under /dev, as it is a whitelisted directory 
INFO[0001] Skipping paths under /kaniko, as it is a whitelisted directory 
INFO[0001] Skipping paths under /proc, as it is a whitelisted directory 
INFO[0001] Skipping paths under /sys, as it is a whitelisted directory 
INFO[0001] Skipping paths under /workspace, as it is a whitelisted directory 
INFO[0001] USER 1000:1000                               
INFO[0001] cmd: USER                                    
error building image: error building stage: open /etc/passwd: no such file or directory

Kaniko failure mode 2:
Note the addition of the RUN directive and the different error message from Kaniko

$ cat usertest2
FROM alpine
RUN test 0
USER 1000:1000
CMD ["whoami"]

$ docker run --rm -it -v $PWD:/workspace gcr.io/kaniko-project/executor:latest --dockerfile=usertest2 --no-push --context=dir:///workspace
INFO[0000] Downloading base image alpine                
2018/12/04 13:32:00 No matching credentials were found, falling back on anonymous
INFO[0001] Unpacking rootfs as cmd RUN test 0 requires it. 
INFO[0001] Taking snapshot of full filesystem...        
INFO[0001] Skipping paths under /dev, as it is a whitelisted directory 
INFO[0001] Skipping paths under /kaniko, as it is a whitelisted directory 
INFO[0001] Skipping paths under /proc, as it is a whitelisted directory 
INFO[0001] Skipping paths under /sys, as it is a whitelisted directory 
INFO[0001] Skipping paths under /workspace, as it is a whitelisted directory 
INFO[0001] RUN test 0                                   
INFO[0001] cmd: /bin/sh                                 
INFO[0001] args: [-c test 0]                            
INFO[0001] Taking snapshot of full filesystem...        
INFO[0001] Skipping paths under /dev, as it is a whitelisted directory 
INFO[0001] Skipping paths under /kaniko, as it is a whitelisted directory 
INFO[0001] Skipping paths under /proc, as it is a whitelisted directory 
INFO[0001] Skipping paths under /sys, as it is a whitelisted directory 
INFO[0001] Skipping paths under /workspace, as it is a whitelisted directory 
INFO[0001] No files were changed, appending empty layer to config. No layer added to image. 
INFO[0001] USER 1000:1000                               
INFO[0001] cmd: USER                                    
error building image: error building stage: user: unknown userid 1000
@miguelitoq76
Copy link

@hrobertson could be so kind to test the build with this modification?

add a FROM scratch at the top of the dockerfile

FROM scratch
FROM alpine
USER 1000:1000
CMD ["whoami"]

Please post your result thanks a lot, it would help to find the error!!

@priyawadhwa priyawadhwa added the kind/bug Something isn't working label Dec 4, 2018
@hrobertson
Copy link
Author

@miguelitoq76 I assume you didn't mean to have two FROMs?

$ cat usertest3
FROM scratch
USER 1000:1000
CMD ["whoami"]
$
$ docker run --rm -it -v $PWD:/workspace gcr.io/kaniko-project/executor:latest --dockerfile=usertest3 --no-push --context=dir:///workspace
INFO[0000] No base image, nothing to extract            
INFO[0000] Taking snapshot of full filesystem...        
INFO[0000] Skipping paths under /dev, as it is a whitelisted directory 
INFO[0000] Skipping paths under /kaniko, as it is a whitelisted directory 
INFO[0000] Skipping paths under /proc, as it is a whitelisted directory 
INFO[0000] Skipping paths under /sys, as it is a whitelisted directory 
INFO[0000] Skipping paths under /workspace, as it is a whitelisted directory 
INFO[0000] USER 1000:1000                               
INFO[0000] cmd: USER                                    
error building image: error building stage: open /etc/passwd: no such file or directory

In case you really did mean to have the two FROMs:

$ cat usertest4
FROM scratch
FROM alpine
USER 1000:1000
CMD ["whoami"]
$
$ docker run --rm -it -v $PWD:/workspace gcr.io/kaniko-project/executor:latest --dockerfile=usertest4 --no-push --context=dir:///workspace
INFO[0000] No base image, nothing to extract            
INFO[0000] Taking snapshot of full filesystem...        
INFO[0000] Skipping paths under /dev, as it is a whitelisted directory 
INFO[0000] Skipping paths under /kaniko, as it is a whitelisted directory 
INFO[0000] Skipping paths under /proc, as it is a whitelisted directory 
INFO[0000] Skipping paths under /sys, as it is a whitelisted directory 
INFO[0000] Skipping paths under /workspace, as it is a whitelisted directory 
INFO[0000] Deleting filesystem...                       
INFO[0000] Downloading base image alpine                
2018/12/05 12:44:45 No matching credentials were found, falling back on anonymous
INFO[0002] Taking snapshot of full filesystem...        
INFO[0002] Skipping paths under /dev, as it is a whitelisted directory 
INFO[0002] Skipping paths under /kaniko, as it is a whitelisted directory 
INFO[0002] Skipping paths under /proc, as it is a whitelisted directory 
INFO[0002] Skipping paths under /sys, as it is a whitelisted directory 
INFO[0002] Skipping paths under /workspace, as it is a whitelisted directory 
INFO[0002] USER 1000:1000                               
INFO[0002] cmd: USER                                    
error building image: error building stage: open /etc/passwd: no such file or directory

@hrobertson
Copy link
Author

hrobertson commented Dec 5, 2018

I have done some further investigation and the behaviour is even more erroneous than I first thought!

Here I build three images containing the kaniko executor binary. One pulls the binary into a scratch base, the next into an alpine base, and the next creates a user within the image.

$ cat Dockerfile
FROM gcr.io/kaniko-project/executor:latest as kaniko

FROM scratch as scratch
COPY --from=kaniko /kaniko/executor /kaniko/executor
COPY --from=kaniko /kaniko/ssl/certs/ca-certificates.crt /kaniko/ssl/certs/
ENV SSL_CERT_DIR=/kaniko/ssl/certs
WORKDIR /workspace

ENTRYPOINT ["/kaniko/executor"]

FROM alpine as alpine
COPY --from=kaniko /kaniko/executor /kaniko/executor
COPY --from=kaniko /kaniko/ssl/certs/ca-certificates.crt /kaniko/ssl/certs/
ENV SSL_CERT_DIR=/kaniko/ssl/certs
WORKDIR /workspace

ENTRYPOINT ["/kaniko/executor"]

FROM alpine as alpine-with-user
RUN addgroup -g 1000 test && adduser -D -u 1000 -G test test

I build the three separate images:

$ docker build -t kaniko:scratch --target scratch .
$ docker build -t kaniko:alpine --target alpine .
$ docker build -t kaniko:alpine-with-user --target alpine-with-user .

And here I build the usertest3 image using kaniko inside each of those images.

$ cat usertest3
FROM scratch
USER 1000:1000
CMD ["whoami"]

kaniko binary in scratch

$ docker run --rm -it -v $PWD:/workspace --entrypoint=/kaniko/executor kaniko:scratch --dockerfile=usertest3 --context=dir:///workspace --no-push
INFO[0000] No base image, nothing to extract            
INFO[0000] Taking snapshot of full filesystem...        
INFO[0000] Skipping paths under /dev, as it is a whitelisted directory 
INFO[0000] Skipping paths under /kaniko, as it is a whitelisted directory 
INFO[0000] Skipping paths under /proc, as it is a whitelisted directory 
INFO[0000] Skipping paths under /sys, as it is a whitelisted directory 
INFO[0000] Skipping paths under /workspace, as it is a whitelisted directory 
INFO[0000] USER 1000:1000                               
INFO[0000] cmd: USER                                    
error building image: error building stage: open /etc/passwd: no such file or directory

kaniko binary in alpine

$ docker run --rm -it -v $PWD/test:/workspace --entrypoint=/kaniko/executor kaniko:alpine --dockerfile=usertest3 --context=dir:///workspace --no-push 
INFO[0000] No base image, nothing to extract            
INFO[0000] Taking snapshot of full filesystem...        
INFO[0000] Skipping paths under /dev, as it is a whitelisted directory 
INFO[0000] Skipping paths under /kaniko, as it is a whitelisted directory 
INFO[0000] Skipping paths under /proc, as it is a whitelisted directory 
INFO[0000] Skipping paths under /sys, as it is a whitelisted directory 
INFO[0000] Skipping paths under /var/run, as it is a whitelisted directory 
INFO[0000] Skipping paths under /workspace, as it is a whitelisted directory 
INFO[0000] USER 1000:1000                               
INFO[0000] cmd: USER                                    
error building image: error building stage: user: unknown userid 1000

As you can see, kaniko is erroneously looking for the /etc/passwd file in the environment in which kaniko is running, not the filesystem of the image it's building! Note it shouldn't even look for a passwd file at all when IDs are specified rather than names.

kaniko binary in alpine with an existing user:
Note this user does not exist in the image kaniko is building, but in the environment in which kaniko itself is running.

$ docker run -it --rm -v $PWD:/workspace kaniko:alpine-with-user --dockerfile=usertest3 --context=dir:///workspace --no-push
INFO[0000] Downloading base image alpine                
2018/12/05 13:32:25 Unable to read "/root/.docker/config.json": open /root/.docker/config.json: no such file or directory
2018/12/05 13:32:25 No matching credentials were found, falling back on anonymous
INFO[0001] Taking snapshot of full filesystem...        
INFO[0001] Skipping paths under /dev, as it is a whitelisted directory 
INFO[0001] Skipping paths under /kaniko, as it is a whitelisted directory 
INFO[0001] Skipping paths under /proc, as it is a whitelisted directory 
INFO[0001] Skipping paths under /sys, as it is a whitelisted directory 
INFO[0001] Skipping paths under /var/run, as it is a whitelisted directory 
INFO[0001] Skipping paths under /workspace, as it is a whitelisted directory 
INFO[0001] USER 1000:1000                               
INFO[0001] cmd: USER                                    
INFO[0001] CMD ["whoami"]                               
INFO[0001] Skipping push to container registry due to --no-push flag

Until the first RUN directive causes Unpacking rootfs , kaniko is using the filesystem of the environment that Kaniko is in rather than that of the base image.
As you can see below, using kaniko in the kaniko:alpine-with-user image to build the usertest2 dockerfile (see the initial post), again fails to find the user. Contrast this with the above output building the usertest3 dockerfile.

$ docker run -it --rm -v $PWD:/workspace kaniko:alpine-with-user --dockerfile=usertest2 --context=dir:///workspace --no-push
INFO[0000] Downloading base image alpine                
2018/12/05 19:12:18 No matching credentials were found, falling back on anonymous
INFO[0001] Unpacking rootfs as cmd RUN test 0 requires it. 
INFO[0001] Taking snapshot of full filesystem...        
INFO[0001] Skipping paths under /dev, as it is a whitelisted directory 
INFO[0001] Skipping paths under /kaniko, as it is a whitelisted directory 
INFO[0001] Skipping paths under /proc, as it is a whitelisted directory 
INFO[0001] Skipping paths under /sys, as it is a whitelisted directory 
INFO[0001] Skipping paths under /var/run, as it is a whitelisted directory 
INFO[0001] Skipping paths under /workspace, as it is a whitelisted directory 
INFO[0002] RUN test 0                                   
INFO[0002] cmd: /bin/sh                                 
INFO[0002] args: [-c test 0]                            
INFO[0002] Taking snapshot of full filesystem...        
INFO[0002] Skipping paths under /dev, as it is a whitelisted directory 
INFO[0002] Skipping paths under /kaniko, as it is a whitelisted directory 
INFO[0002] Skipping paths under /proc, as it is a whitelisted directory 
INFO[0002] Skipping paths under /sys, as it is a whitelisted directory 
INFO[0002] Skipping paths under /var/run, as it is a whitelisted directory 
INFO[0002] Skipping paths under /workspace, as it is a whitelisted directory 
INFO[0002] No files were changed, appending empty layer to config. No layer added to image. 
INFO[0002] USER 1000:1000                               
INFO[0002] cmd: USER                                    
error building image: error building stage: user: unknown userid 1000

@MMeent
Copy link
Contributor

MMeent commented Dec 13, 2018

It looks like this is also an issue when using --cache=true: When reaching a USER statement on a cached layer that is not available locally, it fails:

INFO[0000] Downloading base image openjdk:11            
2018/12/13 10:41:49 No matching credentials were found, falling back on anonymous
WARN[0001] Error while retrieving image from cache: getting image from path: open /cache/sha256:c7381bfd53670f1211314885b03b98f5e13fddf6958afeec61092b07c56ddef1: no such file or directory 
2018/12/13 10:41:50 No matching credentials were found, falling back on anonymous
INFO[0002] Executing 0 build triggers                   
INFO[0002] Checking for cached layer docker.company.io/project/cache:3bee254e641f46abcfa0822e3d1600938df2905e0aadba41756a7c6884cbd16d... 
INFO[0002] Using caching version of cmd: RUN groupadd -r $APPLICATION_USER && useradd --no-log-init -r -g $APPLICATION_USER $APPLICATION_USER 
INFO[0002] Checking for cached layer docker.company.io/project/cache:a1875d7dd74f1d18a37dab3e6e75f2f3f08cc58f1ded0d6d92e35647883c0686... 
INFO[0002] Using caching version of cmd: RUN mkdir /app 
INFO[0002] Checking for cached layer docker.company.io/project/cache:b9fafa07e45363ebd9db5304781b7536e15ad7bf028cdd60867fc55151c3cb4f... 
error building image: error building stage: open /etc/passwd: no such file or directory
INFO[0002] Using caching version of cmd: RUN chown -R $APPLICATION_USER /app 
INFO[0002] cmd: USER                                    

@keaaa
Copy link

keaaa commented Jan 3, 2019

Anyone been able to work around this?

@pbuszka
Copy link

pbuszka commented Jan 6, 2019

I encountered the same problem with a Dockerfile which has relevant lines like:

FROM MyCustomImage
...
FROM node:8.11.4-slim
USER node
...

and it fails with the same problem as above.

error building image: error building stage: open /etc/passwd: no such file or directory

I think it is a regression introduced somewhere between 3 Oct and 15 Nov. I use kaniko invoked from skaffold and skaffold v0.18 works fine (it uses kaniko v0.4 from 3 Oct) but v0.19 fails (it uses kaniko @ commit 0c29413 from Nov 15). skaffold v20 with kaniko v0.7 fails as well with the same problem.

Maybe this might help to pinpoint the problem.

pmcq referenced this issue in algorithmiaio/kaniko Jan 8, 2019
…s for chowning purposes, otherwise /etc/passwd isn't found, or wouldn't have the users/groups from the base image
@pmcq
Copy link

pmcq commented Jan 8, 2019

I think it's just this line

userObj, err = user.LookupId(userStr)
that calls os.user.LookupId which results in trying to read from /etc/passwd and similarly
userObj, err = user.LookupId(userStr)
does this for the group id.

Since all the code does is gets a User object and then returns the UID (which would match the userStr that we looked up in the first place) I feel like the correct behavior would be something like:

  1. If userStr is numeric, just use that as a userId (optionally try user.LookupId(userStr) and log a warning or something if the userid isn't found?)
  2. If it is not numeric string try looking up userStr with user.Lookup, and if it is not found then return an error
    does that seem like it would give consistent behavior?

edit: playing around with this a little bit one issue seems to be that environment variables like HOME won't be set automatically if you use a numeric userid but that can be worked around if needed.

@everflux
Copy link

everflux commented Feb 2, 2019

Maybe unrelated, but using
USER 0
to switch to the root user did not work without any error message. (I would have expected a similar message as when using USER root)

@mamoit
Copy link

mamoit commented Feb 6, 2019

It happened to me with the cache enabled, as soon as I disabled it showed Unpacking rootfs as cmd USER root requires it and started working.

@matti
Copy link

matti commented Feb 18, 2019

same as @mamoit building first without USER & cache enabled and then using USER will trigger this

@timowuttke
Copy link

For me, a USER command in a cached layer will produce the already mentioned error message. Is there some kind of workaround for this?

@aberres
Copy link

aberres commented Feb 21, 2019

Happens for us as well and stops us from using Kaniko.

@marcomancuso
Copy link

Any update on this?

@geekofalltrades
Copy link

geekofalltrades commented Apr 5, 2019

I think I was able to work around this just now by adding a minimal /etc/passwd to my container before the USER instruction.

Dockerfile:

ARG UPSTREAM_IMAGE=prom/alertmanager
ARG UPSTREAM_TAG=v0.15.3

FROM $UPSTREAM_IMAGE:$UPSTREAM_TAG as upstream

FROM scratch

LABEL project=<my-project>

COPY --from=upstream /bin/alertmanager /bin/alertmanager
COPY --from=upstream /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=upstream /etc/nsswitch.conf /etc/nsswitch.conf

# Work around Kaniko bug.
COPY --chown=0:0 passwd /etc/passwd

EXPOSE 9093

USER 4191718

ENTRYPOINT ["/bin/alertmanager"]
CMD        [ "--config.file=/etc/alertmanager/config.yml", \
             "--storage.path=/alertmanager" ]

passwd:

alertmanager:x:4191718:4191718:Alertmanager Service User::/bin/false

Kaniko build log:

INFO[0000] Downloading base image prom/alertmanager:v0.15.3 
2019/04/05 20:05:50 No matching credentials were found, falling back on anonymous
INFO[0001] Error while retrieving image from cache: getting file info: stat /cache/sha256:196af0317d3449c1300aa26ff0366f68c67d04581d2c9f8609cbb227424e226c: no such file or directory 
INFO[0001] Downloading base image prom/alertmanager:v0.15.3 
2019/04/05 20:05:51 No matching credentials were found, falling back on anonymous
INFO[0001] Executing 0 build triggers                   
INFO[0001] Skipping unpacking as no commands require it. 
INFO[0001] Taking snapshot of full filesystem...        
INFO[0001] Storing source image from stage 0 at path /kaniko/stages/0 
INFO[0003] Deleting filesystem...                       
INFO[0003] No base image, nothing to extract            
INFO[0003] Unpacking rootfs as cmd USER 4191718 requires it. 
INFO[0003] Taking snapshot of full filesystem...        
INFO[0004] LABEL project=<my-project>                       
INFO[0004] Applying label project=<my-project>              
INFO[0004] COPY --from=upstream /bin/alertmanager /bin/alertmanager 
INFO[0004] Taking snapshot of files...                  
INFO[0004] COPY --from=upstream /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt 
INFO[0004] Taking snapshot of files...                  
INFO[0004] COPY --from=upstream /etc/nsswitch.conf /etc/nsswitch.conf 
INFO[0004] Taking snapshot of files...                  
INFO[0004] Using files from context: [/builds/wmd/hopcloud-thirdparty/hopcloud-alertmanager/passwd] 
INFO[0004] COPY --chown=0:0 passwd /etc/passwd          
INFO[0004] Taking snapshot of files...                  
INFO[0004] EXPOSE 9093                                  
INFO[0004] cmd: EXPOSE                                  
INFO[0004] Adding exposed port: 9093/tcp                
INFO[0004] USER 4191718                                 
INFO[0004] cmd: USER                                    
INFO[0004] ENTRYPOINT ["/bin/alertmanager"]             
INFO[0004] CMD        [ "--config.file=/etc/alertmanager/config.yml",              "--storage.path=/alertmanager" ] 
2019/04/05 20:05:55 pushed blob sha256:755f71954f5678ea5d7abaa9324959e536067a51332c7fcb9e1832d47b3d0fec
2019/04/05 20:05:55 pushed blob sha256:f74df0e0e91fe2c111555688756efeb59cfb29d4d75ab90b7fbcf6bed6bbdae6
2019/04/05 20:05:55 pushed blob sha256:3da111e28cadb8bb46f10bed9de53f0de7cbb74dc4fd22db1ded8ab81f52c206
2019/04/05 20:05:55 pushed blob sha256:9e57d8d1fbc22639da8d7019ea323f99fd639a1520312fc65f065f524959375f
2019/04/05 20:05:57 pushed blob sha256:83ebe99b26904b90173246b81fa7a9ea6f65f021f6350d2eb5303dc75e553508
2019/04/05 20:05:58 <my-registry>/<my-repo>:<my-tag>: digest: sha256:5c1b3eab5f386f9722c50d7b795dc89220184a35751861ad6e73aa148699296c size: 912

(This Dockerfile is taking Prometheus' alertmanager, copying the binary and some other important stuff into a scratch container, and forcing it to run as a non-root user. Stripping down third-party images like this is part of the security process where I work.)

This is a clunky workaround and I'd still appreciate this bug being fixed.

@rnsv
Copy link

rnsv commented Apr 28, 2019

Using gcr.io/kaniko-project/executor:debug allowed me to workaround this issue

When using executor:latest

INFO[0000] USER 10400:10400
INFO[0000] cmd: USER
error building image: error building stage: open /etc/passwd: no such file or directory

When using executor:debug

INFO[0002] Taking snapshot of full filesystem...
INFO[0002] USER 10400:10400
INFO[0002] cmd: USER
2019/04/28 07:34:31 existing blob: sha256:67ddbfb20a22d7c0ea0df56

@djuarezg
Copy link

djuarezg commented Apr 29, 2019

Suffering from this as well on gcr.io/kaniko-project/executor:debug-v0.9.0 :

INFO[0000] cmd: USER                                    
error building image: error building stage: open /etc/passwd: no such file or directory

With the latest version:

error building image: error building stage: user: invalid userid <username>

@gugahoi
Copy link

gugahoi commented May 11, 2019

Using gcr.io/kaniko-project/executor:debug allowed me to get past this issue when running both locally and on Cloud Build. Shasum: 5a6b3193506212ea7703214bf3b21cd898e5e2ec14d6b988cc9d11a9b34f32d4

@tejal29
Copy link
Member

tejal29 commented Aug 9, 2019

@hrobertson, i tried reproducing error cases Failure 1 and failure 2. I don't see this error anymore for gcr.io/kaniko-project/executor:latest

tejaldesai@ $ kubectl logs kaniko -f
INFO[0000] Resolved base name alpine to alpine          
INFO[0000] Resolved base name alpine to alpine          
INFO[0000] Downloading base image alpine                
ERROR: logging before flag.Parse: E0809 05:31:37.759262       1 metadata.go:142] while reading 'google-dockercfg' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg
ERROR: logging before flag.Parse: E0809 05:31:37.764295       1 metadata.go:159] while reading 'google-dockercfg-url' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg-url
2019/08/09 05:31:37 No matching credentials were found, falling back on anonymous
INFO[0000] Error while retrieving image from cache: getting file info: stat /cache/sha256:57334c50959f26ce1ee025d08f136c2292c128f84e7b229d1b0da5dac89e9866: no such file or directory 
INFO[0000] Downloading base image alpine                
2019/08/09 05:31:38 No matching credentials were found, falling back on anonymous
INFO[0001] Built cross stage deps: map[]                
INFO[0001] Downloading base image alpine                
2019/08/09 05:31:38 No matching credentials were found, falling back on anonymous
INFO[0001] Error while retrieving image from cache: getting file info: stat /cache/sha256:57334c50959f26ce1ee025d08f136c2292c128f84e7b229d1b0da5dac89e9866: no such file or directory 
INFO[0001] Downloading base image alpine                
2019/08/09 05:31:39 No matching credentials were found, falling back on anonymous
INFO[0002] Unpacking rootfs as cmd RUN test 0 requires it. 
INFO[0002] Taking snapshot of full filesystem...        
INFO[0002] RUN test 0                                   
INFO[0002] cmd: /bin/sh                                 
INFO[0002] args: [-c test 0]                            
INFO[0002] Taking snapshot of full filesystem...        
INFO[0002] No files were changed, appending empty layer to config. No layer added to image. 
INFO[0002] USER 1000:1000                               
INFO[0002] cmd: USER                                    
INFO[0002] CMD ["whoami"]                               
2019/08/09 05:31:40 existing blob: sha256:0503825856099e6adb39c8297af09547f69684b7016b7f3680ed801aa310baaa
2019/08/09 05:31:41 pushed blob: sha256:09baf2d94f9afe6148bb4203991a5329ea4362c47c6c4e4f5a206e20791633e5
2019/08/09 05:31:42 gcr.io/tejal-test/test:latest: digest: sha256:5e7d175de7068f4276facbb0ab8ad18621df81790411af2a88e380fb01fe72d2 size: 428
tejaldesai@ $

@tejal29 tejal29 added norepro core team can't replicate this issue, require more info and removed kind/bug Something isn't working labels Aug 9, 2019
@tejal29
Copy link
Member

tejal29 commented Aug 9, 2019

closing this, since its an year old and i was not able to reproduce it

@abdennour
Copy link

based on what this issue is closed?
this is critical issue for compatibility concern

@HildaHay
Copy link

HildaHay commented Jan 6, 2022

This issue has still not been resolved, despite being closed.

@imjasonh imjasonh reopened this Jan 6, 2022
@abdennour
Copy link

This issue has still not been resolved, despite being closed.

That's why we cannot centralize it as builder engine in pipeline systems because we want to have same DX between his local env & pipeline build env.

Any news ?

@imjasonh
Copy link
Collaborator

imjasonh commented Jan 6, 2022

Any news ?

Nope.

If anybody reading this is interested in digging into this and submitting a fix I'll be happy to review it.

@pmcq
Copy link

pmcq commented Jan 6, 2022

I believe this comment still holds pointing to where the issue is and a suggested way to resolve - #477 (comment)

@imjasonh
Copy link
Collaborator

imjasonh commented Jan 6, 2022

I believe this comment still holds pointing to where the issue is and a suggested way to resolve - #477 (comment)

Great, sounds like we have an integration test Dockerfile that fails before the change, and an idea of where to make the change to make the test pass.

All we need now is someone to propose the change 😇

@imjasonh imjasonh added good first issue Good for newcomers and removed norepro core team can't replicate this issue, require more info labels Jan 6, 2022
@abdennour
Copy link

Ok let me save you! Contributing guide is simple ?

@sybernatus
Copy link

sybernatus commented May 29, 2022

Same issue with the kaniko image gcr.io/kaniko-project/executor:v1.8.1-debug

INFO[0227] Taking snapshot of full filesystem...        
INFO[0306] WORKDIR /build                               
INFO[0306] cmd: workdir                                 
INFO[0306] Changed working directory to /build          
INFO[0306] Creating directory /build                    
INFO[0306] Pushing layer registry.com/xxx to cache now 
INFO[0306] Taking snapshot of files...                  
INFO[0306] COPY package.json .                          
INFO[0306] Taking snapshot of files...                  
INFO[0306] COPY yarn.lock .                             
INFO[0306] Taking snapshot of files...                  
INFO[0306] Pushing image to registry.com/xxx
INFO[0306] RUN chown -R 1001:1001 /build                
INFO[0306] cmd: /bin/sh                                 
INFO[0306] args: [-c chown -R 1001:1001 /build]         
INFO[0306] util.Lookup returned: &{Uid:0 Gid:0 Username:root Name: HomeDir:/root} 
INFO[0306] performing slow lookup of group ids for root 
INFO[0306] Running: [/bin/sh -c chown -R 1001:1001 /build] 
INFO[0306] Taking snapshot of full filesystem...        
INFO[0342] Pushed registry.com/xxx 
INFO[0353] USER 1001:1001                               
INFO[0353] Pushing layer registry.com/xxx to cache now 
INFO[0353] cmd: USER                                    
INFO[0353] No files changed in this command, skipping snapshotting. 
INFO[0353] RUN yarn --silent install --prod --ignore-scripts --network-timeout 9000 
INFO[0353] cmd: /bin/sh                                 
INFO[0353] args: [-c yarn --silent install --prod --ignore-scripts --network-timeout 9000] 
INFO[0353] Pushing image to registry.com/xxx
error building image: error building stage: failed to execute command: credentials: get uid/gid: user: unknown user 1001

@aaron-prindle aaron-prindle added priority/p0 Highest priority. Break user flow. We are actively looking at delivering it. differs-from-docker works-with-docker kind/enhancement New feature or request kind/bug Something isn't working area/permissions area/dockerfile-command For all bugs related to dockerfile file commands area/uid and removed area/permissions labels Jun 7, 2023
@aaron-prindle
Copy link
Collaborator

aaron-prindle commented Jul 15, 2023

Can anyone in the thread here confirm if this is still an issue w/ the latest version of Kaniko - v1.12.1? This PR was merged #2269 ~Sep 2022 which mentions it might fix this issue

@aaron-prindle
Copy link
Collaborator

aaron-prindle commented Jul 15, 2023

When I attempt the repro Dockerfile(s) in the initial post I am seeing build success w/ the latest version of Kaniko v1.12.1:

aprindle@aprindle-ssd ~/kaniko  [aprindle-debug]./run_in_docker.sh /workspace/Dockerfile /usr/local/google/home/aprindle/kaniko-ctx-dir gcr.io/aprindle-test-cluster/kaniko-test:latest false |& tee ~/out.txt
INFO[0000] Retrieving image manifest alpine             
INFO[0000] Retrieving image alpine from registry index.docker.io 
INFO[0001] Built cross stage deps: map[]                
INFO[0001] Retrieving image manifest alpine             
INFO[0001] Returning cached image manifest              
INFO[0001] Executing 0 build triggers                   
INFO[0001] Building stage 'alpine' [idx: '0', base-idx: '-1'] 
INFO[0001] Skipping unpacking as no commands require it. 
INFO[0001] USER 1000:1000                               
INFO[0001] Cmd: USER                                    
INFO[0001] CMD ["whoami"]                               
INFO[0001] Pushing image to gcr.io/aprindle-test-cluster/kaniko-test:latest 
INFO[0004] Pushed gcr.io/aprindle-test-cluster/kaniko-test@sha256:bdb931cde77849f6b3d918775444aa2e649e047898905198d1f82c72fd78e02e 
aprindle@aprindle-ssd ~/kaniko  [aprindle-debug]./run_in_docker.sh /workspace/Dockerfile /usr/local/google/home/aprindle/kaniko-ctx-dir gcr.io/aprindle-test-cluster/kaniko-test:latest false |& tee ~/out.txt
INFO[0000] Retrieving image manifest alpine             
INFO[0000] Retrieving image alpine from registry index.docker.io 
INFO[0001] Built cross stage deps: map[]                
INFO[0001] Retrieving image manifest alpine             
INFO[0001] Returning cached image manifest              
INFO[0001] Executing 0 build triggers                   
INFO[0001] Building stage 'alpine' [idx: '0', base-idx: '-1'] 
INFO[0001] Skipping unpacking as no commands require it. 
INFO[0001] USER 1000:1000                               
INFO[0001] Cmd: USER                                    
INFO[0001] CMD ["whoami"]                               
INFO[0001] Pushing image to gcr.io/aprindle-test-cluster/kaniko-test:latest 
INFO[0004] Pushed gcr.io/aprindle-test-cluster/kaniko-test@sha256:aa0e279593171ee037a36011132dddea17bee12f600025c94c4bb9436d751c14 

@hrobertson
Copy link
Author

Yes this looks like it's fixed. Thanks for flagging that up @aaron-prindle .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/dockerfile-command For all bugs related to dockerfile file commands area/uid categorized differs-from-docker good first issue Good for newcomers issue/unknown-user issue/USER kind/bug Something isn't working kind/enhancement New feature or request priority/p0 Highest priority. Break user flow. We are actively looking at delivering it. works-with-docker
Projects
None yet
Development

No branches or pull requests