Wildcard ignore string issue

[probwebdev]

Description of the issue

When buildkit is active wildcard ignore string not usable

Context information

Output of docker version

Client: Docker Engine - Community
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:54:27 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.8
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.6
  Git commit:       75249d8
  Built:            Fri Jul 30 19:52:33 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.9
  GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
 runc:
  Version:          1.0.1
  GitCommit:        v1.0.1-0-g4144b63
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Steps to reproduce the issue

Output of .dockerignore

# Docker
.dockerignore
docker
!**/docker-entrypoint*.sh
!docker/nginx/*
docker-compose*.yml

1. Add an existing directory to .dockerignore
2. Add wildcard ignore rule for that directory
3. Run DOCKER_BUILDKIT=1 docker build .

Observed result

COPY directive for ignored directory couldn't be executed even with existing exception rule

Expected result

Wildcard exception rule should be used for ignored directory and specified files should be copied

Stacktrace / full error message

DOCKER_BUILDKIT=1 docker build -f ./docker/project/dev.Dockerfile .
[+] Building 1.1s (12/13)                                                             
 => [internal] load build definition from dev.Dockerfile                         0.0s
 => => transferring dockerfile: 42B                                              0.0s
 => [internal] load .dockerignore                                                0.0s
 => => transferring context: 246B                                                0.0s
 => [internal] load metadata for docker.io/library/node:16-buster-slim           0.9s
 => [1/9] FROM docker.io/library/node:16-buster-slim@sha256:af6f241029c4d63107c  0.0s
 => [internal] load build context                                                0.2s
 => => transferring context: 265.72kB                                            0.2s
 => CACHED [2/9] RUN apt-get update &&     apt-get install gosu &&     rm -rf /  0.0s
 => CACHED [3/9] RUN mkdir /home/node/app                                        0.0s
 => CACHED [4/9] WORKDIR /home/node/app                                          0.0s
 => CACHED [5/9] COPY --chown=node:node package.json yarn.lock ./                0.0s
 => CACHED [6/9] RUN yarn install &&     yarn cache clean                        0.0s
 => CACHED [7/9] COPY --chown=node:node . ./                                     0.0s
 => ERROR [8/9] COPY ./docker/project/docker-entrypoint.sh /usr/local/bin/do  0.0s
------
 > [8/9] COPY ./docker/project/docker-entrypoint.sh /usr/local/bin/docker-entrypoint:
------
failed to compute cache key: "/docker/project/docker-entrypoint.sh" not found: not found

Additional context

docker/compose#8642

[tonistiigi]

@aaronlehmann Do you know about this case?

[aaronlehmann]

I think dockerignore handling uses llb.Local, which uses fsutil.Walk (not 100% sure as I haven't worked with this code). The include/exclude handling in fsutil.Walk was pretty broken (and did not support double-star) until tonistiigi/fsutil#108. I don't believe Docker has been updated to use this new code. I attempted to update fsutil in moby/moby#42758, but ran into problems because of the circular dependency.

cc @thaJeztah

[tonistiigi]

@robertgorbul Can you test with moby/buildkit:master?

[probwebdev]

Hey @tonistiigi! Sorry for abandoning this issue. Seems it's still persist with master but I'm not sure if I tested it properly.
Steps:

1. docker pull moby/buildkit:master
2. docker run -d --name buildkitd --privileged moby/buildkit:master
3. export BUILDKIT_HOST=docker-container://buildkitd
4. brew install buildkit
5. Try to build

buildctl build --frontend dockerfile.v0 --local context=. --local dockerfile=./docker/mywebentry --output type=image,name=test        
[+] Building 1.1s (13/14)                                                                                                                           
 => [internal] load build definition from Dockerfile                                                                                           0.0s
 => => transferring dockerfile: 604B                                                                                                           0.0s
 => [internal] load .dockerignore                                                                                                              0.0s
 => => transferring context: 374B                                                                                                              0.0s
 => [internal] load metadata for docker.io/library/node:16-buster-slim                                                                         0.8s
 => CACHED [ 1/10] FROM docker.io/library/node:16-buster-slim@sha256:269885d847d382df99cc93ab5c37d3aea9f81fd8add895a9a3c639f84ae45aa4          0.0s
 => => resolve docker.io/library/node:16-buster-slim@sha256:269885d847d382df99cc93ab5c37d3aea9f81fd8add895a9a3c639f84ae45aa4                   0.0s
 => [internal] load build context                                                                                                              0.0s
 => => transferring context: 15.05kB                                                                                                           0.0s
 => CANCELED [ 2/10] RUN apt-get update &&     apt-get install gosu &&     rm -rf /var/lib/apt/lists/*                                         0.1s
 => CACHED [ 3/10] RUN mkdir /home/node/app                                                                                                    0.0s
 => CACHED [ 4/10] WORKDIR /home/node/app                                                                                                      0.0s
 => CACHED [ 5/10] COPY --chown=node:node package.json yarn.lock .yarnrc.yml ./                                                                0.0s
 => CACHED [ 6/10] COPY --chown=node:node .yarn/ .yarn/                                                                                        0.0s
 => CACHED [ 7/10] RUN yarn install                                                                                                            0.0s
 => CACHED [ 8/10] COPY --chown=node:node . ./                                                                                                 0.0s
 => ERROR [ 9/10] COPY ./docker/mywebentry/docker-entrypoint.sh /usr/local/bin/docker-entrypoint                                               0.0s
------
 > [ 9/10] COPY ./docker/mywebentry/docker-entrypoint.sh /usr/local/bin/docker-entrypoint:
------
Dockerfile:25
--------------------
  23 |     USER root
  24 |     
  25 | >>> COPY ./docker/mywebentry/docker-entrypoint.sh /usr/local/bin/docker-entrypoint
  26 |     RUN chmod +x /usr/local/bin/docker-entrypoint
  27 |     
--------------------
error: failed to solve: failed to compute cache key: failed to calculate checksum of ref ax5jeb29ollkk8zwfovg64a8k::g88neqc0pf5leh6woz5yrf4g4: "/docker/mywebentry/docker-entrypoint.sh": not found

6. Container logs

docker logs 8a9ccbf8c3d2 
time="2021-12-19T09:43:41Z" level=info msg="auto snapshotter: using overlayfs"
time="2021-12-19T09:43:41Z" level=warning msg="using host network as the default"
time="2021-12-19T09:43:41Z" level=info msg="found worker \"ax5jeb29ollkk8zwfovg64a8k\", labels=map[org.mobyproject.buildkit.worker.executor:oci org.mobyproject.buildkit.worker.hostname:8a9ccbf8c3d2 org.mobyproject.buildkit.worker.snapshotter:overlayfs], platforms=[linux/amd64 linux/386]"
time="2021-12-19T09:43:41Z" level=warning msg="skipping containerd worker, as \"/run/containerd/containerd.sock\" does not exist"
time="2021-12-19T09:43:41Z" level=info msg="found 1 workers, default=\"ax5jeb29ollkk8zwfovg64a8k\""
time="2021-12-19T09:43:41Z" level=warning msg="currently, only the default worker can be used."
time="2021-12-19T09:43:41Z" level=info msg="running server on /run/buildkit/buildkitd.sock"
time="2021-12-19T09:50:34Z" level=error msg="/moby.buildkit.v1.Control/Solve returned error: rpc error: code = Unknown desc = target stage foo could not be found\n"
time="2021-12-19T09:55:23Z" level=error msg="/moby.buildkit.v1.Control/Solve returned error: rpc error: code = Unknown desc = failed to compute cache key: failed to calculate checksum of ref ax5jeb29ollkk8zwfovg64a8k::5vokgae5z3fkiw7q0n9yukhbn: \"/docker/mywebentry/docker-entrypoint.sh\": not found\n"
time="2021-12-19T09:57:32Z" level=error msg="/moby.buildkit.v1.Control/Solve returned error: rpc error: code = Canceled desc = context canceled\n"
time="2021-12-19T09:57:50Z" level=error msg="failed to kill runc p6vluswj3xpgjhxrrmw6ulpg9: buildkit-runc did not terminate successfully: exit status 1: container \"p6vluswj3xpgjhxrrmw6ulpg9\" does not exist\n" span="[ 2/10] RUN apt-get update &&     apt-get install gosu &&     rm -rf /var/lib/apt/lists/*"
time="2021-12-19T09:57:50Z" level=error msg="/moby.buildkit.v1.Control/Solve returned error: rpc error: code = Unknown desc = failed to compute cache key: failed to calculate checksum of ref ax5jeb29ollkk8zwfovg64a8k::sgfzb7g48mvmaxrnw2ou87ern: \"/docker/mywebentry/docker-entrypoint.sh\": not found\n"
time="2021-12-19T09:59:33Z" level=error msg="failed to kill runc k3ru7k4o7gcyeu13ydqr2kemd: buildkit-runc did not terminate successfully: exit status 1: container \"k3ru7k4o7gcyeu13ydqr2kemd\" does not exist\n" span="[ 2/10] RUN apt-get update &&     apt-get install gosu &&     rm -rf /var/lib/apt/lists/*"
time="2021-12-19T09:59:33Z" level=error msg="/moby.buildkit.v1.Control/Solve returned error: rpc error: code = Unknown desc = failed to compute cache key: failed to calculate checksum of ref ax5jeb29ollkk8zwfovg64a8k::g88neqc0pf5leh6woz5yrf4g4: \"/docker/mywebentry/docker-entrypoint.sh\": not found\n"

7. Update .dockerignore

#!**/docker-entrypoint*.sh
!docker/mywebentry/docker-entrypoint*.sh

8. Build again

buildctl build --frontend dockerfile.v0 --local context=. --local dockerfile=./docker/mywebentry --output type=image,name=test
[+] Building 102.3s (16/16) FINISHED                                                                                                                
 => [internal] load build definition from Dockerfile                                                                                           0.0s
 => => transferring dockerfile: 604B                                                                                                           0.0s
 => [internal] load .dockerignore                                                                                                              0.0s
 => => transferring context: 374B                                                                                                              0.0s
 => [internal] load metadata for docker.io/library/node:16-buster-slim                                                                         2.0s
 => [auth] library/node:pull token for registry-1.docker.io                                                                                    0.0s
 => CACHED [ 1/10] FROM docker.io/library/node:16-buster-slim@sha256:269885d847d382df99cc93ab5c37d3aea9f81fd8add895a9a3c639f84ae45aa4          0.0s
 => => resolve docker.io/library/node:16-buster-slim@sha256:269885d847d382df99cc93ab5c37d3aea9f81fd8add895a9a3c639f84ae45aa4                   0.0s
 => [internal] load build context                                                                                                              0.1s
 => => transferring context: 15.71kB                                                                                                           0.0s
 => [ 2/10] RUN apt-get update &&     apt-get install gosu &&     rm -rf /var/lib/apt/lists/*                                                  6.9s
 => [ 3/10] RUN mkdir /home/node/app                                                                                                           0.1s
 => [ 4/10] WORKDIR /home/node/app                                                                                                             0.1s 
 => [ 5/10] COPY --chown=node:node package.json yarn.lock .yarnrc.yml ./                                                                       0.8s 
 => [ 6/10] COPY --chown=node:node .yarn/ .yarn/                                                                                               0.0s 
 => [ 7/10] RUN yarn install                                                                                                                  69.4s 
 => [ 8/10] COPY --chown=node:node . ./                                                                                                        0.1s 
 => [ 9/10] COPY ./docker/mywebentry/docker-entrypoint.sh /usr/local/bin/docker-entrypoint                                                     0.0s 
 => [10/10] RUN chmod +x /usr/local/bin/docker-entrypoint                                                                                      0.1s 
 => exporting to image                                                                                                                        22.6s 
 => => exporting layers                                                                                                                       22.5s 
 => => exporting manifest sha256:ef877d5e9f553b68cc2b84ec2d09e44ed56dac936545d96ab1af6a28c0b56129                                              0.0s 
 => => exporting config sha256:fa9ca94edda800d0b743b57e4050cd3d3efd7e21c3826ce421d292d119ee6715                                                0.0s