Buildx is not honouring the insecure registry config in toml config provided by the default builder.

[jithinchandranj]

Specified the config file as:

vi /etc/buildkit/buildkitd.toml
debug = true
insecure-entitlements = [ "network.host", "security.insecure"]

[registry."192.168.x.x"]
  http = true
  insecure = true

docker buildx create --name mutli-platform-builder --config /etc/buildkit/buildkitd.toml --driver-opt network=host --use
docker buildx inspect --bootstrap

My sample dockerfile reads like:

FROM 192.168.x.x/docker/alpine AS builder
RUN apk add build-base
WORKDIR /home
COPY hello.c .
RUN gcc "-DARCH=\"`uname -a`\"" hello.c -o hello

FROM 192.168.x.x/docker/alpine
WORKDIR /home
COPY --from=builder /home/hello .
ENTRYPOINT ["./hello"]

When I run the build docker buildx build --platform linux/arm,linux/arm64,linux/amd64 -t 192.168.x.x/docker/muti-arch-test:1.0.0 . --push, error is thrown like error: failed to solve: 192.168.x.x/docker/alpine: failed to do request: Head "https://192.168.x.x/v2/docker/alpine/manifests/latest": dial tcp 192.168.x.x:443: connect: connection refused

From the earlier raised issues, got to know that the global daemon.json insecure registry configuration won't be taken by the builder, and hence I've provided the config and passed to the builder as specified. But still, it is trying to connect to the https port, instead of the http port. Am I doing something wrong here?

[martadinata666]

My setup on gitlab CI

buildkit-config.tom;

# Custom registry.
[registry."192.168.0.2:5050"]
  mirrors = ["192.168.0.2:5050"]
  http = true
  insecure = true

On .gitlab-ci.yml

- docker buildx create --use --config buildkit-config.toml
- docker buildx inspect --bootstrap
- docker buildx build --push --platform linux/amd64,linux/arm64  ....

[tonistiigi]

Seems to work for me

 # cat buildkitd.toml
[registry."192.168.1.2"]
  http = true
  insecure = true
 # cat Dockerfile
FROM 192.168.1.2/docker/alpine AS builder
RUN apk add build-base
 # docker buildx create --config buildkitd.toml --name regtest
BUILDX_BUILDER=regtest docker buildx build .

in logs I see

time="2021-09-28T01:53:06Z" level=info msg="trying next host" error="failed to do request: Head \"http://192.168.1.2/v2/docker/alpine/manifests/latest\": dial tcp 192.168.1.2:80: i/o timeout" spanID=b444a816dc89fd5f traceID=3c3a77a5267e622d1c4a8b5e6dacd90d

So it is going to port 80

[tonistiigi]

My setup on gitlab CI

Why are you setting a mirror to the same name as your server?

[martadinata666]

My setup on gitlab CI

Why are you setting a mirror to the same name as your server?

oh, because i hosted gitlab myself, so it return back to gitlab local registry. As gitlab.com got connection issue last few days

[tonistiigi]

oh, because i hosted gitlab myself, so it return back to gitlab local registry. As gitlab.com got connection issue last few days

This doesn't make any sense. In the worst case, it creates a loop.

[martadinata666]

#registry configures a new Docker register used for cache import or output.

[registry."docker.io"]
  mirrors = ["hub.docker.io"]
  http = true
  insecure = true
  ca=["/etc/config/myca.pem"]
  [[registry."docker.io".keypair]]
    key="/etc/config/key.pem"
    cert="/etc/config/cert.pem"

Mirrors part taken from buildkitd.toml.md
it doesn't really tell much abt what mirror do, and i thought,
this part for naming[registry."docker.io"]
and this part mirrors = ["hub.docker.io"] is the true URL, not really sure. Or vice versa, so i put name same to both. May i get more insight on this?

[tonistiigi]

That is an example that fills all field to show them. Mirror is a server that is connected to instead of the registry. If you don't use mirrors then don't set it.

[martadinata666]

Oh i see, after looking at my runner log carefully, it pushes image twice. Thanks for the insight, gonna remove it now.