Issue with building within a container

[darewreck54]

I'm trying to build the skopeo within a container with the following docker file.

Dockerfile

FROM golang

# Install Skopeo
RUN apt-get update && apt-get install -y \
    git libgpgme-dev libassuan-dev libbtrfs-dev libdevmapper-dev \
    && apt-get clean

RUN git clone https://github.com/containers/skopeo $GOPATH/src/github.com/containers/skopeo \
    && cd $GOPATH/src/github.com/containers/skopeo && make binary-local-static BUILDTAGS='containers_image_openpgp containers_image_ostree_stub containers_image_storage_stub' \
    && cp $GOPATH/src/github.com/containers/skopeo/skopeo /usr/bin/skopeo

This use to work a couple months ago, but it doesn't work anymore. Did something change?

Error:

$ docker build -t container/skopeo:latest .
Sending build context to Docker daemon  4.608kB
Step 1/4 : FROM golang
 ---> 52b59e9ead8e
Step 2/4 : RUN apt-get update && apt-get install -y     git libgpgme-dev libassuan-dev libbtrfs-dev libdevmapper-dev     && apt-get clean
 ---> Using cache
 ---> 9d6e37aa3d97
Step 3/4 : RUN git clone https://github.com/containers/skopeo $GOPATH/src/github.com/containers/skopeo     && cd $GOPATH/src/github.com/containers/skopeo && make binary-local-static BUILDTAGS='containers_image_openpgp containers_image_ostree_stub containers_image_storage_stub'     && cp $GOPATH/src/github.com/containers/skopeo/skopeo /usr/bin/skopeo
 ---> Running in 6282f54d622d
Cloning into '/go/src/github.com/containers/skopeo'...
GO111MODULE=on go build -mod=vendor -ldflags "-extldflags \"-static\" -X main.gitCommit=ce6ec7720e33b6fac1d7e7a8c4c9af3099bfc3f9" -gcflags "" -tags "containers_image_openpgp containers_image_ostree_stub containers_image_storage_stub" -o skopeo ./cmd/skopeo
build github.com/containers/skopeo/cmd/skopeo: cannot load github.com/containers/image/v5/storage: no Go source files
make: *** [Makefile:107: binary-local-static] Error 1
The command '/bin/sh -c git clone https://github.com/containers/skopeo $GOPATH/src/github.com/containers/skopeo     && cd $GOPATH/src/github.com/containers/skopeo && make binary-local-static BUILDTAGS='containers_image_openpgp containers_image_ostree_stub containers_image_storage_stub'     && cp $GOPATH/src/github.com/containers/skopeo/skopeo /usr/bin/skopeo' returned a non-zero code: 2

looks like it's complaining

github.com/containers/skopeo/cmd/skopeo: cannot load github.com/containers/image/v5/storage: no Go source files
make: *** [Makefile:107: binary-local-static] Error 1

Strange thou tat I don't have any issue building this locally on a mac, but issue with building it within the container.

Thanks

[TomSweeneyRedHat]

@darewreck54 I think this is due to this PR that recently merged: #748. I think the containers/images package on your machine needs to be updated, it went from v4 to v5, but I'm not sure how that would happen in ubuntu.

@lsm5 @rhatdan @vrothberg any ideas on how to make that happen?

FWIW, I dropped the static build part of your Dockerfile (see below) and that worked for me.

FROM golang

# Install Skopeo
RUN apt-get update && apt-get install -y \
    git libgpgme-dev libassuan-dev libbtrfs-dev libdevmapper-dev \
    && apt-get clean

RUN git clone https://github.com/containers/skopeo $GOPATH/src/github.com/containers/skopeo \
    && cd $GOPATH/src/github.com/containers/skopeo && make binary-local \
    && cp $GOPATH/src/github.com/containers/skopeo/skopeo /usr/bin/skopeo

[darewreck54]

When you say static part do you mean the build tags? In our use case, we want the build tags ... I’m guessing that is what causes the need for the v5 dependency

…

Sent from my iPhone On Nov 26, 2019, at 3:10 PM, Tom Sweeney <notifications@github.com> wrote:  @darewreck54<https://github.com/darewreck54> I think this is due to this PR that recently merged: #748<#748>. I think the containers/images package on your machine needs to be updated, it went from v4 to v5, but I'm not sure how that would happen in ubuntu. @lsm5<https://github.com/lsm5> @rhatdan<https://github.com/rhatdan> @vrothberg<https://github.com/vrothberg> any ideas on how to make that happen? FWIW, I dropped the static build part of your Dockerfile (see below) and that worked for me. FROM golang # Install Skopeo RUN apt-get update && apt-get install -y \ git libgpgme-dev libassuan-dev libbtrfs-dev libdevmapper-dev \ && apt-get clean RUN git clone https://github.com/containers/skopeo $GOPATH/src/github.com/containers/skopeo \ && cd $GOPATH/src/github.com/containers/skopeo && make binary-local \ && cp $GOPATH/src/github.com/containers/skopeo/skopeo /usr/bin/skopeo — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#771?email_source=notifications&email_token=AD3MFTT4KPQ2U5BLRB46LZ3QVWUEZA5CNFSM4JR6BI22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFHXT3I#issuecomment-558856685>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AD3MFTUNDXC7YFZFSFCJ5QTQVWUEZANCNFSM4JR6BI2Q>.

[vrothberg]

Thanks for opening the issue, @darewreck54. We actually want to drop the static builds (see #755) but didn't do the cleanup work yet. Please use make binary-local instead of make binary-local-static, which will resolve the issue.

[darewreck54]

@vrothberg what’s the implication with dropping the static builds?

https://github.com/containers/image

Does this mean that i have to install

gpgme-devel libassuan-devel ostree-devel

as a replacement for the build tags?

I'm guessing

covers it except for ostree

[TomSweeneyRedHat]

I'm going to reopen this issue given the discussion.

[vrothberg]

@vrothberg what’s the implication with dropping the static builds?

The binary will be dynamically linked and will won't suffer from the restrictions of static binary (see #670 for background).

Does this mean that i have to install [...] as a replacement for the build tags?

You're already installing libgpgme-dev libassuan-dev libbtrfs-dev libdevmapper-dev, so no changes are needed. The Makefile will take care of setting the build tags if needed. containers_image_openpgp and containers_image_ostree can be manually set (BUILDTASGS="..." make ...) if desired

[darewreck54]

@vrothberg

when you say manually set build stags, do you refer to

FROM golang:1.13.4

ARG SKOPEO_COMMIT=be6146b0a8471b02e776134119a2c37dfb70d414
ARG DOCKER_CREDENTAIL_GCR_VERSION=v2.0.0

# Install Skopeo
RUN apt-get update && apt-get install -y \
  git libgpgme-dev libassuan-dev libbtrfs-dev libdevmapper-dev ostree\
  && apt-get clean

RUN git clone https://github.com/containers/skopeo/  $GOPATH/src/github.com/containers/skopeo \
  && cd $GOPATH/src/github.com/containers/skopeo && git checkout ${SKOPEO_COMMIT} &&  make binary-local BUILDTAGS='containers_image_openpgp containers_image_ostree_stub containers_image_storage_stub' \
  && cp $GOPATH/src/github.com/containers/skopeo/skopeo /usr/bin/skopeo

This will result in the same error

$ docker build -t container/skopeo:latest .
Sending build context to Docker daemon  4.608kB
Step 1/6 : FROM golang:1.13.4
 ---> a2e245db8bd3
Step 2/6 : ARG SKOPEO_COMMIT=be6146b0a8471b02e776134119a2c37dfb70d414
 ---> Using cache
 ---> 1aee940d5b64
Step 3/6 : ARG DOCKER_CREDENTAIL_GCR_VERSION=v2.0.0
 ---> Using cache
 ---> d109d8256dc5
Step 4/6 : RUN apt-get update && apt-get install -y   git libgpgme-dev libassuan-dev libbtrfs-dev libdevmapper-dev ostree  && apt-get clean
 ---> Using cache
 ---> 2370b231b67f
Step 5/6 : RUN git clone https://github.com/containers/skopeo/  $GOPATH/src/github.com/containers/skopeo   && cd $GOPATH/src/github.com/containers/skopeo && git checkout ${SKOPEO_COMMIT} &&  make binary-local BUILDTAGS='containers_image_openpgp containers_image_ostree_stub containers_image_storage_stub'   && cp $GOPATH/src/github.com/containers/skopeo/skopeo /usr/bin/skopeo
 ---> Running in 053daebcfc44
Cloning into '/go/src/github.com/containers/skopeo'...
Note: checking out 'be6146b0a8471b02e776134119a2c37dfb70d414'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at be6146b0 release skopeo v0.1.40
GO111MODULE=on go build -mod=vendor "-buildmode=pie" -ldflags "-X main.gitCommit=be6146b0a8471b02e776134119a2c37dfb70d414" -gcflags "" -tags "containers_image_openpgp containers_image_ostree_stub containers_image_storage_stub" -o skopeo ./cmd/skopeo
build github.com/containers/skopeo/cmd/skopeo: cannot load github.com/containers/image/v5/storage: no Go source files
make: *** [Makefile:104: binary-local] Error 1
The command '/bin/sh -c git clone https://github.com/containers/skopeo/  $GOPATH/src/github.com/containers/skopeo   && cd $GOPATH/src/github.com/containers/skopeo && git checkout ${SKOPEO_COMMIT} &&  make binary-local BUILDTAGS='containers_image_openpgp containers_image_ostree_stub containers_image_storage_stub'   && cp $GOPATH/src/github.com/containers/skopeo/skopeo /usr/bin/skopeo' returned a non-zero code: 2
C

How would the build tags be needed if the make file will dynamically handle it for you?

[vrothberg]

How would the build tags be needed if the make file will dynamically handle it for you?

Some are not dynamically set (e.g., containers_image_openpgp).

build github.com/containers/skopeo/cmd/skopeo: cannot load github.com/containers/image/v5/storage: no Go source files

Curious, this error only occurs when "containers_image_ostree_stub" is specified twice in the build tags. It should work when removing the redundant tag from the Dockerfile. I can't tell you why go behaves this way; never seen it before.

Does that fix your issue?

[darewreck54]

@vrothberg How is "containers_image_ostree_stub" specified twice in the build tags?

BUILDTAGS='containers_image_openpgp containers_image_ostree_stub containers_image_storage_stub'

I'm seeing three different build tags? It's also strange because this worked before unless it suddenly became non dynamic?

Also if I don't include those build tags, would the functionality be the same if it's already installed into the system?

[vrothberg]

@darewreck54, thanks for double checking. You're absolutely correct!

The build fails because containers_image_storage_stub is set, and skopeo has direct reference on c/image/v5/storage. I'll prepare a fix for it.