Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[nix] Add nix derivation for static builds #2380

Merged
merged 1 commit into from
Jun 18, 2020

Conversation

hswong3i
Copy link
Collaborator

@hswong3i hswong3i commented May 28, 2020

@hswong3i
Copy link
Collaborator Author

hswong3i commented May 28, 2020

@saschagrunert hope to get some hints from you, now I get:

hswong3i@hswong3i-XPS-13-7390:~/Documents/alvistack/_fork/buildah$ ./result-bin/bin/buildah --version
buildah version 1.15.0-dev (image-spec 1.0.1-dev, runtime-spec 1.0.2-dev)

hswong3i@hswong3i-XPS-13-7390:~/Documents/alvistack/_fork/buildah$ ls -lah ./result-bin/bin/buildah
-r-xr-xr-x 1 hswong3i hswong3i 26M Jan  1  1970 ./result-bin/bin/buildah

hswong3i@hswong3i-XPS-13-7390:~/Documents/alvistack/_fork/buildah$ ldd ./result-bin/bin/buildah
	linux-vdso.so.1 (0x00007fff53150000)
	libpthread.so.0 => /nix/store/9rabxvqbv0vgjmydiv59wkz768b5fmbc-glibc-2.30/lib/libpthread.so.0 (0x00007fc7666ea000)
	libc.so.6 => /nix/store/9rabxvqbv0vgjmydiv59wkz768b5fmbc-glibc-2.30/lib/libc.so.6 (0x00007fc76652b000)
	/nix/store/9rabxvqbv0vgjmydiv59wkz768b5fmbc-glibc-2.30/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007fc76670d000)

hswong3i@hswong3i-XPS-13-7390:~/Documents/alvistack/_fork/buildah$ file ./result-bin/bin/buildah
./result-bin/bin/buildah: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /nix/store/9rabxvqbv0vgjmydiv59wkz768b5fmbc-glibc-2.30/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, Go BuildID=EEkKBjx2kvYeCH8iCgxn/hNQJT3BGnpEsIKYKH9sX/-jf7vS87RMJOChvaKfY8/zgAksVKCYVyjmYS0dML5, not stripped

Even build pcre statically. Already google for a while but no luck...

@TomSweeneyRedHat
Copy link
Member

@hswong3i I'm not at all nix fluent, can you describe what benefit this PR brings?

@hswong3i
Copy link
Collaborator Author

hswong3i commented May 28, 2020

As an end user (Ansible Role developer for buildah installation, support Ubuntu 18.04+ / CentOS 7+ / openSUSE 15.1 / Debian 10 / Fedora 32, https://github.com/alvistack/ansible-role-buildah) I am looking for:

  1. Able to install buildah with a unified style across different OS, not some with kubic support but some only able to compile from source code
  2. Able to pin on specific version in a simple way, no matter for testing latest release or rolling back to previous version for debug
  3. No chicken or egg issue, eg installing repo’s runs/conmon/podman in order to run a container for compile crun static binary (CentOS /openSUSE systemd limitation) for replacing runc to crun...

Today my answer for above is: install and compile buildah manually on all of my supported OS (https://github.com/alvistack/ansible-role-buildah/blob/619f9e5d838b6427681e333f36e148f78f1af653/tasks/main.yml#L64-L102). Yes this looks really silly and time consuming :-(

What if we have a static binary? Case study from crun could be a good example:

  1. Precompile static binary with nix (even latest master for PR and debug, see https://github.com/alvistack/crun/releases/tag/37c385d4), simply without above chicken or egg issue
  2. Installation and testing now become download static binary and go (https://github.com/alvistack/ansible-role-crun/blob/8a03c9a831ce11a78ca745a15c2c75fb2b760e6c/tasks/main.yml#L63-L90), again simple stupid and elegant

Other else benefit with nix could be:

  1. Newbie contributors as me could quickly setup a R&D environment by nix, which still keep my running OS clean without installing additional -dev packages for compile from source code
  2. No matter which host OS developers are using, we could still having a unified and predictable compile result for debug (with identical sha256 in result binary due to nix pinning on dependency libraries)
  3. After initial dev and debug with nix, we could left the platform native developer to handle its repo package packaging, with benefit of dynamic linked binary

End user direct benefit could also be:

  1. To whom looking for simplicity and support could go for kubic repo dynamic linked package
  2. To whom facing bug or looking for latest release could simply download the static linked binary to /usr/local/bin and go, or rollback by delete it

P.S. I am a nix newbie for only 2 weeks experience, but now I could give a hand for PR with @saschagrunert kindly help ;-)

P.P.S. After we support nix derivation inside our source code, newbie could start by:

curl -L https://nixos.org/nix/install | sh
git clone https://github.com/containers/buildah.git && cd buildah
nix build -f nix/
./result/bin/buildah --version

@TomSweeneyRedHat
Copy link
Member

@hswong3i Thank you for the very thorough explanation. The gating tests are not happy with this PR. Could you first try squashing all of your commits and repushing? I'm not able to easily tell at the moment what's wrong.

@saschagrunert could I prevail on you to take a peak at this PR?

@hswong3i hswong3i force-pushed the master-linux-amd64 branch 2 times, most recently from 8de3829 to 4d1948c Compare May 31, 2020 04:15
@hswong3i hswong3i changed the title [WIP] Add nix derivation for static builds [nix] Add nix derivation for static builds May 31, 2020
@hswong3i
Copy link
Collaborator Author

hswong3i commented May 31, 2020

Compile now successful: https://github.com/alvistack/buildah/releases/download/v1.14.9/buildah-v1.14.9-linux-amd64

$ ls -lah ./result/bin/buildah
-r-xr-xr-x 1 hswong3i hswong3i 28M Jan  1  1970 ./result/bin/buildah

$ ldd ./result/bin/buildah
    not a dynamic executable

$ file ./result/bin/buildah
./result/bin/buildah: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.32, Go BuildID=CwDA7s6xVnsDUBJPuqry/IZqF2YtxU4fbYRuchBlM/Edc89OPzZ9tveCQwtZy6/nw9UlhKTg7CwTXWavcC3, stripped

$ sudo ./result/bin/buildah version
Version:         1.14.9
Go Version:      go1.14.2
Image Spec:      1.0.1-dev
Runtime Spec:    1.0.1-dev
CNI Spec:        0.4.0
libcni Version:  v0.7.2-0.20190904153231-83439463f784
image Version:   5.4.3
Git Commit:      2dd31e4d
Built:           Thu Jan  1 08:00:01 1970
OS/Arch:         linux/amd64

Static binaries:

Therefore could focus on CI/CD fixes ;-)

Copy link
Member

@saschagrunert saschagrunert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

nix/default.nix Outdated Show resolved Hide resolved
nix/default.nix Outdated Show resolved Hide resolved
nix/default.nix Outdated Show resolved Hide resolved
nix/default.nix Outdated Show resolved Hide resolved
nix/nixpkgs.json Show resolved Hide resolved
cben referenced this pull request Jun 16, 2020
Add a pull request template.  Modeled after CRI-O (Thanks @saschagrunert!) and
Dockers.

Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
@hswong3i
Copy link
Collaborator Author

@saschagrunert No changes required since #2380 (review), just rebase with master and updated with make nixpkgs, result with:

$ ldd ./result/bin/buildah 
	not a dynamic executable

$ file ./result/bin/buildah
./result/bin/buildah: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.32, Go BuildID=9BIh_c3VBDBGDOcp1yn3/mFcOlv758UmQ-s-5YU09/tqs5v72uinffxblF8s50/_sRiMEJOqip4jnjgiIYJ, stripped

$ ls -lah ./result/bin/buildah
-r-xr-xr-x 1 hswong3i hswong3i 24M Jan  1  1970 ./result/bin/buildah

$ ./result/bin/buildah --version
buildah version 1.15.0-dev (image-spec 1.0.1-dev, runtime-spec 1.0.2-dev)

Copy link
Member

@saschagrunert saschagrunert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hswong3i
Copy link
Collaborator Author

@saschagrunert shall we ignore those Cancelled/Failing CI? Are they related to this PR?

@hswong3i hswong3i force-pushed the master-linux-amd64 branch 8 times, most recently from 91407e7 to 66e3501 Compare June 18, 2020 11:05
Signed-off-by: Wong Hoi Sing Edison <hswong3i@gmail.com>
@hswong3i
Copy link
Collaborator Author

@rhatdan
Copy link
Member

rhatdan commented Jun 18, 2020

bors r+

bors bot added a commit that referenced this pull request Jun 18, 2020
2380: [nix] Add nix derivation for static builds r=rhatdan a=hswong3i

#### What type of PR is this?

<!--
Please label this pull request according to what type of issue you are
addressing, especially if this is a release targeted pull request.

Uncomment only one `/kind <>` line, hit enter to put that in a new line, and
remove leading whitespace from that line:
-->

> /kind feature

#### What this PR does / why we need it:

Similar PR goes for crun/conmon/libpod/cri-o/etc, too.


Also see:
  - ~~containers/crun#372
  - ~~containers/conmon#161
  - containers/skopeo#932
  - #2380
  - containers/podman#6402
  - cri-o/cri-o#3804


Static binaries:
  - [crun-0.13-linux-amd64](https://github.com/alvistack/crun/releases/download/0.13/crun-0.13-linux-amd64)
  - [conmon-v2.0.17-linux-amd64](https://github.com/alvistack/conmon/releases/download/v2.0.17/conmon-v2.0.17-linux-amd64)
  - [skopeo-v1.0.0-linux-amd64](https://github.com/alvistack/skopeo/releases/download/v1.0.0/skopeo-v1.0.0-linux-amd64)
  - [buildah-v1.14.9-linux-amd64](https://github.com/alvistack/buildah/releases/download/v1.14.9/buildah-v1.14.9-linux-amd64)
  - [podman-v1.9.3-linux-amd64](https://github.com/alvistack/libpod/releases/download/v1.9.3/podman-v1.9.3-linux-amd64)
  - [cri-o-v1.17.4-linux-amd64.tar.gz](https://github.com/alvistack/cri-o/releases/download/v1.17.4/cri-o-v1.17.4-linux-amd64.tar.gz)
  - [cri-o-v1.18.1-linux-amd64.tar.gz](https://github.com/alvistack/cri-o/releases/download/v1.18.1/cri-o-v1.18.1-linux-amd64.tar.gz)

Ansible Roles:
  - https://github.com/alvistack/ansible-role-crun
  - https://github.com/alvistack/ansible-role-conmon
  - https://github.com/alvistack/ansible-role-skopeo
  - https://github.com/alvistack/ansible-role-buildah 
  - https://github.com/alvistack/ansible-role-podman
  - https://github.com/alvistack/ansible-role-cri_o


#### How to verify it

```
nix build -f nix/
```

#### Which issue(s) this PR fixes:

<!--
Automatically closes linked issue when PR is merged.
Uncomment the following comment block and include the issue
number or None on one line.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`, or `None`.
-->

<!--
Fixes #
or
None
-->

#### Special notes for your reviewer:


Here I skip the btrfs and lvm2 support for static binary, because:
1. btrfs will not support in CentOS 8
2. With skopeo experience both btrfs and lvm2 are not easy for compile as static binary

Also see:
- containers/podman#6402 (comment)

#### Does this PR introduce a user-facing change?

<!--
If no, just write `None` in the release-note block below. If yes, a release note
is required: Enter your extended release note in the block below. If the PR
requires additional action from users switching to the new release, include the
string "action required".

For more information on release notes please follow the kubernetes model:
https://git.k8s.io/community/contributors/guide/release-notes.md
-->

```release-note

```



Co-authored-by: Wong Hoi Sing Edison <hswong3i@gmail.com>
@bors
Copy link
Contributor

bors bot commented Jun 18, 2020

Build failed:

  • cirrus-ci/success

@hswong3i
Copy link
Collaborator Author

@rhatdan sorry i have no way to view the log from https://app.bors.tech/repositories/22803/log#batch-73778 for debug...

@rhatdan
Copy link
Member

rhatdan commented Jun 18, 2020

[+0025s] Trying to pull quay.io/libpod/in_podman:master...
[+0025s] can't talk to a V1 docker registry
[+0025s] Error: unable to pull quay.io/libpod/in_podman:master: unable to pull image: Error initializing source docker://quay.io/libpod/in_podman:master: can't talk to a V1 docker registry
[08:35:26] END - [+0025s] total duration since START

@rhatdan
Copy link
Member

rhatdan commented Jun 18, 2020

bors retry

@bors
Copy link
Contributor

bors bot commented Jun 18, 2020

Build succeeded:

@bors bors bot merged commit c10d3a0 into containers:master Jun 18, 2020
@TomSweeneyRedHat
Copy link
Member

@hswong3i and @saschagrunert TY both VERY MUCH for working your way through this and putting the functionality into Buildah!

@nalind nalind mentioned this pull request Jul 31, 2020
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 2, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants