build static jq binaries

[kranurag7]

upstream ref: https://github.com/jqlang/jq/blob/c1d885b0249f3978de3a21ea2bfb7ced06ed2aff/.github/workflows/ci.yml#L85-L94

this commit builds jq as per upstream releases and links statically without relying on any system deps.

the current jq executable dynamically links to some system dependencies.

/ # apk add cmd:jq cmd:ldd
fetch https://packages.wolfi.dev/os/aarch64/APKINDEX.tar.gz
(1/6) Installing oniguruma (6.9.9-r3)
(2/6) Installing jq (1.7.1-r2)
(3/6) Installing ncurses-terminfo-base (6.5_p20240629-r0)
(4/6) Installing ncurses (6.5_p20240629-r0)
(5/6) Installing bash (5.2.21-r5)
(6/6) Installing posix-libc-utils (2.39-r7)
OK: 19 MiB in 20 packages
/ # ldd $(command -v jq)
	linux-vdso.so.1 (0x0000e63c4f41b000)
	libm.so.6 => /lib/libm.so.6 (0x0000e63c4f2b0000)
	libonig.so.5 => /usr/lib/libonig.so.5 (0x0000e63c4f200000)
	libc.so.6 => /lib/libc.so.6 (0x0000e63c4f040000)
	/lib/ld-linux-aarch64.so.1 (0x0000e63c4f3de000)

# test for this patch:
[sdk] ❯ apk add --allow-untrusted ./packages/aarch64/jq-1.7.1-r2.apk
(1/1) Replacing jq (1.7.1-r2 -> 1.7.1-r2)
OK: 1089 MiB in 67 packages
[sdk] ❯ ldd $(command -v jq)
	not a dynamic executable
[sdk] ❯

[xnox]

this commit builds jq as per upstream releases and links statically without relying on any system deps.

That is a CI build, which also uses embedded / vendored copy of dependencies.

the current jq executable dynamically links to some system dependencies.

Which results in better tracking of CVEs by all scanners, and allows resolving CVEs without a rebuild, when dependencies are upgraded with CVE fixes (for example glibc upgrade alone, resolves CVE that may be in use by jq).

Note jq package builds a shared library which other packages depend on. And generally it is not advised to mix static and shared linked libraries - as if in jq library is statically linked, then used as shared library dependency in falco, and falco otherwise shared linked against the same library as jq needs at runtime there could be symbol resolution clashes.

The upstream ci workflow job is not a justification suitable here.

Why do you want jq binary or libjq library statically linked, and what does doing that gain? Which comes at a cost of increasing work for static linking tracking, and rebuilds when static library dependencies move.

[xnox]

Needs more information

[kranurag7]

Thanks for the additional info @xnox, I missed the point that I'm vendoring glibc this way. In hindsight, I realize, I should have been careful after that this patch #12914

I don't have a strong use case, I was just hacking and I was trying to make the following work.

Details

docker run --name wolfi-install -v $HOME/.arkade/bin:/usr/bin --rm -it cgr.dev/chainguard/wolfi-base:latest sh -c 'apk add jq'

Having said that, I'll close this PR with an agreement on the points you wrote above.

[xnox]

@kranurag7

I don't have a strong use case, I was just hacking and I was trying to make the following work.

why would you not create container that has jq package installed, and call said container as an entry point? You don't need jq binary back on the host, as you can always call docker run --entrypoint /usr/bin/jq my-jq-container on the host, as equivalent of jq.

extracting binaries from one container to a random host OS, are not guaranteed to work - even when statically linked. Especially whenever dlopen is involved.

Statically linked binaries are not portable.
Statically linked C/C++ binaries hide CVEs, as there is no introspection available as to which statically linked binaries were linked.

[kranurag7]

why would you not create container that has jq package installed

I created an image using apko using the following config.

apko config for jq

contents:
  keyring:
    - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
  repositories:
    - https://packages.wolfi.dev/os
  packages:
    - jq

entrypoint:
  command: /usr/bin/jq

accounts:
  groups:
    - groupname: nonroot
      gid: 65532
  users:
    - username: nonroot
      uid: 65532
      gid: 65532
  run-as: 65532

archs:
- x86_64
- arm64

so far this works great and I'm happy to put this behind an alias something like this:

    alias jq 'docker run --rm -i apko.local/cache:aa09cca0ad3a67a015713c490ca4c74bbe067e8a97fc90417a385351abdc77fa'
    alias crane 'docker run --rm cgr.dev/chainguard/crane:latest'
    alias aws 'docker run -v ~/.aws:/home/nonroot/.aws cgr.dev/chainguard/aws-cli-v2:latest-dev'

some shortcomings were:

1. It's being executed inside container, color support is missing inside the container. (I fixed the color one by passing --color-output to the container.)
2. jq test.json is no longer working and I've to use jq < test.json which is a tradeoff I'm willing to accept for running things inside container.

extracting binaries from one container to a random host OS, are not guaranteed to work - even when statically linked.

I'm exactly doing this thing (please correct if you think this is really wrong) and so far I extract static go binaries from the container by mounting a location in my path. (usually some path from $HOME, something like ~/.local/bin or ~/.arkade/bin). I wrote about it here

Earlier I only used to extract go static binaries but in the past few months I'm doing the same for rust binaries as well. Especially after https://www.chainguard.dev/unchained/changes-to-static-git-and-busybox-developer-images-2 (thank you).

So, after this when I setup a VM/baremetal server for self hosting or working on projects, I rely on this to download a lot of toolings i use on a daily basis (zellij, ripgrep, fd, bat, lazygit, helix, kind, helm, kubectl, kustomize, atuin and a lot more.)

I have two just global recipes defines in my global justfile and I use it almost daily. I have this defined in my dotfiles as well.

The reason for doing this is because apk toolings are quite nice and fast. I've tried scripting out things in the past and nix as well but so far I like approach of extracting binaries. I've really spent good chunk of time in finding the best package manager but I'm still navigating that journey for my host OS :). Sometimes, extracting binaries from containers directly breaks for sure with some dynamically linked binaries but with my usage overtime, I have identified those and I don't download them.

Now, I'm aware of that the binaries are coming as of now with root permission on the host but again I'm happy to take that tradeoff given it's fast. I tried to pull it using non-root user but that'll fail inside container as expected.

$ docker run --user 1000 --rm -v ~/.arkade/bin/:/usr/bin/ cgr.dev/chainguard/wolfi-base:latest sh -c 'apk add lazygit'
ERROR: Unable to lock database: Permission denied
ERROR: Failed to open apk database: Permission denied

Especially whenever dlopen is involved.

Sorry, I don't know this but I'll search and study more about it. Any references would mean a lot.

Statically linked binaries are not portable.

Not sure about this but go binaries are meant to be portable when statically linked without CGO_ENABLED. Please let me know if you meant something else by portable.

Statically linked C/C++ binaries hide CVEs, as there is no introspection available as to which statically linked binaries were linked.

Yes, that's true.

@xnox, Please let me know if you've any comments on the current workflow. I'll study more on the knowledge gaps from the conversation and I'm excited about it.

[xnox]

@kranurag7 what is the benefit of mixing host os & container os? Why whould you not create a container with all the apk packages you like, enter said container and use that as your daily shell?

For example in wolfi-dev/os in Makefile we have this target

dev-container:
        docker run --privileged --rm -it \
            -v "${PWD}:${PWD}" \
            -w "${PWD}" \
            -e SOURCE_DATE_EPOCH=0 \
            ghcr.io/wolfi-dev/sdk:latest@sha256:0385b70eda5f4cec8f6f71063e95c6f3c5b80578cd2218a12e609cf35f5dba04

Which bind mounts current working directory into the container as read-write, enters the container which has the wolfi sdk build with all the developer tools installed. And then one can execute anything one wants, interractively, inside it.

With clean separation of the host and container - no duplicates of binaries leaking from the container to the host, only data is shared.

Can you make apko image with all the tools you like, and then locally make your gnome terminal execute docker run -ti --entrypoint sh <yourcontainer> as your shell, with the volume mount of your home directory into the container. Then color ouptut will work, and you can apk add any tool you need, or rebuild the container to upgrade things.

That is a more natural dev workflow (bind mount your home-dir into wolfi container) rather than the inverse - copying out wolfi binaries from a container onto the host and hoping they still all work and are compatible with any other host os tooling.

Also lots of IDEs are compatible with containers published like that - i.e. VS Code, Google workstations, Github code workspace - such that you can have Wolfi container as your working environment both locally and on remote systems.

Even static binaries, can contain, runtime dependencies which are not otherwise observable resulting at runtime failures.

[kranurag7]

@xnox thanks a lot for the suggestion, the overall workflow sounds good to me. I can start a shell inside the container and work from there. I've used the following config to create a dev image for me.

contents:
  keyring:
    - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
  repositories:
    - https://packages.wolfi.dev/os
  packages:
    - wolfi-base
    - chezmoi
    - go
    - crane
    - oras
    - helix
    - fish
    - zellij
    - kubectl
    - helm
    - jq

entrypoint:
  command: /usr/bin/fish

work-dir: /home/nonroot

paths:
  - path: /home
    type: directory
    permissions: 0o755
    # uid: 65532
    # gid: 65532

environment:
  PATH: /usr/sbin:/sbin:/usr/bin:/bin:/app/.arkade/bin
  TERM: xterm-256color
  COLORTERM: truecolor

# accounts:
#   groups:
#     - groupname: nonroot
#       gid: 65532
#   users:
#     - username: nonroot
#       uid: 65532
#       gid: 65532
#   run-as: 65532

archs:
- x86_64
- arm64

some minor problems were:

• I complile my editor I use from main branch and given it's a bit behind in wolfi, my editor config is breaking. I understand and don't expect wolfi to build from main but just writing this down if it's helpful to any.
• sometimes the config for applications (considering the config inside /etc) is way more stable when it comes to using it locally then using it inside container.
• At this point of time, I'm so used to working on host that I start invoking commands like docker, systemctl and more commands tied to host OS.
• A few tools are not part of wolfi packages that I use and I can add them but not sure if that's useful to all. Something like lazydocker, systemctl-tui and these kind of stuff. They are not meant to be run inside a container but while working on a project, I use them to go through logs.

For now, I think I'll try out this workflow and it'll take me a few days to get used to this. If I don't then I'll have it project specific something like dev-container target of wolfi-dev/os repo and have a separate tab open for doing host level things.

Thanks again for your inputs. :)

[xnox]

• At this point of time, I'm so used to working on host that I start invoking commands like docker, systemctl and more commands tied to host OS.

You can bind-mount socket files from host, into the container, such that docker and systemctl commands work, and execute things on the host.

Example for docker https://stackoverflow.com/a/61406232

[xnox]

• I complile my editor I use from main branch and given it's a bit behind in wolfi, my editor config is breaking. I understand and don't expect wolfi to build from main but just writing this down if it's helpful to any.

They tagged a release 2 weeks ago, and we have it in wolfi. If it is not good enough, ask upstream to tag new releases.

For example vim tags like every push, and we do build/release it sometimes multiple times a day. https://github.com/vim/vim/tags

[kranurag7]

• I'm aware of mounting docker and containerd sockets inside container and will try that out in coming days. I've started using this workflow in few projects. If mounting complete home directory doesn't work out good then per project approach of having a common dev-container is also looking good. I'll use that alongside host OS but yes primarily I was looking for a good and fast workflow for development and that seems close now if this works out.

They tagged a release 2 weeks ago, and we have it in wolfi. If it is not good enough, ask upstream to tag new releases.

Yes, people asked for nightly something like vim and neovim but as of now, it's not going in that direction. Ref: helix-editor/helix#6362

I wanted to try out new features so I built a pipeline using by wolfi-base here https://github.com/kranurag7/nightly-helix/blob/main/.github/workflows/nightly.yaml