Docker arm image fails to run

[markg85]

Hi,

The current IPFS docker images "install just fine" on ARM but give you this error when running:
exec user process caused "exec format error"

Now i've been trying to resolve that.
The first step was to build the docker image for arm (just rebuilding it on an arm machine). This on it's own does make me suspect that the current docker images are not build for ARM64. I hope that could be added?

Rebuilding from git works just fine (not when using a zip, see #7420), but running is then again giving an error. Just a different one. This time:
exec user process caused "no such file or directory"

Next i took a deep dive in the docker container. Running it with:
docker run -d -it --name ipfs_host --entrypoint /bin/sh --restart unless-stopped -v $ipfs_staging:/export -v $ipfs_data:/data/ipfs -p 4001:4001 -p 127.0.0.1:8080:8080 -p 127.0.0.1:5001:5001 ipfs

Note the --entrypoint /bin/sh override and -it so that i can actually get inside that container and run commands.

Which i can then do with:
docker exec -it ipfs_host /bin/sh

Attaching probably works too, i just picked this route.
Then manually running what the actual endpoint did gives me this:

/ # /sbin/tini -- /usr/local/bin/start_ipfs
/bin/sh: /sbin/tini: not found

Which is weird as /sbin/tini and /usr/local/bin/start_ipfs definitely do exist as files.
Just running /usr/local/bin/start_ipfs won't work either:

/ # /usr/local/bin/start_ipfs
Changing user to ipfs
/usr/local/bin/start_ipfs: line 9: su-exec: not found
/usr/local/bin/start_ipfs: exec: line 11: su-exec: not found

Now the issue is getting interesting as su-exec does in fact exist.
So manually running tini and su-exec gave me even more interesting results...

/ # /sbin/tini
/bin/sh: /sbin/tini: not found
/ # /sbin/su-exec 
/bin/sh: /sbin/su-exec: not found

I am suspecting a dependency to be missing here, but i don't know which one. It's surprisingly difficult to get LDD working in that container so i kinda left it there.
I did download tini-static-arm64 to confirm my suspicion. And sure enough, the static version does run!

However, su-exec still has the same issue. I wouldn't be surprised if the fix for tini also fixes su-exec.

Cheers,
Mark

[Stebalien]

We're building su-exec from source. Are you cross-building?

[markg85]

Nope. Building it directly on the Rock Pi 4b.
It doesn't take that long for it to build on there, just a few minutes for the whole image.

[Stebalien]

Could you run ls -Rla / on the container?

[markg85]

That's a lot of output (like find /)!
I'm sure that pasting nearly 43k lines here would be a massive no-go ;)

I uploaded it here: https://p.sc2.nl/hcXjp

If you want, i can modify the dockerfile and copy more files. My first hunch is a file missing in the glibc area. I haven't tested anything there yet though.

[Stebalien]

Hm. I really don't know. You're probably right and it's probably some kind of missing library.

Could you try #7505?

[markg85]

Well, good news and bad news.

Good news: That fix made both commands give output. So on it's own, that's a fix worth having.
Bad news: My command line is now spamming "Changing user to ipfs"

This is with the following docker run line:
docker run -d --name ipfs_host --restart unless-stopped -v $ipfs_staging:/export -v $ipfs_data:/data/ipfs -p 4001:4001 -p 127.0.0.1:8085:8080 -p 127.0.0.1:5001:5001 ipfs

Which gives (with docker logs -f ipfs_host):

Changing user to ipfs
Changing user to ipfs
Changing user to ipfs
Changing user to ipfs
Changing user to ipfs
Changing user to ipfs
Changing user to ipfs
Changing user to ipfs
Changing user to ipfs
Changing user to ipfs
Changing user to ipfs
Changing user to ipfs
Changing user to ipfs
Changing user to ipfs

I have no clue what might cause that...

[Stebalien]

Hm. It looks like su-exec is failing to actually change the user so we're recursively calling su-exec repeatedly. I don't know why that might be happening. Are you running docker with any special restrictions?

[markg85]

Not that i'm aware of. This is a libreelec distribution running on the Rock Pi 4b. It has docker for the specific reason to run other services if one needs it :)

I can try running the same commands on a archlinuxarm based odroid-xu4 if you want? I doubt the results is any different though.

[Stebalien]

Please try it. su-exec should work.

[markg85]

Hi @Stebalien

Sorry for the late reply!
I did just test this on my archlinuxarm odroid-xu4.
I'm happy to report that it's working! Awesome! :)

Why it's not working on my Rock Pi but is on my odroid is something i don't get yet. But i'm assuming that's something on my end and not specifically for IPFS. I'll add this feedback to the pull request too.

Side note though.
Creating the docker image does give me this "warning" (nothing says warning, it's displayed in red though):
debconf: delaying package configuration, since apt-utils is not installed

That might be worth fixing regardless.

[Stebalien]

Creating the docker image does give me this "warning" (nothing says warning, it's displayed in red though):
debconf: delaying package configuration, since apt-utils is not installed

Could you file a new bug?

[markg85]

Creating the docker image does give me this "warning" (nothing says warning, it's displayed in red though):
debconf: delaying package configuration, since apt-utils is not installed

Could you file a new bug?

Done: #7521

Thank you for your work on this! It's much appreciated!