Docker out of memory crash (32 bit binary used)

[klauspost]

It seems like the Docker image uses a 32 bit ipfs binary.

This causes out-of-memory crashes when I import many files at once, for instance: crash.txt.

Is there any reason for not using a 64 bit binary?

[klauspost]

@Luzifer - do you remember why you chose the 32 bit executable?

[Luzifer]

Because the 64bit binary didn't work in the docker container on all of my test systems… (Until today I'm putting 32bit binaries into containers because of the 64bit binaries are failing to start)

Just tried again:

# docker run --rm -ti -v ~/tmp/ipfs:/data/ipfs ipfs64
/usr/local/bin/start_ipfs: line 11: /usr/local/bin/ipfs: No such file or directory
Running ...
/usr/local/bin/start_ipfs: line 16: /usr/local/bin/ipfs: No such file or directory
/usr/local/bin/start_ipfs: line 17: /usr/local/bin/ipfs: No such file or directory
/usr/local/bin/start_ipfs: line 18: /usr/local/bin/ipfs: No such file or directory
/usr/local/bin/start_ipfs: line 21: /usr/local/bin/ipfs: No such file or directory

Change used for this test:

diff --git a/Dockerfile b/Dockerfile
index 88a8bf4..4106339 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -15,10 +15,10 @@ ADD bin/container_shacheck /usr/local/bin/shacheck
 RUN adduser -D -h /data -u 1000 ipfs \
  && mkdir -p /data/ipfs && chown ipfs:ipfs /data/ipfs \
  && apk add --update bash curl wget ca-certificates zip \
- && wget https://gobuilder.me/get/github.com/ipfs/go-ipfs/cmd/ipfs/ipfs_${VERSION}_linux-386.zip \
- && /bin/bash /usr/local/bin/shacheck ${VERSION} ipfs_${VERSION}_linux-386.zip \
- && unzip ipfs_${VERSION}_linux-386.zip \
- && rm ipfs_${VERSION}_linux-386.zip \
+ && wget https://gobuilder.me/get/github.com/ipfs/go-ipfs/cmd/ipfs/ipfs_${VERSION}_linux-amd64.zip \
+ && /bin/bash /usr/local/bin/shacheck ${VERSION} ipfs_${VERSION}_linux-amd64.zip \
+ && unzip ipfs_${VERSION}_linux-amd64.zip \
+ && rm ipfs_${VERSION}_linux-amd64.zip \
  && mv ipfs/ipfs /usr/local/bin/ipfs \
  && chmod 755 /usr/local/bin/start_ipfs \
  && apk del wget zip curl

[Kubuxu]

It looks like ipfs might be missing some lib. Can you run 'ldd /usr/local/bin/ipfs' in the container?

[klauspost]

I seems like all gobuilder builds are failing: https://gobuilder.me/github.com/ipfs/go-ipfs leaving only empty files. My guess is that even the 32 bit build is broken ATM.

[Kubuxu]

Or that. I've heard hoofbeets and thought of zebras instead of horses.

[Luzifer]

@klauspost You should look at https://gobuilder.me/github.com/ipfs/go-ipfs/cmd/ipfs - There you will find the binaries used for the container.

LDD:

> docker run --rm -ti --entrypoint=/bin/bash ipfs64
bash-4.3$ ldd /usr/local/bin/ipfs
    /lib64/ld-linux-x86-64.so.2 (0x562e4880b000)
    libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x562e4880b000)
    libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x562e4880b000)
bash-4.3$ /usr/local/bin/ipfs
bash: /usr/local/bin/ipfs: No such file or directory
bash-4.3$

Edit1: Found the reason: https://github.com/gliderlabs/docker-alpine/blob/master/docs/caveats.md (Even when we're not using the alpine build of gliderlabs this seems to be true for us too…)

Edit2: Looks like setting a env variable during build could solve this… http://blog.xebia.com/create-the-smallest-possible-docker-container/

Edit3: Found a solution working on gobuilder:

> docker run --rm -ti --entrypoint=/bin/bash 20c98f4608e6
bash-4.3$ ldd /usr/local/bin/ipfs_static
ldd: /usr/local/bin/ipfs_static: Not a valid dynamic program
bash-4.3$ /usr/local/bin/ipfs_static
USAGE:

    ipfs - global p2p merkle-dag filesystem

    ipfs [<flags>] <command> [<arg>] ...

Edit4: Opened a PR proposal

[whyrusleeping]

I think this has since been resolved. If not, its very likely not a problem anymore.