The Problem with python:3.6-alpine

[An4ik]

Hi. I'm teacher at the university and with students found a problem:

FROM python:3.6-alpine

This is the first line in my DockerFile for our class project. I've successfully build & run it 9 days ago (01.02.2019), but today (09.02.2019) no one has build built it (+20 students).

So I've removed that image in my local machine and tried to built it and got the same error as everyone in my class:

Error relocating /usr/local/lib/libpython3.6m.so.1.0: getrandom: symbol not found ERROR: Service 'django' failed to build: The command '/bin/sh -c pip install -r /requirements/local.txt' returned a non-zero code: 127

So we did a little research and found that the last changes has been done 3 days ago (06.02.2019)
commit and I think that is the reason of our failures.

I've changed it to FROM python:3.6.8-alpine3.8 and everyone has build it successfully.

Thanks.

[netok421]

The bug is from alpine 3.9
Bug report

[tianon]

Can you provide a minimal reproducer for your issue? (just the error message doesn't really help a whole lot in this case -- the best we can do is guess, which is where the suggestion of https://bugs.alpinelinux.org/issues/9959 comes from)

[An4ik]

FROM python:3.6-alpine

ENV PYTHONUNBUFFERED 1

RUN echo http://mirror.yandex.ru/mirrors/alpine/v3.8/main > /etc/apk/repositories; \
    echo http://mirror.yandex.ru/mirrors/alpine/v3.8/community >> /etc/apk/repositories

RUN apk update \
  # psycopg2 dependencies
  && apk add --virtual build-deps gcc python3-dev musl-dev \
  && apk add postgresql-dev \
  # Pillow dependencies
  && apk add jpeg-dev zlib-dev freetype-dev lcms2-dev openjpeg-dev tiff-dev tk-dev tcl-dev \
  # CFFI dependencies
  && apk add libffi-dev py-cffi \
  # Translations dependencies
  && apk add gettext \
  # https://docs.djangoproject.com/en/dev/ref/django-admin/#dbshell
  && apk add postgresql-client

# Requirements are installed here to ensure they will be cached.
COPY ./requirements /requirements
RUN pip install -r /requirements/local.txt

COPY ./compose/production/django/entrypoint /entrypoint
RUN sed -i 's/\r//' /entrypoint
RUN chmod +x /entrypoint

COPY ./compose/local/django/start /start
RUN sed -i 's/\r//' /start
RUN chmod +x /start

WORKDIR /app

ENTRYPOINT ["/entrypoint"]

This is the first version of my docker file. So before demonstrating it I've build it in my local machine on 01.02.2019. it worked.

But when my students tried to build it. (20 laptops) all of them failed. I've removed all images and tried to rebuild it. then got the same error.

googled it, and found that it's ordering problem. and then found a commit where that order was changed.

Once I've changed the version (just have selected previous version of from docker hub)
FROM python:3.6.8-alpine3.8

everyone in my class and me have built and run. (docker-compose build and docker-compose run)

so you can just copy docker config above and try to build it. I think will find that errors and more details if my is not enough)))

[tianon]

That's not exactly minimal. 😅 Any chance you could "code golf" that down to something smaller that can still reproduce the failure to help narrow down what's going wrong?

[yosifkit]

The big things that stand out to me are the Alpine 3.8 mirrors that almost definitely will break on an Alpine 3.9 based image (especially with the transition on system packages from libressl back to openssl); and then the installation of python3-dev for basically the same reasons we blocked php-* packages in docker-library/php#542 (tldr, you get modules compiled against a different version of [python] and probably two versions of [python] installed).

[ziserendexin]

I have same problem.

The most confused is, when I retry it at alpine`s sh in container.

When I splite this cmd to four cmd, and run.every thing is ok.

apk add --no-cache --virtual .build-deps gcc musl-dev curl 

apk add --no-cache --virtual .build-deps 
apk add --no-cache --virtual gcc 
apk add --no-cache --virtual musl-dev 
apk add --no-cache --virtual curl

But run at one line, it will give this error.

/ # apk add --no-cache --virtual .build-deps gcc musl-dev curl 
fetch http://mirrors.aliyun.com/alpine/v3.6/main/x86_64/APKINDEX.tar.gz
fetch http://mirrors.aliyun.com/alpine/v3.6/community/x86_64/APKINDEX.tar.gz
(1/21) Downgrading musl (1.1.20-r4 -> 1.1.16-r15)
(2/21) Installing binutils-libs (2.30-r1)
(3/21) Installing binutils (2.30-r1)
(4/21) Installing gmp (6.1.2-r0)
(5/21) Installing isl (0.17.1-r0)
(6/21) Installing libgomp (6.3.0-r4)
(7/21) Installing libatomic (6.3.0-r4)
(8/21) Installing pkgconf (1.3.7-r0)
(9/21) Installing libgcc (6.3.0-r4)
(10/21) Installing mpfr3 (3.1.5-r0)
(11/21) Installing mpc1 (1.0.3-r0)
(12/21) Installing libstdc++ (6.3.0-r4)
(13/21) Installing gcc (6.3.0-r4)
(14/21) Installing musl-dev (1.1.16-r15)
(15/21) Installing libressl2.5-libcrypto (2.5.5-r2)
(16/21) Installing libssh2 (1.8.2-r0)
(17/21) Installing libressl2.5-libssl (2.5.5-r2)
(18/21) Installing libcurl (7.61.1-r2)
(19/21) Installing curl (7.61.1-r2)
(20/21) Installing .build-deps (0)
(21/21) Downgrading musl-utils (1.1.20-r4 -> 1.1.16-r15)
Executing busybox-1.29.3-r10.trigger
OK: 115 MiB in 54 packages
/ # python
Error relocating /usr/local/lib/libpython3.7m.so.1.0: getrandom: symbol not found
/ # exit

# apk add --no-cache --virtual .build-deps 
fetch http://mirrors.aliyun.com/alpine/v3.6/main/x86_64/APKINDEX.tar.gz
fetch http://mirrors.aliyun.com/alpine/v3.6/community/x86_64/APKINDEX.tar.gz
(1/1) Installing .build-deps (0)
OK: 18 MiB in 36 packages
/ # apk add --no-cache --virtual gcc 
fetch http://mirrors.aliyun.com/alpine/v3.6/main/x86_64/APKINDEX.tar.gz
fetch http://mirrors.aliyun.com/alpine/v3.6/community/x86_64/APKINDEX.tar.gz
(1/12) Installing binutils-libs (2.30-r1)
(2/12) Installing binutils (2.30-r1)
(3/12) Installing gmp (6.1.2-r0)
(4/12) Installing isl (0.17.1-r0)
(5/12) Installing libgomp (6.3.0-r4)
(6/12) Installing libatomic (6.3.0-r4)
(7/12) Installing pkgconf (1.3.7-r0)
(8/12) Installing libgcc (6.3.0-r4)
(9/12) Installing mpfr3 (3.1.5-r0)
(10/12) Installing mpc1 (1.0.3-r0)
(11/12) Installing libstdc++ (6.3.0-r4)
(12/12) Installing gcc (6.3.0-r4)
Executing busybox-1.29.3-r10.trigger
OK: 102 MiB in 48 packages
/ # apk add --no-cache --virtual musl-dev
fetch http://mirrors.aliyun.com/alpine/v3.6/main/x86_64/APKINDEX.tar.gz
fetch http://mirrors.aliyun.com/alpine/v3.6/community/x86_64/APKINDEX.tar.gz
(1/1) Installing musl-dev (0)
OK: 102 MiB in 49 packages
/ # apk add --no-cache --virtual curl
fetch http://mirrors.aliyun.com/alpine/v3.6/main/x86_64/APKINDEX.tar.gz
fetch http://mirrors.aliyun.com/alpine/v3.6/community/x86_64/APKINDEX.tar.gz
(1/5) Installing libressl2.5-libcrypto (2.5.5-r2)
(2/5) Installing libssh2 (1.8.2-r0)
(3/5) Installing libressl2.5-libssl (2.5.5-r2)
(4/5) Installing libcurl (7.61.1-r2)
(5/5) Installing curl (7.61.1-r2)
Executing busybox-1.29.3-r10.trigger
OK: 105 MiB in 54 packages
/ # python
Python 3.7.3 (default, Mar 27 2019, 23:48:15) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

... What happen?

And it not depends on image of python. Image of 3 weeks ago and image of 4 days ago have same problem.
And I builded it success at several weeks ago...0.0?

PS: Image of 3 weeks ago: (bb1ccaa5880c). Image of 4 days ago:(f8aff02aba66)

Is this is apk`s problem?

Proposed the same reply. #9947

[ziserendexin]

all right, it isn't every thing ok.

Installing collected packages: zope.interface, constantly, incremental, attrs, six, Automat, idna, hyperlink, PyHamcrest, twisted, pyserial, pymodbus, pytzdata, python-dateutil, pendulum, redis, chardet, urllib3, certifi, requests, pytz, influxdb, msgpack-python, toml, dnspython, python-etcd, python-mimeparse, falcon, waitress, greenlet, gevent, logbook, xmltodict, python-snap7, bitstruct, diskcache, textparser, wrapt, python-can, cantools, tzlocal, apscheduler, pyCryptodome, u-msgpack-python, python-gitlab, jdcal, et-xmlfile, openpyxl, xlsxwriter, xlrd
  Running setup.py install for zope.interface: started
    Running setup.py install for zope.interface: finished with status 'done'
  Running setup.py install for twisted: started
    Running setup.py install for twisted: finished with status 'error'
    Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-qdpddr19/twisted/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-x0jj2u_z/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/twisted
    copying src/twisted/__main__.py -> build/lib.linux-x86_64-3.7/twisted
...........
lot of thing
..........
    copying src/twisted/words/xish/xpathparser.g -> build/lib.linux-x86_64-3.7/twisted/words/xish
    running build_ext
    building 'twisted.test.raiser' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/src
    creating build/temp.linux-x86_64-3.7/src/twisted
    creating build/temp.linux-x86_64-3.7/src/twisted/test
    gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/local/include/python3.7m -c src/twisted/test/raiser.c -o build/temp.linux-x86_64-3.7/src/twisted/test/raiser.o
    In file included from src/twisted/test/raiser.c:4:0:
    /usr/local/include/python3.7m/Python.h:11:20: fatal error: limits.h: No such file or directory
     #include <limits.h>
                        ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    
    ----------------------------------------
Command "/usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-qdpddr19/twisted/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-x0jj2u_z/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-qdpddr19/twisted/

dockerfile:

FROM python:latest
ENV TZ Asia/Shanghai
WORKDIR /usr/mabo/
COPY ./mabopython_requirements.txt ./simplecannet-0.0.1-py3-none-any.whl ./ 
RUN  echo http://mirrors.aliyun.com/alpine/v3.6/main > /etc/apk/repositories; \
    echo http://mirrors.aliyun.com/alpine/v3.6/community >> /etc/apk/repositories \
    && apk add --no-cache tzdata \
    && ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone \
    && apk add --no-cache --virtual .build-deps \
    && apk add --no-cache --virtual gcc \
    && apk add --no-cache --virtual musl-dev \
    && apk add --no-cache --virtual curl \
    && pip install -i https://pypi.tuna.tsinghua.edu.cn/simple --no-cache-dir -r mabopython_requirements.txt \
    && pip install --no-cache-dir simplecannet-0.0.1-py3-none-any.whl \
    && rm mabopython_requirements.txt \
    && rm simplecannet-0.0.1-py3-none-any.whl

requirements:(name is : mabopython_requirements.txt)

zope.interface
constantly
incremental
attrs
Automat
twisted
pymodbus
pendulum
redis
influxdb
msgpack-python
toml
python-etcd
falcon
waitress
greenlet
gevent
logbook
xmltodict
python-snap7
cantools
apscheduler
pyCryptodome
u_msgpack_python
python-gitlab
openpyxl
xlsxwriter
xlrd

[yosifkit]

@ziserendexin, First off, that Dockerfile shouldn't work since FROM python:latest would be Debian based and not Alpine. Assuming that you pulled python:alpine and tagged it as python:latest, it looks like you are doing a similar error as above in adding an apk mirror for the wrong Alpine Linux release. In this case, a mirror for Alpine 3.6 (http://mirrors.aliyun.com/alpine/v3.6/main) to an image based on Alpine 3.9.

[tianon]

For further help debugging your Dockerfiles/environments, please try the Docker Community Forums, the Docker Community Slack, or Stack Overflow.

[tianon]

(Closed since the original issue was resolved in #378 (comment), and turned out to be user error.)

[ziserendexin]

@ziserendexin, First off, that Dockerfile shouldn't work since FROM python:latest would be Debian based and not Alpine. Assuming that you pulled python:alpine and tagged it as python:latest, it looks like you are doing a similar error as above in adding an apk mirror for the wrong Alpine Linux release. In this case, a mirror for Alpine 3.6 (http://mirrors.aliyun.com/alpine/v3.6/main) to an image based on Alpine 3.9.

About the origin image, it's the Alpine 3.9.
I forget to show the download shell.
I use this shell to download and tag the image. :D

#！/bin/bash
tag=3.7-alpine
docker pull python:$tag
docker tag python:$tag python:latest
docker build -f mabo_base_python_Dockerfile -t base_python:$tag .

[ziserendexin]

@ziserendexin, First off, that Dockerfile shouldn't work since FROM python:latest would be Debian based and not Alpine. Assuming that you pulled python:alpine and tagged it as python:latest, it looks like you are doing a similar error as above in adding an apk mirror for the wrong Alpine Linux release. In this case, a mirror for Alpine 3.6 (http://mirrors.aliyun.com/alpine/v3.6/main) to an image based on Alpine 3.9.

According to your suggestion, he is working properly.Thanks a lot.

change to /alpine/v3.9/main， it helpful.

RUN  echo http://mirrors.aliyun.com/alpine/v3.9/main > /etc/apk/repositories; \
    echo http://mirrors.aliyun.com/alpine/v3.9/community >> /etc/apk/repositories 

Embarrassing mistake. Orz....