Does apt install of python3 affect python in slim?

[tisdall]

I was looking at reducing my build size by using the slim build and then only including build dependencies while building and then purging them. Everything seems to work fine. However, I noticed that when I include glib that apt pulls in python3. I see in #253 that this is to be expected, but I was wondering if this has any potential in breaking the python already within the container.

Specifically, does having something like this in Dockerfile have any potential in breaking something?

RUN apt-get update && \
    apt-get install -y --no-install-recommends python3 && \
    apt-get purge --autoremove -y python3 && \
    rm -rf /var/lib/apt/lists/*

This is obviously contrived, but does apt adding it's own python3 and removing it alter things that need to not be altered?

[ad-m-ss]

This is not a direct answer to your question, but if you want to optimize the image, you can think about multi-stage build for building dependencies , then you don't need care to remove Python because it will be in a different stage:

FROM python:3.10.7-bullseye AS build-pip
ENV PIP_CACHE_DIR=/var/cache/buildkit/pip
RUN mkdir -p $PIP_CACHE_DIR
RUN --mount=type=cache,target=/var/cache/buildkit/pip \
    pip install -U pip wheel

FROM build-pip AS build-app
COPY ./requirements.txt .
RUN --mount=type=cache,target=/var/cache/buildkit/pip \
    pip wheel --wheel-dir /wheels -r requirements.txt

FROM python:3.10.7-slim-bullseye

RUN --mount=type=cache,target=/var/cache/apt apt-get update \
    && apt-get install -y --no-install-recommends unixodbc git libxml2 openssh-client \
    && rm -rf /var/lib/apt/lists/*
ENV PIP_CACHE_DIR=/var/cache/buildkit/pip
RUN mkdir -p $PIP_CACHE_DIR

# Install requirements
WORKDIR /app
COPY requirements.txt ./

COPY --from=build-app /wheels /wheels

RUN --mount=type=cache,target=/var/cache/buildkit/pip \
    pip install --find-links /wheels --no-index -r requirements.txt

COPY . .

[tianon]

Installing Python from APT ends up with another copy/version of Python in your image, which definitely causes bloat -- it shouldn't directly interfere with anything for this image's installation of Python, but it does increase the chances that you accidentally invoke the wrong version and then see odd behavior (different available packages, etc).

[Mongkol67]

This is not a direct answer to your question, but if you want to optimize the image, you can think about multi-stage build for building dependencies , then you don't need care to remove Python because it will be in a different stage:

FROM python:3.10.7-bullseye AS build-pip
ENV PIP_CACHE_DIR=/var/cache/buildkit/pip
RUN mkdir -p $PIP_CACHE_DIR
RUN --mount=type=cache,target=/var/cache/buildkit/pip \
    pip install -U pip wheel

FROM build-pip AS build-app
COPY ./requirements.txt .
RUN --mount=type=cache,target=/var/cache/buildkit/pip \
    pip wheel --wheel-dir /wheels -r requirements.txt

FROM python:3.10.7-slim-bullseye

RUN --mount=type=cache,target=/var/cache/apt apt-get update \
    && apt-get install -y --no-install-recommends unixodbc git libxml2 openssh-client \
    && rm -rf /var/lib/apt/lists/*
ENV PIP_CACHE_DIR=/var/cache/buildkit/pip
RUN mkdir -p $PIP_CACHE_DIR

# Install requirements
WORKDIR /app
COPY requirements.txt ./

COPY --from=build-metadata /wheels /wheels

RUN --mount=type=cache,target=/var/cache/buildkit/pip \
    pip install --find-links /wheels --no-index -r requirements.txt

COPY . .

[tisdall]

Installing Python from APT ends up with another copy/version of Python in your image, which definitely causes bloat -- it shouldn't directly interfere with anything for this image's installation of Python, but it does increase the chances that you accidentally invoke the wrong version and then see odd behavior (different available packages, etc).

In my case, I'm adding it and then removing it in a single RUN, so no bloat. However, it's possible that apt does something like change "alternatives" (maybe that's not used at all for python3 added to the container?) and then doesn't return everything back to it was before apt install python3. Or similar to what you said about using the wrong version, my compiling of dependencies in that intermediate step accidentally uses the wrong python. These are my concerns.

I probably shouldn't have trimmed my example down so much... What I'm doing is like this:

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        gcc g++ libtool libffi-dev libcurl4-openssl-dev libpq-dev libbz2-dev libc6-dev \
        libevent-dev libgmp-dev liblzma-dev libncurses5-dev libncursesw5-dev \
        libreadline-dev libsqlite3-dev libssl-dev libxml2-dev libyaml-dev zlib1g-dev \
        curl git \
        libpq5 openssl gettext && \
    python setup-docker.py && \
    apt-get purge --autoremove -y \
        gcc g++ libtool libffi-dev libcurl4-openssl-dev libpq-dev libbz2-dev libc6-dev \
        libevent-dev libgmp-dev liblzma-dev libncurses5-dev libncursesw5-dev \
        libreadline-dev libsqlite3-dev libssl-dev libxml2-dev libyaml-dev zlib1g-dev \
        curl git && \
    rm -rf /var/lib/apt/lists/*

The python setup-docker.py step is doing pip installs which is using some of those dependencies for compiling. This version isn't pulling in python3, but if I include the glib components then python3 gets pulled in and makes we wonder if the wrong dev headers are used or who knows what!

[tisdall]

@ad-m-ss , I really appreciate the reply! I'm not familiar with this technique so I'll have to look into it. Does this work as a single Dockerfile or is this broken down in separate files? (found the docs! thanks again!)

[tisdall]

@ad-m-ss Should COPY --from=build-metadata /wheels /wheels be COPY --from=build-app /wheels /wheels? I'm still trying to figure this out, but I don't see where build-metadata is defined and /wheels seems to be created in build-app.

[yosifkit]

The python binaries and associated libraries provided by the image are installed to /usr/local/* so it isn't affected by any of the Debian alternatives stuff. The PATH is set so that /usr/local/bin is first.

I think the second python can become a problem when a python file is directly executed that uses a static path to python in the hashbang rather than #!/usr/bin/env python (so, python ./script.py is fine, while ./script.py, might do the wrong thing).

[ad-m-ss]

@ad-m-ss Should COPY --from=build-metadata /wheels /wheels be COPY --from=build-app /wheels /wheels?

Yes, I updated my comment. This is the result of an incomplete redaction from a private Dockerfile. It's a bit more complicated for us. Sorry for the confusion.

[tisdall]

@ad-m-ss No worries! Just wanted to confirm I was understanding it correctly. Thank you!

[tisdall]

I used a multi-stage build and that worked well for me in preventing extra versions of python.