Skip to content

Commit

Permalink
Fix CI: Use 18.04 base docker image + run lint w/ py3.8 (#532)
Browse files Browse the repository at this point in the history
### Summary:

Fix CI builds by upgrading the base docker image to 18.04 and running lint with py3.8.

The ppa we use to install python in our docker images recently dropped support for Ubuntu 16.04.

Also, Python 3.6 recently passed EOL so pip complains and fails during the lint build

### Test Plan:

CI
  • Loading branch information
VivekPanyam authored Feb 17, 2022
1 parent de304c4 commit 925ca9f
Show file tree
Hide file tree
Showing 3 changed files with 9 additions and 13 deletions.
8 changes: 4 additions & 4 deletions build/ci/install_lint_deps.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
set -e

# We need a newer version of libstdc++ for infer to run
# We also need python3.6 for black to run
# We also need python3.8 for black to run
apt-get install -y --no-install-recommends software-properties-common
add-apt-repository -y ppa:ubuntu-toolchain-r/test
add-apt-repository -y ppa:deadsnakes/ppa
apt-get update
apt-get install -y --no-install-recommends libstdc++6 python3.6 python3.6-dev tzdata jq
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends libstdc++6 python3.8 python3.8-dev tzdata jq
apt-get install -y git

# Install infer
Expand All @@ -21,8 +21,8 @@ wget https://raw.githubusercontent.com/Sarcasm/run-clang-format/de6e8ca07d171a7f

# Install pip, black, and flake8
wget https://bootstrap.pypa.io/get-pip.py
python3.6 get-pip.py
python3.6 -m pip install black==19.10b0 flake8
python3.8 get-pip.py
python3.8 -m pip install black==19.10b0 flake8

# Get buildifier (for linting bazel files)
wget https://github.com/bazelbuild/buildtools/releases/download/2.2.1/buildifier -O /tmp/buildifier
Expand Down
2 changes: 1 addition & 1 deletion build/install_python_deps.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ set -e
NEUROPOD_PYTHON_BINARY="python${NEUROPOD_PYTHON_VERSION}"

# Install pip
wget https://bootstrap.pypa.io/pip/2.7/get-pip.py -O /tmp/get-pip.py
wget https://bootstrap.pypa.io/pip/3.6/get-pip.py -O /tmp/get-pip.py
${NEUROPOD_PYTHON_BINARY} /tmp/get-pip.py

# Setup a virtualenv
Expand Down
12 changes: 4 additions & 8 deletions build/neuropod.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# FROM ubuntu:16.04 as neuropod-base
ARG NEUROPOD_CUDA_VERSION=10.0
ARG NEUROPOD_CUDNN_VERSION=7
FROM nvidia/cuda:${NEUROPOD_CUDA_VERSION}-cudnn${NEUROPOD_CUDNN_VERSION}-runtime-ubuntu16.04 as neuropod-base
FROM nvidia/cuda:${NEUROPOD_CUDA_VERSION}-cudnn${NEUROPOD_CUDNN_VERSION}-runtime-ubuntu18.04 as neuropod-base

# Use utf8
ENV LC_ALL=C.UTF-8
Expand Down Expand Up @@ -34,15 +34,11 @@ ENV NEUROPOD_PYTHON_VERSION=$NEUROPOD_PYTHON_VERSION

# Install python
RUN sudo apt-get update && \
sudo apt-get install -y "python${NEUROPOD_PYTHON_VERSION}" "python${NEUROPOD_PYTHON_VERSION}-dev" && \
sudo apt-get install -y "python${NEUROPOD_PYTHON_VERSION}" \
"python${NEUROPOD_PYTHON_VERSION}-dev" \
"python${NEUROPOD_PYTHON_VERSION}-distutils" && \
ln -s "$(which python3)" /usr/bin/python

# For python 3.5, we need to install ffi
RUN if [ "${NEUROPOD_PYTHON_VERSION}" = "3.5" ] ; then sudo apt-get install -y libffi6 libffi-dev ; fi

# For python 3.8, we need to install distutils
RUN if [ "${NEUROPOD_PYTHON_VERSION}" = "3.8" ] ; then sudo apt-get install -y "python${NEUROPOD_PYTHON_VERSION}-distutils" ; fi

# Copy the python code into the image
RUN mkdir -p /usr/src/source/python /usr/src/source/neuropod/python
COPY build/install_python_deps.sh /usr/src/build/install_python_deps.sh
Expand Down

0 comments on commit 925ca9f

Please sign in to comment.