Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Putting install scripts into dockerfile #822

Closed
Closed
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
24c6467
moved install scripts into Dockerfile
shimwell Jul 25, 2022
c2dba7a
improved docs/comments
shimwell Jul 25, 2022
5255950
added PR 817
shimwell Jul 25, 2022
2f558e4
Merge branch 'develop' into moving_scripts_into_dockerfile
shimwell Jul 25, 2022
11b4be9
Merge branch 'develop' into moving_scripts_into_dockerfile
shimwell Aug 18, 2022
14a9291
[skip ci] compile cores review suggestion by @gonuke
shimwell Aug 18, 2022
ad88096
[skip ci] removed compiler arg fully
shimwell Aug 18, 2022
8c8d579
more consistent dirs for build
shimwell Aug 18, 2022
21af7e7
ordering and intenting args
shimwell Aug 18, 2022
5571c7a
reduced layers
shimwell Aug 18, 2022
5cdad9a
added missing hdf5 install dir env
shimwell Aug 18, 2022
e331db4
commented out moab static build
shimwell Aug 18, 2022
8507e48
corrected embree dirs
shimwell Aug 18, 2022
75d2419
using embree dir in rm -rf cmd
shimwell Aug 18, 2022
3881de6
cd to safe place
shimwell Aug 18, 2022
85884d6
removed 2nd mkdir build in embree compile
shimwell Aug 19, 2022
d4723f8
remove env for dag
shimwell Aug 23, 2022
7c93d5d
Remove making python path folders
shimwell Aug 24, 2022
421e7ad
prepend pythonpath as fix for MOAB build
gonuke Aug 27, 2022
48a97a0
keep previous pythonpath as well
gonuke Aug 27, 2022
f77d721
whitespace change to changelog
gonuke Aug 27, 2022
dc2945c
correct pythonpath
gonuke Aug 28, 2022
de0f876
need to mkdir for temp install
gonuke Aug 29, 2022
0b20fb3
add pythonpath for install as well
gonuke Aug 29, 2022
10162c5
comment out env source b/c it should all be in docker image
gonuke Aug 30, 2022
da899cc
Add build and install dir to the environment
gonuke Aug 30, 2022
5a84ef1
define build_dir from env
gonuke Aug 30, 2022
f5c70dd
read build dir from the env
gonuke Aug 30, 2022
488bf87
workflow changes to correct directory
gonuke Aug 30, 2022
e071b9f
Merge pull request #4 from gonuke/moving_scripts_into_dockerfile
shimwell Sep 1, 2022
5340950
moved hdf5_install_dir to global ARGs
shimwell Jan 1, 2023
1b438e8
single ENV entry used in multiple stages
shimwell Jan 1, 2023
b0dbb59
Merge branch 'svalinn:develop' into moving_scripts_into_dockerfile
shimwell Jan 2, 2023
28a5cbf
moab to 5.3.0
shimwell Jan 5, 2023
ebeaf59
[skip ci] added double-down install, started adding dagmc stage
shimwell Jan 5, 2023
6b89dd2
building and testing dagmc in docker
shimwell Jan 6, 2023
48ae87a
removed static from tag
shimwell Jan 6, 2023
0ff7d85
corrected install dir typo
shimwell Jan 13, 2023
ebc35e0
merge conflict solved
shimwell Jan 13, 2023
a30b95d
updated actions
shimwell Jan 13, 2023
a872a23
added docker buildx to step
shimwell Jan 14, 2023
92db99c
added path before make test cmd
shimwell Jan 19, 2023
47afa61
Merge branch 'svalinn:develop' into moving_scripts_into_dockerfile
shimwell Feb 2, 2023
97e1438
added inline CTEST_OUTPUT_ON_FAILURE=1
shimwell Feb 2, 2023
2669098
added ci stage for dagmc
shimwell Feb 6, 2023
b25e3c8
dagmc_dirs as ENV not ARGS
shimwell Feb 6, 2023
ad9c95c
corrected tag and container names
shimwell Feb 7, 2023
59e843a
pusing dagmc image so it can be found by testing image
shimwell Feb 7, 2023
33f0e07
combining two steps
shimwell Feb 8, 2023
051ec10
making use of qemu, removed buildx
shimwell Feb 8, 2023
8e4da44
housebuilding only if build-test works
shimwell Feb 8, 2023
5ddc8f8
housebuilding only if build-test works
shimwell Feb 8, 2023
4a123b5
docker qemu usage in dag step same as other stages
shimwell Feb 9, 2023
eb91b60
not running dag intsall step in container
shimwell Feb 9, 2023
d3edab8
disable reg tests as models missing
shimwell Feb 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
171 changes: 134 additions & 37 deletions CI/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,25 @@
# Svalinn DAGMC Dockerfile

ARG UBUNTU_VERSION=18.04

ARG ci_jobs=8
ARG MOAB_BRANCH=5.4.0
ARG EMBREE_BRANCH='v3.6.1'
ARG geant4_version=10.5.1

ARG HDF5_VERSION=1.10.4
# currently major version must be manually set to match HDF5 version
ARG HDF5_VERSION_major=1.10

ARG build_dir=/root/build_dir
ARG install_dir=/root/opt

# clang and gcc are options for the compiler but to use them CC and CXX must be set
# if clang then change cc to clang and cxx to clang++
# if gcc then change cc to gcc and cxx to g++
ARG CC=gcc
ARG CXX=g++

FROM ubuntu:${UBUNTU_VERSION} AS base

# Use bash as the default shell
Expand Down Expand Up @@ -34,63 +55,139 @@ RUN apt-get -y update; \
pip install cython;


# Copy scripts to docker image
RUN mkdir -p /root/etc/
COPY CI/ /root/etc/CI
#TODO move sh file contents into this Dockerfile
ENV docker_env=/root/etc/CI/env.sh

ENV build_dir=/root/build_dir
ENV install_dir=/root/opt


FROM base as external_deps

#setting the COMPILER variable
ARG COMPILER=gcc
ENV COMPILER=${COMPILER}
gonuke marked this conversation as resolved.
Show resolved Hide resolved

# Set Geant4 env variable
ARG build_dir
ARG install_dir
ARG COMPILER
ARG ci_jobs
ARG geant4_version
ARG CC
ARG CXX
ARG EMBREE_BRANCH

ENV geant4_basename=geant4-v${geant4_version}
ENV geant4_tarball=${geant4_basename}.tar.gz
ENV geant4_shasum=2397eb859dc4de095ff66059d8bda9f060fdc42e10469dd7890946293eeb0e39
ENV geant4_build_dir=${build_dir}/geant4
ENV geant4_install_dir=${install_dir}/geant4

# Build Geant4
#TODO move sh file contents into this Dockerfile
RUN /root/etc/CI/docker/build_geant4.sh

RUN mkdir -p ${geant4_build_dir}/bld
RUN cd ${geant4_build_dir} && \
wget https://gitlab.cern.ch/geant4/geant4/-/archive/v${geant4_version}/${geant4_tarball} --no-check-certificate && \
tar -xzf ${geant4_tarball} && \
cd bld && \
cmake ../${geant4_basename} -DBUILD_STATIC_LIBS=ON \
-DGEANT4_USE_SYSTEM_EXPAT=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=${CC} \
-DCMAKE_CXX_COMPILER=${CXX} \
-DCMAKE_INSTALL_RPATH=${geant4_install_dir}/lib \
-DCMAKE_INSTALL_PREFIX=${geant4_install_dir} && \
make -j${ci_jobs} && \
make install && \
cd && \
rm -rf ${geant4_build_dir}

# these two double down ENVs don't appear to be used yet
# However they were in orginal dockerfile, so they have been kept
# TODO add double down to the dockerfile
ENV double_down_build_dir=${build_dir}/double-down/
ENV double_down_install_dir=${install_dir}/double-down/

# Build Embree
#TODO move sh file contents into this Dockerfile
RUN /root/etc/CI/docker/build_embree.sh
ENV embree_install_dir=${install_dir}/embree

RUN mkdir EMBREE && \
shimwell marked this conversation as resolved.
Show resolved Hide resolved
cd EMBREE && \
git clone -b ${EMBREE_BRANCH} https://github.com/embree/embree && \
mkdir build && \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer needed - done in #103

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I am not quite sure which part of this embree clone/install was done in #103

I have removed the 2nd mkdir build if that is what was needed

shimwell marked this conversation as resolved.
Show resolved Hide resolved
cd build && \
cmake ../embree -DCMAKE_INSTALL_PREFIX=${embree_install_dir} \
-DEMBREE_ISPC_SUPPORT=OFF \
-DEMBREE_TASKING_SYSTEM=INTERNAL \
-DEMBREE_TUTORIALS=OFF \
-DEMBREE_TBB_ROOT=/usr && \
make -j2 && make -j2 install && \
shimwell marked this conversation as resolved.
Show resolved Hide resolved
rm -rf $HOME/EMBREE/embree
shimwell marked this conversation as resolved.
Show resolved Hide resolved


FROM external_deps AS hdf5
gonuke marked this conversation as resolved.
Show resolved Hide resolved

# Set HDF5 env variable
ARG build_dir
ARG install_dir
ARG ci_jobs
ARG HDF5_VERSION
ARG HDF5_VERSION_major
ARG CC
ARG CXX

ENV hdf5_build_dir=${build_dir}/hdf5
ENV hdf5_install_dir=${install_dir}/hdf5

# Build HDF5
# HDF5 argument possible value: 1.10.4 or system
ARG HDF5=1.10.4
ENV HDF5_VERSION=${HDF5}
#TODO move sh file contents into this Dockerfile
RUN /root/etc/CI/docker/build_hdf5.sh
RUN mkdir -p ${hdf5_build_dir}/bld && \
cd ${hdf5_build_dir} && \
wget https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-${HDF5_VERSION_major}/hdf5-${HDF5_VERSION}/src/hdf5-${HDF5_VERSION}.tar.gz && \
tar -xzf hdf5-${HDF5_VERSION}.tar.gz && \
cd bld && \
../hdf5-${HDF5_VERSION}/configure --enable-shared \
--prefix=${hdf5_install_dir} \
CC=${CC} \
CXX=${CXX} && \
make -j${ci_jobs} && \
make install && \
cd && \
rm -rf ${hdf5_build_dir}


FROM hdf5 AS moab

ENV PYTHONPATH=/root/build_dir/moab/bld/pymoab/lib/python3.6/site-packages
gonuke marked this conversation as resolved.
Show resolved Hide resolved
ARG ci_jobs
ARG MOAB_BRANCH
ARG CC
ARG CXX
ARG build_dir
ARG install_dir

# Set MOAB env variable
ENV moab_build_dir=${build_dir}/moab
ENV moab_install_dir=${install_dir}/moab

ARG MOAB=5.3.0
ENV MOAB_VERSION ${MOAB}
#TODO move sh file contents into this Dockerfile
RUN if [ "${MOAB_VERSION}" != "master" ] && [ "${MOAB_VERSION}" != "develop" ]; then \
/root/etc/CI/docker/build_moab.sh; \
fi;
RUN mkdir -p ${moab_build_dir}/bld
RUN cd ${moab_build_dir} && \
shimwell marked this conversation as resolved.
Show resolved Hide resolved
git clone -b ${MOAB_BRANCH} --depth 1 https://bitbucket.org/fathomteam/moab && \
cd bld && \
cmake ../moab -DENABLE_HDF5=ON -DHDF5_ROOT=${hdf5_install_dir} \
-DENABLE_BLASLAPACK=OFF \
-DENABLE_FORTRAN=OFF \
-DCMAKE_INSTALL_PREFIX=${moab_install_dir} \
-DCMAKE_C_COMPILER=${CC} \
-DCMAKE_CXX_COMPILER=${CXX} \
-DBUILD_SHARED_LIBS=OFF && \
make -j${ci_jobs} && \
make install && \
rm -rf * && \
cmake ../moab -DENABLE_HDF5=ON -DHDF5_ROOT=${hdf5_install_dir} \
-DENABLE_PYMOAB=ON \
-DENABLE_BLASLAPACK=OFF \
-DENABLE_FORTRAN=OFF \
-DCMAKE_INSTALL_PREFIX=${moab_install_dir} \
-DCMAKE_C_COMPILER=${CC} \
-DCMAKE_CXX_COMPILER=${CXX} \
-DBUILD_SHARED_LIBS=ON \
-DCMAKE_INSTALL_RPATH=${hdf5_install_dir}/lib:${moab_install_dir}/lib && \
# ensure that the installation directory for python packages exist
for d in $(echo $PYTHONPATH | tr : '\n'); do mkdir -p $d; done && \
shimwell marked this conversation as resolved.
Show resolved Hide resolved
shimwell marked this conversation as resolved.
Show resolved Hide resolved
make -j${ci_jobs} && \
make install && \
cd && \
rm -rf ${moab_build_dir}

ENV dagmc_build_dir=${build_dir}/DAGMC
ENV dagmc_install_dir=${install_dir}/DAGMC

ENV dagmc_build_dir_shared=${dagmc_build_dir}-shared
ENV dagmc_install_dir_shared=${dagmc_install_dir}-shared

ENV dagmc_build_dir_static=${dagmc_build_dir}-static
ENV dagmc_install_dir_static=${dagmc_install_dir}-static
shimwell marked this conversation as resolved.
Show resolved Hide resolved
shimwell marked this conversation as resolved.
Show resolved Hide resolved

RUN python -c "import pymoab"
1 change: 1 addition & 0 deletions doc/CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Next version
* Using multi stage Dockerfile to reduce the number of Dockerfile (#813)
* Adding safe folder to allow CI to compile DAGMC (#814)
* Correction to CMake variable name in OpenMC install instructions (#817)
* Move install scripts into Dockerfile (#822)
* Updating documentation publishing URL (#823)


Expand Down