Skip to content

Commit

Permalink
Create BioDynaMo singularity image (#244)
Browse files Browse the repository at this point in the history
  • Loading branch information
BJackal authored Mar 16, 2022
1 parent cdd894e commit ebeeb7e
Show file tree
Hide file tree
Showing 3 changed files with 211 additions and 0 deletions.
16 changes: 16 additions & 0 deletions .github/workflows/singularity-integration.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Singularity Integration

on: [push, pull_request]

jobs:
build:
name: Build
runs-on: ubuntu-latest
container:
image: quay.io/singularity/singularity:v3.9.0
options: "--privileged --workdir /data"
steps:
- uses: actions/checkout@v2
- name: Build singularity image...
run: |
singularity build Singularity.sif Singularity
121 changes: 121 additions & 0 deletions Singularity
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
Bootstrap:docker
From:ubuntu:18.04

%environment

%post -c /bin/bash

export DEBIAN_FRONTEND=noninteractive
export TZ=Europe/Berlin

apt-get -y update &&
apt-get -y install python &&
apt-get -y install python3 &&
apt-get -y install git &&
apt-get -y install curl &&
apt-get -y install make &&
apt-get -y install gcc &&
apt-get -y install g++ &&
apt-get -y install wget &&
apt-get -y install wamerican &&
apt-get -y install libffi-dev &&
apt-get -y install libncurses5-dev &&
apt-get -y install zlib1g &&
apt-get -y install zlib1g-dev &&
apt-get -y install bzip2 &&
apt-get -y install zlib1g-dev &&
apt-get -y install aptitude &&
aptitude -y install libreadline-dev &&
apt-get -y install libssl-dev &&
apt-get -y install libsqlite3-dev &&
apt-get -y install apt-utils &&
apt-get -y install openmpi-bin &&
apt-get -y install libopenmpi-dev &&
apt-get -y install libxkbcommon-x11-dev &&
apt-get -y install bsdmainutils &&
apt-get -y install clang &&
apt-get -y install clang-format &&
apt-get -y install clang-tidy &&
apt-get -y install doxygen &&
apt-get -y install graphviz &&
apt-get -y install libxml2-dev &&
apt-get -y install llvm-7 &&
apt-get -y install llvm-7-dev &&
apt-get -y install llvm-7-runtime &&
apt-get -y install valgrind &&
apt-get -y install libgsl-dev &&
apt-get -y install freeglut3-dev &&
apt-get -y install libbz2-dev &&
apt-get -y install libnuma-dev &&
apt-get -y install libomp5 &&
apt-get -y install libomp-dev &&
apt-get -y install libopenmpi-dev &&
apt-get -y install libpthread-stubs0-dev &&
apt-get -y install zlib1g-dev &&
apt-get -y install libbz2-dev &&
apt-get -y install libffi-dev &&
apt-get -y install liblzma-dev &&
apt-get -y install libreadline-dev &&
apt-get -y install libsqlite3-dev &&
apt-get -y install libssl-dev &&
apt-get -y install python-openssl &&
apt-get -y install tk-dev &&
apt-get -y install xz-utils &&
apt-get -y install zlib1g-dev &&
apt-get -y install sudo &&
apt-get -y install software-properties-common

apt-get -y install locales locales-all
dpkg-reconfigure locales
locale-gen

wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -b -f -p /miniconda3/
rm Miniconda3-latest-Linux-x86_64.sh

export PATH="/miniconda3/bin:$PATH"
conda install -y -c conda-forge pip numpy
conda update -y --all
apt -y install git
apt -y install curl
sudo rm -rf ~/.pyenv
curl https://pyenv.run | bash
export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init --path)"
eval "$(pyenv virtualenv-init -)"
eval "$(pyenv init -)"

export PYENV_ROOT="/opt/pyenv"
export PATH="/opt/pyenv/bin:$PATH"
curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.9.1
export PATH="/opt/pyenv/versions/3.9.1/bin:$PATH"
export PATH="/opt/pyenv/versions/3.9.1/bin/:$PATH"

wget https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py
python3 -m pip install -U pip
python3 -m pip install --upgrade pip
python3 -m pip install numpy
apt-get install -y python3-pip
apt-get install -y freeglut3-dev valgrind

pip install cmake --upgrade
pyenv global 3.9.1

apt-get -y dist-upgrade
sudo apt-get -y install ninja-build

git clone https://github.com/BioDynaMo/biodynamo.git
cd biodynamo
export SILENT_INSTALL=1
./prerequisites.sh all

cmake -G Ninja \
-Dparaview=OFF \
-DCMAKE_BUILD_TYPE=Release \
-B build

cmake --build build --parallel

%runscript
74 changes: 74 additions & 0 deletions doc/user_guide/Singularity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
title: "Singularity"
date: "2022-15-02"
path: "/docs/userguide/singularity/"
meta_title: "BioDynaMo User Guide"
meta_description: "This is the singularity page."
toc: true
image: ""
next:
url: "/docs/userguide/singularity/"
title: "Singularity"
description: "This is the diffusion page."
sidebar: "userguide"
keywords:
-Singularity
-HPC
-performance
---
In addition to running BioDynaMo on standard computers, users can also utilise the BioDynaMo frame work on high
performance computers (HPCs). Due to many HPCs having higher security than standard computers however it is likely
that a user wishing to do this will need to build BioDynaMo within a container on the chosen HPC.

For this we have chosen to Singularity. Singularity is an open source programme for operating-system-level virtualization.
Singularity works simillar to other popular container programmes such as Docker. However, due to its higher level of
security it has seen more widespread uptake by HPCs and we thus beleive the best choise for users.

To create our singularity image for BioDynaMo we can use on of two methods.

## Method 1, directly building on the HPC.
For the first method, we directly import the biodynamo Singularity file from our local PC to the desired HPC as follows.
Firstly, we must scp into the desired HPC from our local PC. This can be done directly through the terminal as follows.
```bash
ssh -x [HPC.ADDRESS]
```
We then must secure copy the BioDynaMo singularity file from the chosen PC to the local HPC.
```bash
scp [user@]SRC_LOCALPC/SingularityFileLocation:]Singularity ~/
```
From here we can build BioDynaMo utilising the fake root feature of singularity.
```bash
singularity build --fakeroot Singularity.sif Singularity
```
Fakeroot is not needed if you have sudo rights on the HPC but this is highly unlikely for most individuals.
In that case however we simply amend the above to:
```bash
sudo singularity build Singularity.sif Singularity
```
Then simply run the BioDynaMo singularity container using:
```bash
singularity run Singularity.sif
```
## Method 2, Export image from local PC to HPC.
If you cannot utilise either sudo or fakeroot on the HPC, we can isntead build the BioDynaMo image on our local PC and export the image.
This works simillar to method 1 but with a change in order. Firstly, we must build our image on the local PC with sudo rights:
```bash
sudo singularity build Singularity.sif Singularity
```
Then ssh into the HPC.
```bash
ssh -x [HPC.ADDRESS]
```
secure copy the image from our PC to the local HPC.
```bash
scp [user@]SRC_LOCALPC/SingularityFileLocation:]Singularity.sif ~/
```
Finally we once again run the Singularity image on the HPC:
```bash
singularity run Singularity.sif
```

If you wish to read further about Singularity, you can find a substantial amount of information on the
singularity home website: https://sylabs.io/guides/3.5/user-guide/introduction.html. Including many tutorials and more
complex use cases.

0 comments on commit ebeeb7e

Please sign in to comment.