oar-docker is a set of docker images especially configured for deploying your own OAR cluster. The main idea is to have a mini development cluster with a frontend, a server and some nodes that launch in just a few seconds on a simple laptop.
- Various case scenarios may affect you:
- Quickly test OAR on a cluster
- Gain time: a ten-node cluster (or more) is launched in just a few seconds and is cleaned in less than a second.
- Save resources: docker allows the user to pool the node between various systems, resource utilization is thus considerably reduced.
- Synced volume : allowing you to continue working on your host machine, but use the resources in the guest machine to compile or run your project.
- Requirements:
- python >= 2.7
- docker >= 1.3
- cgroup v1
You can install, upgrade, uninstall oar-docker with these commands:
$ pip install [--user] oar-docker $ pip install [--user] --upgrade oar-docker $ pip uninstall oar-docker
Or from git (last development version):
$ pip install git+https://github.com/oar-team/oar-docker.git
Or if you already pulled the sources:
$ pip install path/to/sources
Using a virtualenv may help overcome issues between python and your distribution.
Usage: oardocker [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
Manage a small OAR developpement cluster with docker.
Options:
--workdir DIRECTORY Changes the folder to operate on.
--docker-host TEXT The docker socket [default:
unix://var/run/docker.sock].
--cgroup-path TEXT The cgroup file system path [default: /sys/fs/cgroup].
--docker-binary TEXT The docker client binary [default: docker].
--verbose Verbose mode.
--debug Enable debugging
--version Show the version and exit.
-h, --help Show this message and exit.
Commands:
build Build base images
clean Remove all stopped containers and untagged...
connect Connect to a node.
destroy Stop containers and remove all images
exec Run a command in an existing node.
init Initialize a new environment.
install Install and configure OAR from src
logs Fetch the logs of all nodes or only one.
reset Restart the containers
start Create and start the nodes
status Output status of the cluster
stop Stop and remove all nodes
To get started with oar-docker, the first thing to do is to initialize a project:
$ oardocker init
If you already have OAR sources, the best is to initialize directly the oardocker project in the OAR sources directory:
$ cd path/to/oar/src $ oardocker init
You have to do this only once. It allows you to import the Dockerfiles and other configuration files.
We then launch the base image build:
$ oardocker build
Now, we have to install OAR. To do this, several options are available.
If you already have the OAR sources:
$ oardocker install /path/to/oar_src
Or if you want to install from tarball:
$ oardocker install http://oar-ftp.imag.fr/oar/2.5/sources/testing/oar-2.5.4+rc4.tar.gz
You can also launch the installation from a git repository:
$ oardocker install git+https://github.com/oar-team/oar.git
We start a OAR cluster with 5 nodes:
$ oardocker start -n 5
It is possible to share directories between host machines and
all containers with the -v option:
$ oardocker start -v $PWD:/oar_src -v /my/custom/lib:/usr/local/ma_lib
To manage the cluster:
$ oardocker connect frontend|server|nodeXX $ oardocker logs [frontend|server|nodeXX]
To clean:
$ oardocker stop # stops and removes all containers $ oardocker clean # removes all stopped containers (failed) and the untagged images <none:none> $ oardocker destroy # removes all images and containers
With oar-docker, it is possible to chain all commands to go faster:
$ oardocker init -f build install oar-2.5.4+rc4.tar.gz start -n 4 connect -l root frontend
For instance, to develop on OAR, we often need to install OAR, start the cluster and connect to it:
$ oardocker install $PWD start -n 10 -v $PWD:/home/docker/oar_src connect frontend
One last thing to know. The stop command is automatically launched before
every start, install and build ... If we launch multiple times the
last command, we will always obtain the same result. It can be useful to
experiment and develop (even) faster.
oardocker ≥ 1.6.0 does not create the /dev/oar_cgroups_links/ and /dev/cpuset in oardocker install, to let OAR take care of it so that the concerned part of job_resource_manager_cgroup.pl is actually tested.
This breaks job_resource_manager_cgroup.pl before OAR 2.5.9+g5k5. See .oardocker/init-scripts/prepare_oar_cgroup.sh to revert to the old behaviour, by setting:
CREATE_OAR_CGROUPS_LINKS=yes
To use oardocker on a Debian 11 (bullseye) system, one must activate Linux cgroup v1. This can be achieved by including in the linux kernel cmdline: systemd.legacy_systemd_cgroup_controller=true (add those options to your bootloader configuration).
oardocker can be installed in a python3 venv:
$ python3 -m venv oardocker $ cd oardocker $ . bin/activate $ pip install oar-docker $ oardocker init -e bullseye $ oardocker build $ oardocker install path/to/oar/git/repository $ oardocker start $ oardocker connect $ ...
By default, oardocker forwards the 80 and 6667 TCP ports to the frontend
container, for the OAR web services. To add other TCP ports forwarding to
the frontend, modify the .oardocker/manifest.json file, adding extra lines
in the net_services array. For instance:
"network_services": [
["Netcat", " (tcp test)", "5000", "TCP: "],
["Secure web service", "/secure", "443", "https://"],
["Python API", "/newoarapi", "6668"],
...
]
Which translates to the following forwardings:
******************** Network Services *********************
Netcat: TCP: localhost:45000 (tcp test)
Secure web service: https://localhost:40443/secure
Python API: http://localhost:46668/newoarapi
...
***********************************************************
Only the numerical port numbers really matters, texts are only informative.
oar-docker is a development project and a testing one. It is in no way secure. Besides, the private ssh key used is also insecured since it is public (you can find it in the sources).