SpringRTS Build Environment (Docker)

Preamble

This page is intended for users that are already proficient with Git and Unix shell in general (even if on Windows, since the Windows build environment leverages WSL). If you are looking to set up a Windows build environment and can't figure out how to follow this page, try to follow the step-by-step instructions instead.

Introduction

A docker image based on Ubuntu is available to build the engine and provide artifacts without need to set up the compilation toolchain. Alternatively, the image can also be used as a development environment. There is no need to know how to use Docker to compile the engine, as convenient helper scripts are provided for this use case.

Quickstart

Prerequisites

• Access to a Docker environment. If in doubt: Docker Desktop is easier to install, Docker Server is more customizable.
• A local copy of the repository with submodules properly initialized. Quick one-liner:

git clone https://github.com/beyond-all-reason/spring -b BAR105 --recursive

Then, go to the appropriate section depending on your use case:

• Building / Compiling
• Development environment

Windows + WSL

You can use the Windows Subsystem for Linux in order to install Docker on Windows. As noted above, Docker Desktop is easier and will prompt you to set up WSL during installation. For more customization, install WSL manually and then install Docker Engine on Ubuntu in WSL.

In both cases, make sure to clone the repository on the Linux file system, as recommended in both the WSL and Docker docs. Do not use the Windows file system as access from WSL via /mnt/c/ is extremely slow. We recommend cloning only into /home/<username>/ from WSL (\\wsl$\Ubuntu\home\<username>\ from Windows) to avoid file permissions issues.

Please follow the step-by-step instructions if you need any help with the WSL setup.

Building / Compiling

First compilation

cd docker-build
./init_container.sh      # one-time setup, initializes the container
./build.sh               # compiles a build targeting Windows users
./build.sh -p linux-64   # compiles a build targeting Linux users

This will take a long time. Once done, the build artifacts will be located at <spring repository>/build-<platform>-<build type>/install. For example, for the default ./build.sh call: <spring repository>/build-windows-64-RELWITHDEBINFO/install

See Parameters below for full details on how to tweak ./build.sh build flags.

Recompiling

cd docker-build && ./build.sh

Recompilations are much faster as the compilation cache is stored locally. Note that ./init_container.sh does not need to be run again, ./build.sh will automatically detect if the container image is outdated and will tell you to run ./init_container.sh again when needed.

Automatically using latest build in game

The build artifacts located at <spring repository>/build-<platform>-<build type>/install can automatically be made available in your local game environment by creating a symbolic link to them under <game directory>/data/engine/local-build. After the symbolic link has been created, the latest locally compiled engine build will always be available to use with the game.

• On Linux:

ln -s "<spring repository>/build-linux-64-RELWITHDEBINFO/install" "<game directory>/data/engine/local-build"

• On Windows + WSL, via Powershell as administrator:

New-Item -ItemType SymbolicLink -Path "<game directory>/data/engine/local-build" -Target "\\wsl$\Ubuntu\<spring repository>\build-windows-64-RELWITHDEBINFO\install"

Development environment

Build the container

cd docker-build
./init_container.sh      # one-time setup, initializes the container

Start the container

docker run -it bar-spring:latest dev

You will get a bash shell with the spring source code checked out ready to make modifications and compile by typing make or cmake --build .:

----------------------------------------------
SpringRTS development environment has been set up successfully
Source code directory: /spring
Build directory: /spring/build
----------------------------------------------
<springdev> root@e0d3fbe4fffd:/spring/build$ _

Troubleshooting build errors

Running docker as non-root user

If the ./init_container.sh step fails with permission denied, make sure to run docker as a non-root user.

Out of memory; signal killed terminated program

If running in a virtual machine and the ./build.sh command exits with a killed signal terminated error, make sure the virtual machine has at least 12GB of memory.

Delete artifacts on build failure

If a build fails for any reason, make sure to delete the docker artifacts before trying to run the ./build.sh command again.

Docker image details

The image is based on Ubuntu 18.04 so created Linux binaries are runnable on that version and later. Windows binaries are built using a MXE cross-compile environment.

The image utilizes ccache to speedup consecutive compilation processes. It is recommended to reuse the cache data in different containers by configuring the ccache volume accordingly.

Building the container

If building yourself via docker build, the following arguments are available (outdated info, UPDATE):

Parameter	Default	Description
mxe_version	02852a7b690aa411ce2a2089deea25a7292a33d6	Version of MXE to use. This can be a commit hash but also a tag or branch name
mxe_gcc	gcc11	Defines which gcc plugin to use in MXE
cmake_version	3.16.*	Defines a CMake version string to be used when installing CMake via pip
ccache_version	v4.5.1	Version of ccache to use. This can be a commit hash but also a tag or branch name

Commands available

The container can be used with the following commands:

1. build
2. dev
3. shell

Parameters

Both build and run commands can be configured with:

Parameter	Default	Description
-p	windows-64	Which target platform to setup build for (use "linux-64" or "windows-64")
-t	RELWITHDEBINFO	Build type: RELWITHDEBINFO (default), DEBUG, RELEASE, PROFILE
-C	<empty>	additional cmake flags. eg. -C -DTRACY_ENABLE=1. Multiple flags can be passed by specifying the flag multiple times.
-k	false	whether to pack and publish artifacts to the ${buildfolder}/publish dir.
-s	1	Enable stripping of symbols from artifacts
-b	BAR105	The branch to build from the spring project. Only relevant if you set -l to false
-u	https://github.com/beyond-all-reason/spring	URL to a Spring Git repository. Only relevant if you set -l to false
-a	https://github.com/beyond-all-reason	Prefix for the URLs used to clone auxiliary repos. The following URL will e.g. be cloned: https://github.com/beyond-all-reason/BARbarIAn
-c	<empty>	Configures gcc's -march and -mtuneflag
-d	0	Dummy mode: to not actually clone or compile but just produce zero-size artifacts
-e	1	Enable ccache
-f	<empty>	CXXFLAGS and CFLAGS flags for gcc
-l	true	whether this build is a local build. If false, artifacts will be published and repo branch will be cloned out.
-o	false	Disable headless and dedicated builds, default off
-r	depends on built type	CXX flags override for selected build type
-w	true	suppress outdated container warning
-z	false	Enable generation ccache debug data in /ccache_dbg

build command

The container can be started by passing build as the first parameter to run the compilation process for one target platform and produce an archive package that will contain the runnable engine with all library dependencies.

Internally, the build process consists of multiple steps. Each step is represented by a shell script and they are run consecutively:

1. 00_setup.sh
2. 01_clone.sh
3. 02_configure.sh
4. 04_fill_portable_dir_linux-64.shand 04_fill_portable_dir_windows-64.sh
5. 05_fill_debugsymbol_dir.sh
6. 06_fill_build_options_file.sh
7. 07_pack_build_artifacts.sh
8. 08_copy_to_publish_dir.sh

The following directories contain output data and can be used as volumes to access the files (either manually or via Docker Desktop):

1. /publish: directory the build process will copy all produced artifacts into.
2. /ccache: directory where ccache data will be placed (the image is using ccache to speed up the build process). To make use of the cache you have to run different build runs using the same cache data directory.
3. /ccache_dbg: directory where ccache debug data will be placed.

Example:

docker run -v D:\myspringbuild:/publish -it springrts-build

Building from another GitHub Repository

Build branch gl4 from repository at https://github.com/beyond-all-reason/spring:

docker run -it springrts-build build -u https://github.com/beyond-all-reason/spring -b gl4 -p linux-64

dev command

When starting the development mode, then only the first three steps are executed compared to the build command:

1. 00_setup.sh
2. 01_clone.sh
3. 02_configure.sh

So the source code will be cloned into the container and the build will be configured using CMake. You will find yourself inside the development shell ready to start the compilation process.

If you started the container with a raw /bin/bash shell (didn't use build or dev command) you can start the dev shell manually by running dev.sh with the usual arguments.

When working in the development shell you can always start the step scripts manually if needed. But be aware that running e.g. 01_clone.sh will wipe the /spring directory completely and create a clean checkout. So make sure to not lose your changes! As some of the steps will modify shell variables all scripts should be sourced (instead of started regularly), e.g. . /scripts/01.clone.sh.

shell command

In case you want to switch the configuration from within the development shell, you can run 00_setup.sh <arg>.

For example:

. /scripts/00_setup.sh -p linux-64