Skip to content

Latest commit

 

History

History
440 lines (331 loc) · 17.2 KB

CMAKE.md

File metadata and controls

440 lines (331 loc) · 17.2 KB
Raw

Building SWI-Prolog using cmake

As of version 7.7.20, SWI-Prolog ships with cmake CMakeLists.txt configuration files that cover the entire project. Builds have been tested on Ubuntu 16.04, 18.04, 18.10, Fedora 28, MacOSX and cross compilation for Win32 as well as Win64 using Ubuntu 18.04 as host system.

The build has been tested with the "Unix Makefiles" and "Ninja" generators. We use Ninja as it builds faster, avoids warning from being cluttered and better facilitates debugging dependency issues. It can be selected using cmake -G Ninja .., after which the usual make target can be replaced by ninja target. The examples below all use Ninja. Drop -G Ninja to use classical Unix make.

Getting cmake

Building SWI-Prolog requires cmake version 3.5 or later (*). Many Linux systems ship with a cmake package. On MacOS we use the Macport version. If the shipped cmake version is too old you may wish to download cmake from https://cmake.org/download/ On Linux systems, installing e.g., cmake 3.12 (please pick the latest stable version) is as simple as:

wget https://cmake.org/files/v3.12/cmake-3.12.0-Linux-x86_64.sh
sudo sh cmake-3.12.0-Linux-x86_64.sh --prefix=/usr/local --exclude-subdir

(*) Tested with 3.5 (Ubuntu Xenial), 3.10 (Ubuntu Bionic), 3.14 (MacOS).

Native build

Getting the source

The source may be downloaded as a tar ball from http://www.swi-prolog.org or downloaded using git. The git sequence is:

git clone https://github.com/SWI-Prolog/swipl-devel.git
cd swipl-devel
git submodule update --init

If not all modules are needed, one can clone/update particular ones as follows:

git submodule update --init packages/jpl packages/clib packages/sgml

Building from source

The typical sequence to build SWI-Prolog and install in /usr/local is as follows:

cd swipl-devel
mkdir build
cd build
cmake -G Ninja ..
ninja
ctest -j 8
ninja install

Upgrading

In most cases the following should update an installed system to the latest version:

git pull
git submodule update --init
cd build
cmake ..
ninja
ctest -j 8
ninja install

If this fails, one of these measures may be appropriate:

  1. run ninja clean before ninja
  2. remove the entire build directory and re-create it as above. Note that the build process makes no modifications outside the build directory.

Build types

The default build type is RelWithDebInfo. Alternatives may be selected using e.g.,

cmake -DCMAKE_BUILD_TYPE=Debug -G Ninja ..
cmake -DCMAKE_BUILD_TYPE=Release -G Ninja ..

Install location

To install in a particular location, use -DCMAKE_INSTALL_PREFIX. For example, this will build SWI to be installed in /usr/local/swipl-git:

cmake -DCMAKE_INSTALL_PREFIX=/usr/local/swipl-git -G Ninja ..

After sudo ninja install, swipl will be in /usr/local/swipl-git/bin/swipl.

Customizing SWI-Prolog

By default the system configures all features. Several cmake options allow for restricting the system.

Option Description
-DMULTI_THREADED=OFF Drop support for Prolog threads
-DUSE_SIGNALS=OFF Drop signal support
-DUSE_GMP=OFF Drop bignum and rational numbers
-DUSE_TCMALLOC=OFF Do not link against -ltcmalloc
-DSWIPL_SHARED_LIB=OFF Build Prolog kernel as static lib
-DSWIPL_INSTALL_IN_LIB=ON Install libswipl.so in <prefix>/lib
-DSWIPL_INSTALL_IN_SHARE=ON Install docs in <prefix>/share
-DSWIPL_M32=ON Make 32-bit version on 64-bit Linux
-DSWIPL_PACKAGES=OFF Only build the core system
-DSWIPL_PACKAGES_BASIC=OFF Drop all basic packages
-DSWIPL_PACKAGES_ODBC=OFF Drop ODBC and CQL packages
-DSWIPL_PACKAGES_JAVA=OFF Drop JPL Java interface
-DSWIPL_PACKAGES_X=OFF Drop graphics (xpce)
-DBUILD_TESTING=OFF Do not setup for ctest unit tests
-DINSTALL_TESTS=ON Add tests to installed system
-DINSTALL_DOCUMENTATION=OFF Drop generating the HTML docs

Note that packages for which the prerequisites cannot be found are dropped automatically, as are packages for which the sources are not installed.

Embedding SWI-Prolog in Java, C, C++, etc.

If SWI-Prolog is to be embedded in another executable it must be able to find its home directory and the main application must be able to find the SWI-Prolog shared library libswipl.so (extension depends on the platform). The following environment variables are commonly used:

  • SWI_HOME_DIR should point at SWI-Prolog's main directory, e.g. ${CMAKE_INSTALL_PREFIX}/lib/swipl

  • The shared object search path should include the directory where libswipl.{so,dll,...} resides. The variable depends on the platform. Some popular names:

    • LD_LIBRARY_PATH (ELF based systems such as Linux)
    • DYLD_LIBRARY_PATH (MacOS)
    • PATH (Windows)

If you build SWI-Prolog you must remove these variables from the environment when building. Failure to do so may cause the build process to use parts of an incompatible installed system. Running cmake warns if such an environment variable is found, but the environment must be cleaned when running ninja or make.

Profile Guided Optimization

When using Ninja and GCC, the system may be built using Profile Guided Optimization (PGO). This first builds the system instrumented to collect profile information, then runs a benchmark suite and finally recompiles it using the benchmark suite output to help (notably) branch prediction. The performance improvement is about 10 to 20%. Unfortunately the process is a little hard and the entire system needs to be recompiled on any change made to it. To build the PGO version, run the commands below. The ../scripts/pgo-compile.sh performs the three steps above using the core system. The final ninja builds the packages and documentation.

cmake -DCMAKE_BUILD_TYPE=Release -G Ninja ..
../scripts/pgo-compile.sh
ninja

To stop using PGO builds in the current build directory run the commands below. The pgo-compile.sh --off restores the normal build flags, deletes the PGO data and cleans the core system files.

../scripts/pgo-compile.sh --off
ninja

Cross build

Cross building for Windows using (Ubuntu) linux. Use README.mingw to download and build the dependencies. Next, set these environment variables:

  • MINGW64_ROOT must point at the prefix where the dependencies are for 64-bit Windows.
  • MINGW32_ROOT must point at the prefix where the dependencies are for 32-bit Windows.

The cmake toolchain config files (see below) search for Java in $HOME/.wine. Please check these locations.

64 bit Windows from Linux

mkdir win64
cd win64
cmake -DCMAKE_TOOLCHAIN_FILE=../cmake/cross/linux_win64.cmake -G Ninja ..
ninja

32 bit Windows from Linux

mkdir win32
cd win32
cmake -DCMAKE_TOOLCHAIN_FILE=../cmake/cross/linux_win32.cmake -G Ninja ..

WASM (Emscripten)

Note: due to a bug in the current Emscripten directory access functions we need the native friend mechanism to create the library index. The flags below include -DSWIPL_NATIVE_FRIEND=build, assuming you built a native executable in the directory build below the sources. Adjust as necessary.

[Assumes native Prolog in `build`.  See note above]

mkdir build.wasm
cd build.wasm
source ~/emsdk/emsdk_env.sh
cmake -DCMAKE_TOOLCHAIN_FILE=$EMSCRIPTEN/cmake/Modules/Platform/Emscripten.cmake \
      -DCMAKE_BUILD_TYPE=Release \
      -DZLIB_LIBRARY=$HOME/zlib-1.2.11/libz.a \
      -DZLIB_INCLUDE_DIR=$HOME/zlib-1.2.11 \
      -DMULTI_THREADED=OFF \
      -DUSE_SIGNALS=OFF \
      -DUSE_GMP=OFF \
      -DBUILD_SWIPL_LD=OFF \
      -DSWIPL_PACKAGES=OFF \
      -DINSTALL_DOCUMENTATION=OFF \
      -DSWIPL_NATIVE_FRIEND=build \
      -G Ninja ..

Building a 32-bit version on 64-bit Debian based Linux

Building the 32-bit version on a 64 bit platform can be useful for testing and creating 32-bit .qlf files or saved states. A fairly complete system is created using the configuration command below.

cmake -DSWIPL_M32=ON \
      -DSWIPL_PACKAGES_JAVA=OFF -DSWIPL_PACKAGES_QT=OFF \
      -G Ninja ..

Cross-building for targets without an emulator

In the above scenarios we have an emulator (Wine, Node.js) that can run the compiled Prolog system so we can do the Prolog steps of the installation such as building the boot file, building .qlf files, library indexes and the documentation. On some systems we do not have a suitable emulator. Experimental support is provided using the following steps:

  • Build a native Prolog system in a directory, say native. This version must have the same word-size (32 or 64-bits) as the cross-compiled target. One the core Prolog system (no packages) is required and the system only needs to be build, i.e., the install step is allowed but not needed. See above.

  • Specify -DSWIPL_NATIVE_FRIEND=native for the cross-compilation. This will cause the above system to be used for the cross compilation steps.

Development

When building SWI-Prolog using cmake a complete installation is created in the cmake build directory. If possible, the files from the source tree that do not need modification are created as symbolic links to the real sources. This implies that src/swipl can be used as a complete development environment and library and system predicates can be edited using edit/1 and friends.

The script scripts/swi-activate may be used to create symlinks from $HOME/bin to the version in the current working directory. It may be used to activate the system in the build directory or where it is installed. It is called from the build directory as one of:

../scripts/swipl-activate
../scripts/swipl-activate --installed

Developers may wish to set the environment variable SWIPL_INSTALL_PREFIX, which is used as the default for CMAKE_INSTALL_PREFIX. Moreover, if this variable includes @builddir@ this string is replaced with the basename of the current build directory. This aims at the following scenario:

  1. Set e.g. export SWIPL_INSTALL_PREFIX=$HOME/cmake/@builddir@
  2. Use multiple build directories for debug, different targets or different configurations. Typically these are called build.<config>, for example build.single-threaded.
  3. Configure without specifying a CMAKE_INSTALL_PREFIX
  4. Build, test and install. Optionally use swipl-activate to use this version as default.

When developing on the core system one often does not want to re-generate documentation and possible package dependencies. This can be achieved using the target core, which builds swipl, libswipl and boot.prc:

ninja core

Testing

Tests are registered for use with ctest. To run all tests, simply run this in the build directory. Tests can be run concurrently (-j 8 below).

% ctest -j 8

Note that there seem to be few tests. This is misleading. Each ctest test loads a Prolog file that may run hundreds of tests. If a test fails, run the command below to get details. Tests are named package:name, so ctest -V -R clib: runs the tests for the clib package.

% ctest -V -R name

Note that all tests can be executed interactively by loading the test file and calling the entry point as illustrated. The entry point is always the base name of the file (without directory and without extension).

% src/swipl ../src/Tests/core/test_arith.pl
?- test_arith.
% PL-Unit: div ... done
...

Trapping memory issues using AddressSanitizer

AddressSanitizer is an extension to Clang and GCC to instrument executables for finding common memory management issues. It traps similar bugs as Valgrind, but if a suspected bug does not show up using one tool it might be worthwhile to try the other. A nice property of Valgrind is that it can be used directly on the executable without recompilation. The downside is that Valgrind makes the program run about 20 times slower. The slowdown by AddressSanitizer is about a factor two. To compile for using with AddressSanitizer, do e.g.,

% mkdir build.sanitize
% cd build.sanitize
% cmake -DCMAKE_BUILD_TYPE=Sanitize -G Ninja ..
% ninja

See also cmake/BuildType.cmake and PL_halt() in src/pl-fli.c.

You can run the tests normally using ctest. Note that the swipl:GC test requires more stack than the default when using AddressSanitizer. To fix this run (bash) ulimit -s 20000 before running ctest. The test jpl:prolog_in_java will fail because Java is not loaded with AddressSanitizer preloaded. All other tests should pass (about 4 times slower than normal).

Packaging

Windows

The windows installer is created from the cross-compiled version using a Linux native port of the NSIS installer generator. Ensure makensis is installed (apt-get install nsis) and run the commands below in the build directory to create the installer:

cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=../cmake/cross/linux_win64.cmake -G Ninja ..
ninja
cpack

And, for the 32-bit version:

cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=../cmake/cross/linux_win32.cmake -G Ninja ..
ninja
cpack

Debian based Linux systems (.deb or .rpm)

The following commands create swipl-<version>-<nr>.<cpu>.deb/rpm file with SWI-Prolog to be installed in /usr. The process creates a monolithic installer for a particular configuration of SWI-Prolog. This is not what is typically used to create packages for distributions. Distro package maintainers are referred to Modular packages for Linux below. The procedure here is intended to create custom Debian packages for in-house deployment.

cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -G Ninja ..
ninja
cpack -G DEB

for generating an RPM, use cpack -G RPM. The cmake configure run selects a default packager depending on the availability of the package installer apt (assuming DEB) or dnf (assuming RPM).

Modular packages for Linux

Most Linux distributions with to install SWI-Prolog using multiple packages, notably to reduce dependencies. For example, the xpce package os normally provided by a package swi-prolog-x and the core of SWI-Prolog as swi-prolog-nox. This allows installing swi-prolog-nox on headless servers without installing X11.

Modular installation can be based on cmake COMPONENTS. The files for a particular component can be installed using, for example (note this is a one-line command):

DESTDIR=$(pwd)/<component> \
    cmake -DCMAKE_INSTALL_COMPONENT=<component> \
          -P cmake_install.cmake

The defined components are:

Component Description
Core_system Compiler and core libraries
Core_packages Packages with few dependencies
Archive_interface Libarchive binding
Commandline_editors Readline and libedit interfaces
ODBC_interface ODBC binding
BerkeleyDB_interface BDB interface
Perl_regex PCRE library binding
YAML_support Libyaml binding
Java_interface Java interface (JPL)
OpenSSL_interface Binding to OpenSSL/LibreSSL
TIPC_networking Linux TIPC network support
Qt_console Qt windowed interface
Graphics_subsystem The xpce graphics system (needs X11)
Documentation System HTML documentation
Examples Example files

See the debian subdirectory for the complete set of rules we use to generate the Ubuntu PPA releases.

Create a MacOSX Bundle

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_MACOS_BUNDLE=ON -G Ninja ..
ninja
cpack

Issues

  • Provide a FindSWIPL.cmake?
  • Problem compiling SWI when another SWI is installed already and you have environment variables set to facilitate e.g., embedding in Java. The variable names and possibly conflicting values depend on the OS. See issue