-
Notifications
You must be signed in to change notification settings - Fork 448
OmniSciDB Dependencies
OmniSciDB has the following dependencies:
| Package | Min Version | Required |
|---|---|---|
| CMake | 3.3 | yes |
| LLVM | 7.0 (8.0 recommended) | yes |
| GCC | 7.1 | no, if building with clang |
| Go | 1.6 | yes |
| Boost | 1.65.0 | yes |
| OpenJDK | 1.7 | yes |
| CUDA | 10.0 | yes, if compiling with GPU support |
| gperftools | yes | |
| gdal | 2.3 | yes |
| Arrow | 0.13.0 | yes |
Dependencies for omnisci_web_server and other Go utils are in ThirdParty/go. See ThirdParty/go/src/mapd/vendor/README.md for instructions on how to add new deps.
Click the link for instructions on how to use a script to install OmniSciDB dependencies for your chosen platform.
- Installing CentOS 7 Dependencies
- Installing macOS Dependencies
- Installing Ubuntu Dependencies
- Installing Dependencies from Arch User Repository
OmniSciDB requires a number of dependencies which are not provided in the common CentOS/RHEL package repositories. A prebuilt package containing all these dependencies is provided for CentOS 7 (x86_64).
Use the scripts/mapd-deps-prebuilt.sh build script to install prebuilt dependencies.
These dependencies will be installed to a directory under /usr/local/mapd-deps. The mapd-deps-prebuilt.sh script also installs Environment Modules in order to simplify managing the required environment variables. Log out and log back in after running the mapd-deps-prebuilt.sh script in order to active Environment Modules command, module.
The mapd-deps environment module is disabled by default. To activate for your current session, run:
module load mapd-deps
To disable the mapd-deps module:
module unload mapd-deps
WARNING: The mapd-deps package contains newer versions of packages such as GCC and ncurses which might not be compatible with the rest of your environment. Make sure to disable the mapd-deps module before compiling other packages.
Instructions for installing CUDA are below.
It is preferred, but not necessary, to install CUDA and the NVIDIA drivers using the .rpm using the instructions provided by NVIDIA. The rpm (network) method (preferred) will ensure you always have the latest stable drivers, while the rpm (local) method allows you to install does not require Internet access.
The .rpm method requires DKMS to be installed, which is available from the Extra Packages for Enterprise Linux repository:
sudo yum install epel-release
Be sure to reboot after installing in order to activate the NVIDIA drivers.
The mapd-deps-prebuilt.sh script includes two files with the appropriate environment variables: mapd-deps-<date>.sh (for sourcing from your shell config) and mapd-deps-<date>.modulefile (for use with Environment Modules, yum package environment-modules). These files are placed in mapd-deps install directory, usually /usr/local/mapd-deps/<date>. Either of these may be used to configure your environment: the .sh may be sourced in your shell config; the .modulefile needs to be moved to the modulespath.
The scripts/mapd-deps-centos.sh script is used to build the dependencies. Modify this script and run if you would like to change dependency versions or to build on alternative CPU architectures.
cd scripts
module unload mapd-deps
./mapd-deps-centos.sh --compress
scripts/mapd-deps-osx.sh is provided that will automatically install and/or update Homebrew and use that to install all dependencies. Please make sure macOS is completely up to date and Xcode is installed before running. Xcode can be installed from the App Store.
mapd-deps-osx.sh will automatically install CUDA via Homebrew and add the correct environment variables to ~/.bash_profile.
mapd-deps-osx.sh will automatically install Java and Maven via Homebrew and add the correct environment variables to ~/.bash_profile.
Most build dependencies required by OmniSciDB are available via APT. Certain dependencies such as Thrift, Blosc, and Folly must be built as they either do not exist in the default repositories or have outdated versions. A prebuilt package containing all these dependencies is provided for Ubuntu 18.04 (x86_64). The dependencies will be installed to /usr/local/mapd-deps/ by default; see the Environment Variables section below for how to add these dependencies to your environment.
OmniSciDB requires a newer version of Boost than the version which is provided by Ubuntu 16.04. The scripts/mapd-deps-ubuntu1604.sh build script will compile and install a newer version of Boost into the /usr/local/mapd-deps/ directory.
Use the scripts/mapd-deps-prebuilt.sh build script to install prebuilt dependencies.
These dependencies will be installed to a directory under /usr/local/mapd-deps. The mapd-deps-prebuilt.sh script above will generate a script named mapd-deps.sh containing the environment variables which need to be set. Simply source this file in your current session (or symlink it to /etc/profile.d/mapd-deps.sh) in order to activate it:
source /usr/local/mapd-deps/mapd-deps.sh
Some installs of Ubuntu 18.04 may fail while building with a message similar to:
java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
This is a known issue in 18.04 which will be resolved in Ubuntu 18.04.1. To resolve on 18.04:
sudo rm /etc/ssl/certs/java/cacerts
sudo update-ca-certificates -f
The CUDA and mapd-deps lib directories need to be added to LD_LIBRARY_PATH; the CUDA and mapd-deps bin directories need to be added to PATH. The mapd-deps-ubuntu.sh and mapd-deps-prebuilt.sh scripts will generate a script named mapd-deps.sh containing the environment variables which need to be set. Simply source this file in your current session (or symlink it to /etc/profile.d/mapd-deps.sh) in order to activate it:
source /usr/local/mapd-deps/mapd-deps.sh
Recent versions of Ubuntu provide the NVIDIA CUDA Toolkit and drivers in the standard repositories. To install:
sudo apt install -y \
nvidia-cuda-toolkit
Be sure to reboot after installing in order to activate the NVIDIA drivers.
The scripts/mapd-deps-ubuntu.sh and scripts/mapd-deps-ubuntu1604.sh scripts are used to build the dependencies for Ubuntu 18.04 and 16.04, respectively. The scripts will install all required dependencies (except CUDA) and build the dependencies which require it. Modify this script and run if you would like to change dependency versions or to build on alternative CPU architectures.
cd scripts
./mapd-deps-ubuntu.sh --compress
scripts/mapd-deps-arch.sh is provided that will use yay to install packages from the Arch User Repository and a custom PKGBUILD script for Apache Arrow. If you don't have yay yet, install it first: https://github.com/Jguer/yay#installation
Note: Apache Arrow, while available in the AUR, requires a few custom build flags in order to be used with Core. A custom PKGBUILD for it is included.
Note: packages aws-sdk-cpp and folly, while available in the AUR, are not supported while for building Core on Arch. If these packages are installed, support for them should be disabled when building Core. To do so, use the following options when running CMake:
cmake -DENABLE_FOLLY=off -DENABLE_AWS_S3=off ..
CUDA and the NVIDIA drivers may be installed using the following.
yay -S \
linux-headers \
cuda \
nvidia
Be sure to reboot after installing in order to activate the NVIDIA drivers.
The cuda package should set up the environment variables required to use CUDA. If you receive errors saying nvcc is not found, then CUDA bin directories need to be added to PATH: the easiest way to do so is by creating a new file named /etc/profile.d/mapd-deps.sh containing the following:
PATH=/opt/cuda/bin:$PATH
export PATH
| OmniSciDB | Documentation | Setup |Community & Resources| Copyright & 2018 OmniSci, Inc.