Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing cmake build #657

Merged
merged 56 commits into from
Feb 21, 2014
Merged
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
2cb2e82
Update .travis.yml
Feb 13, 2014
cfe940e
Changing into root of project before calling make.
rhyolight Feb 13, 2014
2bd9233
Allowing travis-ci to update submodules.
rhyolight Feb 13, 2014
7432f32
Adding cd into NUPIC before running tests.
rhyolight Feb 13, 2014
3ca4117
Updated README for CMake
rhyolight Feb 13, 2014
5bd2dfa
Removing submodule work within build.sh
rhyolight Feb 13, 2014
3c836d7
Merge branch 'master' of github.com:numenta/nupic into david-patch-2
rhyolight Feb 13, 2014
d80531e
Trying to set default cpp compiler to GSS
rhyolight Feb 14, 2014
bfe666e
Reverting change to default cpp compiler (didn't work).
rhyolight Feb 14, 2014
746fb2a
Trying David's suggestion about default compiler
rhyolight Feb 14, 2014
f44b662
Reverting previous CMake default compiler fix.
rhyolight Feb 14, 2014
b1708bd
Added -Wsign-compare compile option for gcc.
rhyolight Feb 14, 2014
ede1ad0
Removing -std=c++0x compile option for gcc.
rhyolight Feb 17, 2014
f600a8a
Fixed -Wno-sign-compare warning option.
rhyolight Feb 17, 2014
2fec800
Merge branch 'master' into david-patch-2
rhyolight Feb 17, 2014
b1b24db
Merge branch 'david-patch-2' of github.com:rhyolight/nupic into david…
rhyolight Feb 17, 2014
8dbd4df
Update CMakeLists.txt
Feb 18, 2014
b9a9c36
Update CMakeLists.txt
Feb 18, 2014
a5370fe
Update CMakeLists.txt
Feb 18, 2014
93e5802
Update CMakeLists.txt
Feb 18, 2014
cef3232
DEFAULT_COMPILER renamed to USER_CXX_COMPILER
Feb 18, 2014
32b6e14
Update CMakeLists.txt
Feb 18, 2014
1f5e88f
Update CMakeLists.txt
Feb 18, 2014
e5b6fcf
Update CMakeLists.txt
Feb 19, 2014
ce39b7f
Updated an incorrect path in README
rhyolight Feb 19, 2014
f06c5f2
Another update to README
rhyolight Feb 19, 2014
5524f5e
We don't want ./run_tests.sh --all option.
rhyolight Feb 19, 2014
6ccc9c0
Update CMakeLists.txt
Feb 19, 2014
dabb3b1
Update CMakeLists.txt
Feb 19, 2014
ca17282
Update CMakeLists.txt
Feb 19, 2014
1c7f594
Update CMakeLists.txt
Feb 19, 2014
200cd2e
Update CMakeLists.txt
Feb 19, 2014
8bc70be
Update CMakeLists.txt
Feb 19, 2014
b1403e5
Telling Travis not to update submodules
rhyolight Feb 19, 2014
2cf7ded
Merge branch 'david-patch-2' of github.com:rhyolight/nupic into david…
rhyolight Feb 19, 2014
6525f2a
trying previous travis config format
rhyolight Feb 19, 2014
2745a99
Simplified submodule update within cmake.
rhyolight Feb 19, 2014
ee542d0
Update nupic.core submodule.
rhyolight Feb 19, 2014
7d0901d
Removed git submodule update instructions.
rhyolight Feb 19, 2014
8b34f03
Cleanup in README
rhyolight Feb 19, 2014
1f2b311
Replacement of UpdateSubmodules to inline command.
Feb 20, 2014
b6a3cfa
Update CMakeLists.txt
Feb 20, 2014
73942db
Update CMakeLists.txt
Feb 20, 2014
5ecee26
Mac OSX compatibility changes
subutai Feb 20, 2014
b213ecb
Took out -std=c++11 for non OSX
subutai Feb 20, 2014
6bf1e55
Update CMakeLists.txt
Feb 20, 2014
0563fb3
NTAX_DEVELOPER_BUILD included
Feb 20, 2014
eb36dba
Took out -std=c++11 for non OSX platforms
subutai Feb 20, 2014
3ce15e4
Merge branch 'david-patch-2' of github.com:rhyolight/nupic into david…
subutai Feb 20, 2014
c059269
Update CMakeLists.txt
Feb 20, 2014
6aa5581
Update CMakeLists.txt
Feb 20, 2014
f0964cc
Added ref to latest nupic.core changes.
subutai Feb 20, 2014
41ea938
Symlinks were coded (when NTAX_DEVELOPER_BUILD=1)
Feb 21, 2014
da1cc11
Adding back submodule update within build.sh
rhyolight Feb 21, 2014
00eac5a
Merge branch 'master' into david-patch-2
rhyolight Feb 21, 2014
a82f560
Took out check for compiler
subutai Feb 21, 2014
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
@@ -52,14 +52,19 @@ install:
# See: https://github.com/travis-ci/travis-cookbooks/issues/155
- "sudo rm -rf /dev/shm && sudo ln -s /run/shm /dev/shm"
- pip install -q -r $NUPIC/external/common/requirements.txt
- "$NUPIC/build.sh"
- "mkdir -p $TRAVIS_BUILD_DIR/build/scripts"
- "cd $TRAVIS_BUILD_DIR/build/scripts"
- "cmake $NUPIC"
- "make -j3"
- "cd"

script:
# legacy binary tests
- "$NTA/bin/htmtest"
# (not quite everything, really)
- "$NTA/bin/testeverything"
# actual unit tests
- "cd $NUPIC"
- "$NUPIC/run_tests.sh --all"

# run all example files
98 changes: 51 additions & 47 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -21,7 +21,7 @@
# Contributor(s):
# David Ragazzi (@DavidRagazzi): Full conversion and adaptation from Autotools scripts
# --------------------------------------------------------------------------------------------------------------------------


############################################################################################################################
### ###
@@ -65,13 +65,23 @@ macro (set_environment_variable variable value incremental)
endmacro()

# These macros copy all source directories after the configuration is done
macro (copy_directory src dst)
message(STATUS "Copying from '${src}' to '${dst}'")
execute_process(COMMAND ${CMAKE_COMMAND} -E copy_directory ${src} ${dst})
# Note: If NTAX_DEVELOPER_BUILD is set, copy is optimized (directories are linked, and files are not copied if the modification time of the target is more recent).
macro (copy suffix src dst)
if("$ENV{NTAX_DEVELOPER_BUILD}" STREQUAL "1")
message(STATUS "Linking from '${src}' to '${dst}'")
get_filename_component(dst_dir ${dst} PATH)
execute_process(COMMAND ${CMAKE_COMMAND} -E make_directory ${dst_dir})
execute_process(COMMAND ${CMAKE_COMMAND} -E create_symlink ${src} ${dst})
else()
message(STATUS "Copying from '${src}' to '${dst}'")
execute_process(COMMAND ${CMAKE_COMMAND} -E copy${suffix} ${src} ${dst})
endif()
endmacro()
macro (copy_file src dst)
message(STATUS "Copying from '${src}' to '${dst}'")
execute_process(COMMAND ${CMAKE_COMMAND} -E copy ${src} ${dst})
copy("" ${src} ${dst})
endmacro()
macro (copy_directory src dst)
copy("_directory" ${src} ${dst})
endmacro()

# This macro get all files from a directory and its subdirectories.
@@ -165,20 +175,16 @@ endmacro()
############################################################################################################################

#
# Set c and c++ compiler.
# Set C++ compiler.
# According to CMake documentation, this must be done before any language is set (ie before any project() or enable_language() command).
#
set(DEFAULT_COMPILER "" CACHE STRING "Default C/C++ compiler to be used (GCC / CLANG) [default=none]")
if("${DEFAULT_COMPILER}" STREQUAL "CLANG")
set(CMAKE_C_COMPILER "clang")
set(CMAKE_CXX_COMPILER "clang++")
elseif("${DEFAULT_COMPILER}" STREQUAL "GCC")
set(CMAKE_C_COMPILER "gcc")
set(CMAKE_CXX_COMPILER "g++")
set(USER_CXX_COMPILER "" CACHE STRING "Default C++ compiler to be used (GCC / CLANG) [default=system compiler]")
if(NOT "${USER_CXX_COMPILER}" STREQUAL "")
set(CMAKE_CXX_COMPILER "${USER_CXX_COMPILER}")
endif()

cmake_minimum_required(VERSION 2.8)
project(Nupic)
project(Nupic CXX)

#
# Sets default locations.
@@ -214,6 +220,16 @@ set(TEMP_BUILD_DIR "/tmp/ntabuild")
# CMAKE_CURRENT_BINARY_DIR is used by swig module for put wrappers on, so don't erase this line!
set(CMAKE_CURRENT_BINARY_DIR ${TEMP_BUILD_DIR})

#
# Update repository submodules
#
execute_process(COMMAND git submodule update --init
WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}
RESULT_VARIABLE EXIT_CODE)
if(NOT EXIT_CODE EQUAL 0)
message(FATAL_ERROR "Updating submodules within ${PROJECT_SOURCE_DIR} failed.")
endif()

#
# Set OS flags
#
@@ -241,13 +257,13 @@ else()
set(NTA_PLATFORM_ARCH "32")
endif()
if(OSX)
set(CMAKE_OSX_DEPLOYMENT_TARGET "${_CURRENT_OSX_VERSION}")
set(CMAKE_OSX_DEPLOYMENT_TARGET "10.7")
set(NTA_PLATFORM_OS "darwin64")
set(NTA_PLATFORM_CXXFLAGS "-fPIC -DPIC -m64")
set(NTA_PLATFORM_ASFLAGS "")
set(NTA_PLATFORM_DEBUGFLAGS "-gfull")
set(NTA_PLATFORM_LINKFLAGS "-Wl,-u,_munmap")
set(NTA_PLATFORM_LIBS "-liconv -lsqlite3 -framework CoreServices -framework Accelerate")
set(NTA_PLATFORM_LIBS "-stdlib=libstdc++ -std=c++11 -lc++abi -liconv -lsqlite3 -framework CoreServices -framework Accelerate")
set(NTA_PLATFORM_MPI_EXTRA_LIBS "")
set(NTA_PLATFORM_LINKFLAGS_PYMODULE "-bundle -undefined dynamic_lookup -fPIC")
set(NTA_PLATFORM_LINKFLAGS_PYEMBED "")
@@ -257,22 +273,20 @@ elseif(LINUX)
set(NTA_PLATFORM_CXXFLAGS "-fPIC -DPIC -m64")
set(NTA_PLATFORM_ASFLAGS "-msse2")
set(NTA_PLATFORM_DEBUGFLAGS "-g")
# NTA_PLATFORM_LINKFLAGS="-static-libgcc"
set(NTA_PLATFORM_LINKFLAGS "-Wl,--no-as-needed -static-libgcc")
# pthread needed by libapr-1
set(NTA_PLATFORM_LIBS "-lm -lpthread -ldl -lutil -lstdc++")
set(NTA_PLATFORM_LIBS "-stdlib=libstdc++ -lm -lpthread -ldl -lutil")
set(NTA_PLATFORM_MPI_EXTRA_LIBS "-lrt")
set(NTA_PLATFORM_LINKFLAGS_TEST "-Wl,-R\${libdir}/test")
set(NTA_PLATFORM_LINKFLAGS_PYMODULE "-pthread -shared")
set(NTA_PLATFORM_LINKFLAGS_PYEMBED "-Wl,--export-dynamic")
else()
set(NTA_PLATFORM_OS "linux32")
set(NTA_PLATFORM_CXXFLAGS "-fPIC -DPIC -ffloat-store")
set(NTA_PLATFORM_ASFLAGS "")
set(NTA_PLATFORM_DEBUGFLAGS "-g")
set(NTA_PLATFORM_LINKFLAGS "-static-libgcc")
set(NTA_PLATFORM_LIBS "-lm -lstdc++ -lpthread -ldl -lutil -lrt")
set(NTA_PLATFORM_LINKFLAGS "-Wl,--no-as-needed -static-libgcc")
set(NTA_PLATFORM_LIBS "-stdlib=libstdc++ -lm -lpthread -ldl -lutil -lrt")
set(NTA_PLATFORM_MPI_EXTRA_LIBS "-lrt")
set(NTA_PLATFORM_LINKFLAGS_TEST "-Wl,-R\${libdir}/test")
set(NTA_PLATFORM_LINKFLAGS_PYMODULE "-pthread -shared")
set(NTA_PLATFORM_LINKFLAGS_PYEMBED "-Wl,--export-dynamic")
endif()
@@ -284,7 +298,6 @@ elseif(WINDOWS)
set(NTA_PLATFORM_LINKFLAGS "")
set(NTA_PLATFORM_LIBS "")
set(NTA_PLATFORM_MPI_EXTRA_LIBS "")
set(NTA_PLATFORM_LINKFLAGS_TEST "")
set(NTA_PLATFORM_LINKFLAGS_PYMODULE "")
set(NTA_PLATFORM_LINKFLAGS_PYEMBED "")
else()
@@ -299,7 +312,6 @@ message(STATUS " CXX flags: ${NTA_PLATFORM_CXXFLAGS}")
message(STATUS " AS flags: ${NTA_PLATFORM_ASFLAGS}")
message(STATUS " Debug flags: ${NTA_PLATFORM_DEBUGFLAGS}")
message(STATUS " LD flags: ${NTA_PLATFORM_LINKFLAGS}")
message(STATUS " Test library link flags: ${NTA_PLATFORM_LINKFLAGS_TEST}")
message(STATUS " Python extension link flags: ${NTA_PLATFORM_LINKFLAGS_PYMODULE}")
message(STATUS " Python embedding link flags: ${NTA_PLATFORM_LINKFLAGS_PYEMBED}")

@@ -311,7 +323,8 @@ message(STATUS " Python embedding link flags: ${NTA_PLATFORM_LINKFLAGS_PYEMBED}
# Instead we just use a shell command to execute a simple python command, if exit code is 0 ('success') then we have python installed.
#
message(STATUS "Checking software tools...")
execute_process(COMMAND python --version RESULT_VARIABLE EXIT_CODE)
execute_process(COMMAND python --version
RESULT_VARIABLE EXIT_CODE)
if(NOT EXIT_CODE EQUAL 0)
message(FATAL_ERROR "System Python not found. You do not have a system version of Python or it is not set on environment path.")
endif()
@@ -407,9 +420,10 @@ else()
show_environment_variable(NTA_DATA_PATH "$ENV{NTA_DATA_PATH}")
endif()
if(OSX)
set_environment_variable(MACOSX_DEPLOYMENT_TARGET "${_CURRENT_OSX_VERSION}" OFF)
set_environment_variable(MACOSX_DEPLOYMENT_TARGET "${CMAKE_OSX_DEPLOYMENT_TARGET}" OFF)
message(STATUS " (some variables will be updated only after login.)")
endif()
show_environment_variable(NTAX_DEVELOPER_BUILD "$ENV{NTAX_DEVELOPER_BUILD}")

#
# Project details
@@ -440,7 +454,7 @@ set(NTA_INCLUDEFLAGS "-I${PROJECT_SOURCE_DIR} -I${PROJECT_SOURCE_DIR}/external/c
# NTA_INTERNAL tells us that the code is being built under the build system
# and not as a separate program. Used for cppvision example.
#
set(NTA_CXXFLAGS_BASE "${NTA_INCLUDEFLAGS} ${NTA_PLATFORM_CXXFLAGS} -DHAVE_CONFIG_H -DNTA_INTERNAL -DNTA_PLATFORM_${NTA_PLATFORM_OS} -DBOOST_NO_WREGEX -DNUPIC2 -fvisibility=hidden -std=c++0x -Wall -Wreturn-type -Wunused-variable -Wno-deprecated")
set(NTA_CXXFLAGS_BASE "${NTA_INCLUDEFLAGS} ${NTA_PLATFORM_CXXFLAGS} -DHAVE_CONFIG_H -DNTA_INTERNAL -DNTA_PLATFORM_${NTA_PLATFORM_OS} -DBOOST_NO_WREGEX -DNUPIC2 -fvisibility=hidden -Wall -Wreturn-type -Wunused-variable -Wno-deprecated")

#
# All executables and plugins are linked with these flags
@@ -513,7 +527,8 @@ message(STATUS " MPI Support: ${NTA_MPI_SUPPORT}")
#
# Set GNU c/c++ flags in case of CLang is off.
#
if("${DEFAULT_COMPILER}" STREQUAL "GCC")
if(CMAKE_COMPILER_IS_GNUCXX)
set(NTA_CXXFLAGS_BASE "${NTA_CXXFLAGS_BASE} -Wno-sign-compare")
set(NTA_CXXFLAGS_OPTIMIZATION "${NTA_CXXFLAGS_OPTIMIZATION} -falign-loops=16")
endif()

@@ -1044,30 +1059,19 @@ endif()
# Create file with unit tests used by TestEverything
#
set(FILES_RETURNED)
get_files_list(${PROJECT_SOURCE_DIR}/nta "unittests" "Test.cpp")
set(files ${FILES_RETURNED})
set(file_content "")
foreach(file ${files})
get_filename_component(test ${file} NAME_WE)
if(NOT ${test} MATCHES "TesterTest")
set(file_content "${file_content}ADD_TEST(${test});\n")
endif()
endforeach()
file(WRITE "${PROJECT_SOURCE_DIR}/qa/testeverything/everything_addtests.hpp" "${file_content}")

#
# Create file with headers of unit tests used by TestEverything
#
set(FILES_RETURNED)
get_files_list(${PROJECT_SOURCE_DIR}/nta "unittests" "Test.hpp")
set(files ${FILES_RETURNED})
set(file_content "")
set(tests_file_content "")
set(headers_file_content "")
foreach(file ${files})
get_filename_component(test ${file} NAME_WE)
if(NOT ${file} MATCHES "TesterTest")
set(file_content "${file_content}#include \"${file}\"\n")
set(tests_file_content "${tests_file_content}ADD_TEST(${test});\n")
set(headers_file_content "${headers_file_content}#include \"${file}\"\n")
endif()
endforeach()
file(WRITE "${PROJECT_SOURCE_DIR}/qa/testeverything/everything_headers.hpp" "${file_content}")
file(WRITE "${PROJECT_SOURCE_DIR}/qa/testeverything/everything_addtests.hpp" "${tests_file_content}")
file(WRITE "${PROJECT_SOURCE_DIR}/qa/testeverything/everything_headers.hpp" "${headers_file_content}")

# Copy platform independent executable scripts
copy_file(${PROJECT_SOURCE_DIR}/lang/py/bindings/__init__.py ${PYTHON_SITE_PACKAGES_DIR}/nupic/bindings/__init__.py)
76 changes: 44 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
@@ -38,33 +38,13 @@ Currently supported platforms:
Dependencies:
* Python (2.6-2.7) (with development headers)
* GCC (4.6-4.8), or Clang
* Make
* Make or any IDE supported by CMake (Visual Studio, Eclipse, XCode, KDevelop, etc)

The dependencies are included in platform-specific repositories for convenience:

* [nupic-linux64](https://github.com/numenta/nupic-linux64) for 64-bit Linux systems
* [nupic-darwin64](https://github.com/numenta/nupic-darwin64) for 64-bit OS X systems

Add the following to your .bashrc file. Change the paths as needed.

# Installation path
export NTA=$HOME/nta/eng
# Target source/repo path. Defaults to $PWD
export NUPIC=/path/to/repo
# Convenience variable for temporary build files
export BUILDDIR=/tmp/ntabuild
# Number of jobs to run in parallel (optional)
export MK_JOBS=3

# Set up the rest of the necessary env variables. Must be done after
# setting $NTA.
source $NUPIC/env.sh

If you plan on making changes to NuPIC, add the following to your .bashrc file.

# Developer mode: make build use symbolic links from source for Python files instead of copying files
export NTAX_DEVELOPER_BUILD=1

Complete set of python requirements are documented in [requirements.txt](/external/common/requirements.txt),
compatible with [pip](http://www.pip-installer.org/en/latest/cookbook.html#requirements-files):

@@ -76,25 +56,57 @@ _Note_: If using pip 1.5 or later:

_Note_: If you get a "permission denied" error when using pip, you may add the --user flag to install to a location in your home directory, which should resolve any permissions issues. Doing this, you may need to add this location to your PATH and PYTHONPATH. Alternatively, you can run pip with 'sudo'.

Build and install NuPIC:
## Build and test NuPIC:

Important notes:
* $REPOSITORY is the current location of the repository that you downloaded from GitHub.
* After CMake generation, two useful environment variables will be created:
* $NUPIC, which is the same as $REPOSITORY
* $NTA, which references $HOME/nta/eng (the directory with all executables and libraries generated from build process). If this variable is already set, the $REPOSITORY/release will not be created, and $NTA will be used as the release directory.

### Using command line

#### Configure and generate build files:

cd $REPOSITORY/build_system
cmake $REPOSITORY

#### Build:

cd $REPOSITORY/build_system
make -j3

> **Note**: -j3 option specify '3' as the maximum number of parallel jobs/threads that Make will use during the build in order to gain speed. However, you can increase this number depending your CPU.
#### Run the C++ tests:

cd $NTA/bin
htmtest
testeverything

### Using graphical interface

$NUPIC/build.sh
#### Generate the IDE solution:

NuPIC should now be installed in $NTA! If the build failed, check to make sure that $NUPIC is set, and the value is the proper path to the local NuPIC repo.
* Open CMake executable.
* Specify the source folder ($REPOSITORY/source).
* Specify the build system folder ($REPOSITORY/build_system), ie where IDE solution will be created.
* Click 'Generate'.
* Choose the IDE that interest you (remember that IDE choice is limited to your OS, ie Visual Studio is available only on CMake for Windows).

## Try it out!
#### Build:

### Tests
* Open 'Nupic.*proj' solution file generated on $REPOSITORY/build_system.
* Run 'ALL_BUILD' project from your IDE.

Run the C++ tests:
#### Run the C++ tests:

$NTA/bin/htmtest
$NTA/bin/testeverything
* Run 'HtmTest' and 'TestEverything' projects from your IDE (check 'output' panel to see the results).

Run the Python unit tests:
### Run the Python unit tests:

cd $NTA
./bin/run_tests.sh
cd $NUPIC
$NUPIC/run_tests.sh

### Examples