ActualBuild

An up-to-date version of this file is checked in as Timba/README_build.txt.

MeqTrees and the CMAKE Build System
===================================

CMake is a platform independent build system, that allows 
out of source code builds.

Full cmake documentation can be found at:
www.cmake.org

Convenience Configuration Script
================================
A script has been provided in the Tools/Build directory for convenience for the most common
use cases. The function of the script is to create build product locations 
and configure cmake appropriately.

e.g. To create a debug build 
> Tools/Build/bootstrap_cmake debug
> cd build/debug
> make

e.g. To create an optimised build
> Tools/Build/bootstrap_cmake release
> cd build/release
> make

Run the script without arguments to get a synopsis of usage. You may supply various cmake arguments (see below) and they will be passed through.

Setting up paths to run the build directly
==========================================
Developers usually prefer to run MeqTrees directly from the source/build tree. To do this, proceed as follows:

1. Your build directory needs to be called Timba/build/BUILDNAME. The convenience scripts above use the standard BUILDNAMEs 'debug' and 'release'.

2. There has to be a tree of symlinks called Timba/install/symlinked-BUILDNAME. These symlinks point to libraries, etc. in your build tree. If your BUILDNAME is one of the standard, i.e. 'debug' or 'release', the appropriate symlink tree is already provided.

3. If using a nonstandard BUILDNAME, you can make your own symlink tree as follows:

  $ cd Timba/install
  $ ./make_symlink_tree.sh BUILDNAME

4. Purr has to be available on your system. Either check out Purr into your home directory, or make sure that the outer Purr directory (the one containing Purr and Kittens) is in your PYTHONPATH. 

5. Finally, add this to your .bashrc:

  $ source ~/Timba/install/timba-init.sh
  $ timba-setup release  # or 'debug', or whatever your BUILDNAME is

  This will setup all paths to point to the named build. Note that you can reinvoke the timba-setup command manually in your shell, should you wish to switch to a different build.

You should be able to start meqtrees by running 'meqbrowser.py'.
   


Simple Install
==============
Note that the CMAKE_INSTALL_PREFIX variable will define the relative location 
where the install target will put files. On unix platforms this defaults to "/usr/local".

Step 1:
      create a directory for the build to take place
      > mkdir meqtrees_build; cd meqtrees_build
Step 2:
      from this build directory run cmake to create the platform specific make files
      > cmake PATH_TO_SRC
Step 3:
      run the platform specific make command. 
      e.g. unix based systems
      > make install

      e.g. windows based systems (MSVC)
      > nmake install

External Dependencies
=====================
CMake attempts to find all the external dependencies in the project automatically.
If you have these pacakges installed in an unusual place you may have to specify the locations
through cmake variables. The variable is dependent on the package - for the names, look in the 
relevant Find<packageName>.cmake files found in the cmake directory of the src code and the
system cmake module directory (usually /usr/share/cmake/Modules)

The list of external dependency packages are specified by the dependencies variable set
in the top level CMakeList.txt file.

Alternative Builds
==================
Various alternative builds are available. Activate them by setting the 
appropriate Cmake variable:

  Variable                   Description
  ---------------------------------------------------
  USE_MPI                  | Use the mpi parrallelisation library
  USE_DBM                  | Use a database management system



Setting CMake Variables
=======================
Various options and configurations are available by setting CMake variables
CMake variables can be set by calling cmake with the -D option
e.g.
         cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_CXX_COMPILER=g++-4.2 <SRC_DIRECTORY>

Build options
--------------

  Variable                              | Function
  ------------------------------------------------------------
    CMAKE_BUILD_TYPE=DEBUG|RELEASE      |  build debug or release versions 
                                        |
    CMAKE_CXX_FLAGS=                    |  Extra flags to use when compiling C++ source files.
                                        |
    LIBRARY_INSTALL_DIR=                |  specify location for meqtrees libraries
                                        |  ( default: CMAKE_INSTALL_PREFIX/lib )
                                        |
    MEQTREES_INSTALL_DIR=               |  specify location for meqtrees libraries and other resources
                                        |  ( default: LIBRARY_INSTALL_DIR/meqtrees )
                                        |
    ICON_INSTALL_DIR=                   |  directory to install icons (default: MEQTREES_INSTALL_DIR/icons )
                                        |
    PYTHON_INSTALL_DIR=                 |  installation dir for meqtrees specific python scripts
                                        |  (default: MEQTREES_INSTALL_DIR/Timba )

cmake specifc variables can be found here: http://www.cmake.org/Wiki/CMake_Useful_Variables


-------------------------------------- For Developers -----------------------------------------------------------------
MEQTREES Specific MACROS
========================
A number of convenience Macros are provided to ease the creation of MeqTrees packages.
Here is a quick overview of the main functionality, refer to the cmake/meqmacros.cmake
for full information.

The MEQPACKAGE
--------------
Specify the contents are a MeqPackage and the primary dependencies on other MeqTrees Packages
This will ensure all the INCLUDE_PATHS are correct and the MEQPACKAGE_LIBRARIES variable
will contain all the required variables. It will also genertate the correct lofar_config.h file.

e.g. CMakeLists.txt for a simple MEQPACKAGE

    # Exported Paths for External Dependencies
    INCLUDE_DIRECTORIES(${AnExternalPackage_INCLUDE_DIR})
    INCLUDE_DIRECTORIES(${AnotherExternalPackage_INCLUDE_DIR})                      # These INCLUDE paths will be exported to dependent Meq Packages

    MEQPACKAGE(MyPackageName meqPackageDependenyA meqPackageDependencyB)            # declares the MeqPacake "MyPackageName" and its primary dependencies

    # Non-Exported Paths should be specified after the MEQPACKAGE macro
    INCLUDE_DIRECTORIES(${ADependencyNotExported_INCLUDE_DIR})                      # This INCLUDE directory will not be exported - local to this package only

    # Export Header files myHeader.h and myHeader2.h to the MyPackage directory
    INCLUDE_SETUP(MyPackage myHeader.h myHeader2.h)

    # mark libraries for export to other MeqPackages
    # with the MEQPACKAGE_ADD_LIBRARIES macro
    MEQPACKAGE_ADD_LIBRARIES(myPackageLib ${AnExternalPackage_LIB_DIR} ${AnotherExternalPackage_LIB_DIR})

Cmake may require some additional parameters if you want to incorporate MPI distributed processing into the system. Here is an example call to cmake to build against openmpi, made from the directory Timba/build/release

cmake -DUSE_MPI=TRUE -DMPI_INCLUDE_PATH=/usr/local/openmpi/include -DMPI_LIBRARY=/usr/local/openmpi/lib/libmpi.so -DMPI_EXTRA_LIBRARY:STRING="-pthread -L/usr/local/openmpi/lib -lmpi_cxx -lopen-rte -lopen-pal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl" ~/Timba

How do you know which openmpi libraries are needed for the link step and are specified in the MPI_EXTRA_LIBRARY field above? Simple. If you installed openmpi, run the mpi compiler, mpicc as

mpicc -showme:link

On my computer this gives the output

-pthread -L/usr/local/openmpi/lib -lmpi -lopen-rte -lopen-pal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl

Of course you should be aware that the locations of the include files and the libraries may vary from one linux distribution to another.

Software Packages used in October 17, 2012 Build on Scientific Linux

I recently built the whole MeqTrees package and related software from scratch on some relatively ancient Scientific Linux machine that still has Python 2.4 installed.

These are all the third party libraries that were needed to get MeqTrees up and running. Note that getting pyrap to install can be a pain.

*gz
-rw-r--r-- 1 twillis games   2670893 Oct 18 14:25 PyQwt-5.2.0.tar.gz
-rw-r--r-- 1 twillis games    702436 Oct 17 22:18 astLib-0.6.1.tar.gz
-rw-r--r-- 1 twillis games    498749 Oct 17 22:18 Imaging-1.1.7.tar.gz
-rw-r--r-- 1 twillis games   6776998 Oct 17 21:57 scipy-0.11.0.tar.gz
-rw-r--r-- 1 twillis games    591836 Oct 17 20:07 scons-2.2.0.tar.gz
-rw-r--r-- 1 twillis games 233635127 Oct 17 14:40 qt-everywhere-opensource-src-4.8.3.tar.gz
-rw-r--r-- 1 twillis games   2662469 Oct 17 12:06 QScintilla-gpl-2.6.2.tar.gz
-rw-r--r-- 1 twillis games   9824076 Oct 17 12:05 PyQt-x11-gpl-4.9.5.tar.gz
-rw-r--r-- 1 twillis games    742647 Oct 17 12:03 sip-4.14.tar.gz
-rw-r--r-- 1 twillis games 534323341 Oct 17 10:55 measures.tar.gz
-rw-r--r-- 1 twillis games 410309081 Oct 17 10:24 casapy-33.0.16856-002-64b.tar.gz
-rw-r--r-- 1 twillis games    766083 Oct 16 21:13 pyfits-3.1.tar.gz
-rw-r--r-- 1 twillis games   2585329 Oct 16 21:07 numpy-1.6.2.tar.gz
-rw-r--r-- 1 twillis games   3908279 Oct 16 20:44 fftw-3.3.2.tar.gz
-rw-r--r-- 1 twillis games   1015429 Oct 16 20:38 qdbm-1.8.78.tar.gz
-rw-r--r-- 1 twillis games   3124301 Oct 16 20:17 blitz-0.9.tar.gz
-rw-r--r-- 1 twillis games    293176 Oct 16 15:43 lapack-3.3.1-autoconf.tar.gz
-rw-r--r-- 1 twillis games   4945204 Oct 16 15:43 lapack-3.3.1.tgz
-rw-r--r-- 1 twillis games   4664559 Oct 16 13:47 cfitsio3310.tar.gz
-rw-r--r-- 1 twillis games   5658837 Oct 16 13:01 cmake-2.8.9.tar.gz

*bz2
-rw-r--r-- 1 twillis games 50811407 Oct 18 12:02 boost_1_51_0.tar.bz2
-rw-r--r-- 1 twillis games  1188451 Oct 17 22:48 karma-1.7.25-i386_Linux_libc6.tar.bz2
-rw-r--r-- 1 twillis games  1735734 Oct 17 22:48 karma-1.7.25-common.tar.bz2
-rw-r--r-- 1 twillis games  1230906 Oct 17 22:47 karma-1.7.25-amd64_Linux_libc6.3.tar.bz2
-rw-r--r-- 1 twillis games   180394 Oct 17 20:02 pyrap-1.1.0.tar.bz2
-rw-r--r-- 1 twillis games 11793433 Oct 16 20:52 Python-2.7.3.tar.bz2
-rw-r--r-- 1 twillis games  1592952 Oct 16 13:51 wcslib.tar.bz2
-rw-r--r-- 1 twillis games  6327906 Oct 16 12:56 casacore-1.5.0.tar.bz2

****

# in pyrap top directory, run the commands
# a - get rid of any previous run stored stuff
rm ./libpyrap/tags/pyrap-0.3.2/options.cache
# b - then build
./batchbuild.py --casacore-root=/u/twillis/local --cfitsio-root=/u/twillis/local --boost-root=/u/twillis/local --prefix=/u/twillis/local

Note that measures.tar.gz just contains all the CASA files - ephemeris etc needed to supply things like time corrections etc. Since I had no access to sys administration etc, I had to install everything under local in my home directory. Amazingly things seem to be working! The MeqTrees packages (Timba, Cattery etc) were obtained via subversion checkout as described elsewhere on this wiki.

You may end up having to build the casacore package twice. By default the casacore build system generates shared libraries, which are fine for python-based applications such as pyrap. However, my personal experence lately to build MeqTrees itself, is that the linking step to build the meqserver may fail unless static libraries are available. So you may have to build casacore twice, once with shared libraries and once with static libraries.See line 37 or thereabouts of casacore/CMakeLists.txt, where you can change ENABLE_SHARED from YES to NO, or vice-versa.