Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Taurus: SCS5 Update #2667

Merged

Conversation

ax3l
Copy link
Member

@ax3l ax3l commented Aug 6, 2018

Prepare an updated environment for the Taurus SCS5 software stack.
In order to use the new software stack, make sure to log into the login nodes

  • tauruslogin4 or
  • tauruslogin6

If you are an active Taurus user, please feel free to contribute further to this PR.

  • k80 profile
  • k80 tpl
  • k20x profile
  • k20x tpl
  • knl profile
  • knl tpl

Refs

@ax3l ax3l added the documentation regarding documentation or wiki discussions label Aug 6, 2018
@ax3l ax3l changed the title Taurus: SCS5 Taurus: SCS5 Update Aug 6, 2018

# Environment #################################################################
#
#export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$BOOST_LIB

export PNGWRITER_ROOT=$HOME/lib/pngwriter
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New spellings:
PNGwriter_DIR
Splash_DIR

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update export PNGWRITER_ROOT=$HOME/lib/pngwriter to export PNGwriter_DIR=$HOME/lib/pngwriter.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correspondingly update export SPLASH_ROOT=$HOME/lib/splash to export Splash_DIR=$HOME/lib/splash.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make automatically consistent (only one path needs to be adjusted then):

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/lib/pngwriter/lib/ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/lib/splash/lib/

to

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PNGwriter_DIR/lib/ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$Splash_DIR/lib/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general: This proposed solution is messy, but it seems to work.

  1. The proposed module environment is unfortunately bundled (Boost,HDF5,gnuplot) with the foss/2018a module. This introduces a lot of unnecessary modules (51 modules total). The current module list-command results in:
Derzeit geladene Module:
  1) modenv/scs5                                 (S)  14) hwloc/1.11.8-GCCcore-6.4.0    27) gettext/0.19.8.1-GCCcore-6.4.0-libxml2-2.9.7  40) GMP/6.1.2-GCCcore-6.4.0
  2) GCCcore/6.4.0                                    15) OpenMPI/2.1.2-GCC-6.4.0-2.28  28) PCRE/8.41-GCCcore-6.4.0                       41) nettle/3.4-GCCcore-6.4.0
  3) OpenBLAS/0.2.20-GCC-6.4.0-2.28                   16) cURL/7.58.0-GCCcore-6.4.0     29) util-linux/2.31.1-GCCcore-6.4.0               42) libdrm/2.4.91-GCCcore-6.4.0
  4) gompi/2018a                                      17) expat/2.2.5-GCCcore-6.4.0     30) GLib/2.54.3-GCCcore-6.4.0                     43) LLVM/5.0.1-GCCcore-6.4.0
  5) FFTW/3.3.7-gompi-2018a                           18) Perl/5.26.1-GCCcore-6.4.0     31) fontconfig/2.12.6-GCCcore-6.4.0               44) Mesa/17.3.6-foss-2018a
  6) ScaLAPACK/2.0.2-gompi-2018a-OpenBLAS-0.2.20      19) git/2.18.0-GCCcore-6.4.0      32) X11/20180131-GCCcore-6.4.0                    45) libGLU/9.0.0-foss-2018a
  7) foss/2018a                                       20) bzip2/1.0.6-GCCcore-6.4.0     33) cairo/1.14.12-GCCcore-6.4.0                   46) Qt5/5.10.1-foss-2018a
  8) binutils/2.28-GCCcore-6.4.0                      21) libpng/1.6.34-GCCcore-6.4.0   34) NASM/2.13.03-GCCcore-6.4.0                    47) gnuplot/5.2.4-foss-2018a
  9) GCC/6.4.0-2.28                                   22) freetype/2.9-GCCcore-6.4.0    35) libjpeg-turbo/1.5.3-GCCcore-6.4.0             48) Boost/1.66.0-foss-2018a
 10) ncurses/6.0-GCCcore-6.4.0                        23) pixman/0.34.0-GCCcore-6.4.0   36) libgd/2.2.5-GCCcore-6.4.0                     49) Szip/2.1.1-GCCcore-6.4.0
 11) CMake/3.10.2-GCCcore-6.4.0                       24) libffi/3.2.1-GCCcore-6.4.0    37) HarfBuzz/1.7.5-foss-2018a                     50) HDF5/1.10.1-foss-2018a
 12) CUDA/9.2.88                                      25) XZ/5.2.3-GCCcore-6.4.0        38) Pango/1.41.1-foss-2018a                       51) zlib/1.2.11-GCCcore-6.4.0
 13) numactl/2.0.11-GCCcore-6.4.0                     26) libxml2/2.9.7-GCCcore-6.4.0   39) libcerf/1.5-GCCcore-6.4.0

  Wo:
   S:  Das Modul ist angeheftet. Verwenden Sie "--force", um das Modul zu entladen.

To me, this appears to be excessive and will probably invite compatibility issues in the future. Maybe we should just build boost, hdf5 and gnuplot on our own.

  1. There are many Boost-related warnings during cmake configuration:
CMake Warning at /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
   New Boost version may have incorrect or missing dependencies and imported
   targets
 Call Stack (most recent call first):
   /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
   /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
   /home/s0134766/src/picongpu/include/pmacc/PMaccConfig.cmake:296 (find_package)
   CMakeLists.txt:83 (find_package)

...
many more
  1. There is a long stream of warnings during compiling:
/home/s0134766/src/picongpu/thirdParty/alpaka/include/alpaka/vec/Vec.hpp(925): warning: calling a __host__ function("std::__shared_count<( ::__gnu_cxx::_Lock_policy)2> ::__shared_count") from a __host__ __device__ function("alpaka::dev::DevCpu::DevCpu") is not allowed

/home/s0134766/src/picongpu/thirdParty/alpaka/include/alpaka/vec/Vec.hpp(925): warning: calling a __host__ function("std::__shared_count<( ::__gnu_cxx::_Lock_policy)2> ::__shared_count") from a __host__ __device__ function("alpaka::dev::DevCpu::DevCpu [subobject]") is not allowed

/sw/installed/Boost/1.66.0-foss-2018a/include/boost/mpl/map/aux_/item.hpp(54): warning: "boost::mpl::aux::type_wrapper<T> operator/(const boost::mpl::m_item<Key, T, Base> &, boost::mpl::aux::type_wrapper<Key> *)" declares a non-template function -- add <> to refer to a template instance

/sw/installed/Boost/1.66.0-foss-2018a/include/boost/mpl/map/aux_/item.hpp(55): warning: "boost::mpl::aux::type_wrapper<boost::mpl::pair<Key, T>> operator|(const boost::mpl::m_item<Key, T, Base> &, boost::mpl::next<Base::order>::type *)" declares a non-template function -- add <> to refer to a template instance

/sw/installed/Boost/1.66.0-foss-2018a/include/boost/mpl/map/aux_/item.hpp(56): warning: "char (&operator||(const boost::mpl::m_item<Key, T, Base> &, boost::mpl::aux::type_wrapper<Key> *))[boost::mpl::next<Base::order>::type::value]" declares a non-template function -- add <> to refer to a template instance

/sw/installed/Boost/1.66.0-foss-2018a/include/boost/mpl/map/aux_/item.hpp(71): warning: "boost::mpl::aux::type_wrapper<mpl_::void_> operator/(const boost::mpl::m_mask<Key, Base> &, boost::mpl::aux::type_wrapper<Key> *)" declares a non-template function -- add <> to refer to a template instance

/sw/installed/Boost/1.66.0-foss-2018a/include/boost/mpl/map/aux_/item.hpp(72): warning: "boost::mpl::aux::type_wrapper<mpl_::void_> operator|(const boost::mpl::m_mask<Key, Base> &, boost::mpl::x_order_impl<Base, Key>::type *)" declares a non-template function -- add <> to refer to a template instance

/home/s0134766/src/picongpu/include/pmacc/../pmacc/particles/memory/dataTypes/Particle.hpp(167): warning: __device__ annotation is ignored on a function("operator=") that is explicitly defaulted on its first declaration

/home/s0134766/src/picongpu/include/pmacc/../pmacc/particles/memory/dataTypes/Particle.hpp(167): warning: __host__ annotation is ignored on a function("operator=") that is explicitly defaulted on its first declaration
...
Many more

#
module purge
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-insert module purge.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should not purge the environments they set up.


# Environment #################################################################
#
#export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$BOOST_LIB

export PNGWRITER_ROOT=$HOME/lib/pngwriter
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update export PNGWRITER_ROOT=$HOME/lib/pngwriter to export PNGwriter_DIR=$HOME/lib/pngwriter.


# Environment #################################################################
#
#export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$BOOST_LIB

export PNGWRITER_ROOT=$HOME/lib/pngwriter
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correspondingly update export SPLASH_ROOT=$HOME/lib/splash to export Splash_DIR=$HOME/lib/splash.


# Environment #################################################################
#
#export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$BOOST_LIB

export PNGWRITER_ROOT=$HOME/lib/pngwriter
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make automatically consistent (only one path needs to be adjusted then):

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/lib/pngwriter/lib/ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/lib/splash/lib/

to

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PNGwriter_DIR/lib/ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$Splash_DIR/lib/


# Environment #################################################################
#
#export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$BOOST_LIB

export PNGWRITER_ROOT=$HOME/lib/pngwriter
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general: This proposed solution is messy, but it seems to work.

  1. The proposed module environment is unfortunately bundled (Boost,HDF5,gnuplot) with the foss/2018a module. This introduces a lot of unnecessary modules (51 modules total). The current module list-command results in:
Derzeit geladene Module:
  1) modenv/scs5                                 (S)  14) hwloc/1.11.8-GCCcore-6.4.0    27) gettext/0.19.8.1-GCCcore-6.4.0-libxml2-2.9.7  40) GMP/6.1.2-GCCcore-6.4.0
  2) GCCcore/6.4.0                                    15) OpenMPI/2.1.2-GCC-6.4.0-2.28  28) PCRE/8.41-GCCcore-6.4.0                       41) nettle/3.4-GCCcore-6.4.0
  3) OpenBLAS/0.2.20-GCC-6.4.0-2.28                   16) cURL/7.58.0-GCCcore-6.4.0     29) util-linux/2.31.1-GCCcore-6.4.0               42) libdrm/2.4.91-GCCcore-6.4.0
  4) gompi/2018a                                      17) expat/2.2.5-GCCcore-6.4.0     30) GLib/2.54.3-GCCcore-6.4.0                     43) LLVM/5.0.1-GCCcore-6.4.0
  5) FFTW/3.3.7-gompi-2018a                           18) Perl/5.26.1-GCCcore-6.4.0     31) fontconfig/2.12.6-GCCcore-6.4.0               44) Mesa/17.3.6-foss-2018a
  6) ScaLAPACK/2.0.2-gompi-2018a-OpenBLAS-0.2.20      19) git/2.18.0-GCCcore-6.4.0      32) X11/20180131-GCCcore-6.4.0                    45) libGLU/9.0.0-foss-2018a
  7) foss/2018a                                       20) bzip2/1.0.6-GCCcore-6.4.0     33) cairo/1.14.12-GCCcore-6.4.0                   46) Qt5/5.10.1-foss-2018a
  8) binutils/2.28-GCCcore-6.4.0                      21) libpng/1.6.34-GCCcore-6.4.0   34) NASM/2.13.03-GCCcore-6.4.0                    47) gnuplot/5.2.4-foss-2018a
  9) GCC/6.4.0-2.28                                   22) freetype/2.9-GCCcore-6.4.0    35) libjpeg-turbo/1.5.3-GCCcore-6.4.0             48) Boost/1.66.0-foss-2018a
 10) ncurses/6.0-GCCcore-6.4.0                        23) pixman/0.34.0-GCCcore-6.4.0   36) libgd/2.2.5-GCCcore-6.4.0                     49) Szip/2.1.1-GCCcore-6.4.0
 11) CMake/3.10.2-GCCcore-6.4.0                       24) libffi/3.2.1-GCCcore-6.4.0    37) HarfBuzz/1.7.5-foss-2018a                     50) HDF5/1.10.1-foss-2018a
 12) CUDA/9.2.88                                      25) XZ/5.2.3-GCCcore-6.4.0        38) Pango/1.41.1-foss-2018a                       51) zlib/1.2.11-GCCcore-6.4.0
 13) numactl/2.0.11-GCCcore-6.4.0                     26) libxml2/2.9.7-GCCcore-6.4.0   39) libcerf/1.5-GCCcore-6.4.0

  Wo:
   S:  Das Modul ist angeheftet. Verwenden Sie "--force", um das Modul zu entladen.

To me, this appears to be excessive and will probably invite compatibility issues in the future. Maybe we should just build boost, hdf5 and gnuplot on our own.

  1. There are many Boost-related warnings during cmake configuration:
CMake Warning at /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
   New Boost version may have incorrect or missing dependencies and imported
   targets
 Call Stack (most recent call first):
   /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
   /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
   /home/s0134766/src/picongpu/include/pmacc/PMaccConfig.cmake:296 (find_package)
   CMakeLists.txt:83 (find_package)

...
many more
  1. There is a long stream of warnings during compiling:
/home/s0134766/src/picongpu/thirdParty/alpaka/include/alpaka/vec/Vec.hpp(925): warning: calling a __host__ function("std::__shared_count<( ::__gnu_cxx::_Lock_policy)2> ::__shared_count") from a __host__ __device__ function("alpaka::dev::DevCpu::DevCpu") is not allowed

/home/s0134766/src/picongpu/thirdParty/alpaka/include/alpaka/vec/Vec.hpp(925): warning: calling a __host__ function("std::__shared_count<( ::__gnu_cxx::_Lock_policy)2> ::__shared_count") from a __host__ __device__ function("alpaka::dev::DevCpu::DevCpu [subobject]") is not allowed

/sw/installed/Boost/1.66.0-foss-2018a/include/boost/mpl/map/aux_/item.hpp(54): warning: "boost::mpl::aux::type_wrapper<T> operator/(const boost::mpl::m_item<Key, T, Base> &, boost::mpl::aux::type_wrapper<Key> *)" declares a non-template function -- add <> to refer to a template instance

/sw/installed/Boost/1.66.0-foss-2018a/include/boost/mpl/map/aux_/item.hpp(55): warning: "boost::mpl::aux::type_wrapper<boost::mpl::pair<Key, T>> operator|(const boost::mpl::m_item<Key, T, Base> &, boost::mpl::next<Base::order>::type *)" declares a non-template function -- add <> to refer to a template instance

/sw/installed/Boost/1.66.0-foss-2018a/include/boost/mpl/map/aux_/item.hpp(56): warning: "char (&operator||(const boost::mpl::m_item<Key, T, Base> &, boost::mpl::aux::type_wrapper<Key> *))[boost::mpl::next<Base::order>::type::value]" declares a non-template function -- add <> to refer to a template instance

/sw/installed/Boost/1.66.0-foss-2018a/include/boost/mpl/map/aux_/item.hpp(71): warning: "boost::mpl::aux::type_wrapper<mpl_::void_> operator/(const boost::mpl::m_mask<Key, Base> &, boost::mpl::aux::type_wrapper<Key> *)" declares a non-template function -- add <> to refer to a template instance

/sw/installed/Boost/1.66.0-foss-2018a/include/boost/mpl/map/aux_/item.hpp(72): warning: "boost::mpl::aux::type_wrapper<mpl_::void_> operator|(const boost::mpl::m_mask<Key, Base> &, boost::mpl::x_order_impl<Base, Key>::type *)" declares a non-template function -- add <> to refer to a template instance

/home/s0134766/src/picongpu/include/pmacc/../pmacc/particles/memory/dataTypes/Particle.hpp(167): warning: __device__ annotation is ignored on a function("operator=") that is explicitly defaulted on its first declaration

/home/s0134766/src/picongpu/include/pmacc/../pmacc/particles/memory/dataTypes/Particle.hpp(167): warning: __host__ annotation is ignored on a function("operator=") that is explicitly defaulted on its first declaration
...
Many more

@BeyondEspresso
Copy link
Member

LWFA test case in /scratch/p_electron/debus/runs/01_LWFA_example exited with error:

/scratch/p_electron/debus/runs/01_LWFA_example/input/bin/picongpu: error while loading shared libraries: libhdf5.so.10: cannot open shared object file: No such file or directory

@ax3l ax3l mentioned this pull request Sep 6, 2018
8 tasks
@PrometheusPi
Copy link
Member

I will have a look into that issue today.

@PrometheusPi
Copy link
Member

PrometheusPi commented Sep 26, 2018

Without pngwriter and libsplash PIConGPU works fine.

When trying to compile pngwriter, I ran into the following error:

# cmake output
-- The CXX compiler identification is GNU 6.4.0
-- Check for working CXX compiler: /sw/installed/GCCcore/6.4.0/bin/g++
-- Check for working CXX compiler: /sw/installed/GCCcore/6.4.0/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.7") 
-- Found PNG: /usr/lib64/libpng.so (found suitable version "1.5.13", minimum required is "1.2.9") 
-- Found Freetype: /usr/lib64/libfreetype.so (found version "2.4.11") 

PNGwriter build configuration:
  PNGwriter Version: 0.7.0
  C++ Compiler : GNU 6.4.0 
    /sw/installed/GCCcore/6.4.0/bin/g++

  Installation prefix: /scratch/p_electron/richard/libSCS5/lib/pngwriter

  Build Type: Release
  Build Options:
    FREETYPE: ON

-- Configuring done
-- Generating done
-- Build files have been written to: /scratch/p_electron/richard/libSCS5/build

$ make install

[  8%] Building CXX object CMakeFiles/PNGwriter.dir/src/pngwriter.cc.o
[ 16%] Linking CXX static library libPNGwriter.a
[ 16%] Built target PNGwriter
[ 25%] Building CXX object CMakeFiles/blackwhite.dir/tests/blackwhite.cc.o
[ 33%] Linking CXX executable blackwhite
libPNGwriter.a(pngwriter.cc.o):pngwriter.cc:function pngwriter::close(): error: undefined reference to 'png_convert_to_rfc1123_buffer'
collect2: Fehler: ld gab 1 als Ende-Status zurück
make[2]: *** [blackwhite] Fehler 1
make[1]: *** [CMakeFiles/blackwhite.dir/all] Fehler 2
make: *** [all] Fehler 2

The module libpng/1.6.34-GCCcore-6.4.0 is, according to module list, correctly loaded.

@PrometheusPi
Copy link
Member

PrometheusPi commented Sep 26, 2018

libSplash could be build.

Compiling PIConGPU resulted in a number of boost warnings and some alpaka warnings:

/.../picongpu2/thirdParty/alpaka/include/alpaka/vec/Vec.hpp(925): warning: calling a __host__ function("std::__shared_count<( ::__gnu_cxx::_Lock_policy)2> ::__shared_count") from a __host__ __device__ function("alpaka::dev::DevCpu::DevCpu") is not allowed

/.../picongpu2/thirdParty/alpaka/include/alpaka/vec/Vec.hpp(925): warning: calling a __host__ function("std::__shared_count<( ::__gnu_cxx::_Lock_policy)2> ::__shared_count") from a __host__ __device__ function("alpaka::dev::DevCpu::DevCpu [subobject]") is not allowed

Edit:
As @BeyondEspresso already mentioned, there is a massive amount of warnings when compiling PIConGPU. But I additionally encountered a linking error.

pic-build looks fine:

build directory: .build
cmake command: cmake  -DCMAKE_INSTALL_PREFIX=.. -DPIC_EXTENSION_PATH=..   -DALPAKA_ACC_GPU_CUDA_ENABLE=ON -DALPAKA_ACC_GPU_CUDA_ONLY_MODE=ON -DALPAKA_CUDA_ARCH="37" /scratch/.../picongpu2/include/picongpu
-- The C compiler identification is GNU 6.4.0
-- The CXX compiler identification is GNU 6.4.0
-- Check for working C compiler: /sw/installed/GCCcore/6.4.0/bin/gcc
-- Check for working C compiler: /sw/installed/GCCcore/6.4.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /sw/installed/GCCcore/6.4.0/bin/g++
-- Check for working CXX compiler: /sw/installed/GCCcore/6.4.0/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found CUDA: /sw/installed/CUDA/9.2.88 (found suitable version "9.2", minimum required is "7.0") 
-- ALPAKA_ACC_GPU_CUDA_ONLY_MODE
-- ALPAKA_ACC_GPU_CUDA_ENABLED
-- Found alpaka: /scratch/.../picongpu2/thirdParty/alpaka/include (found version "0.3.2") 
-- _cupla_COMPILE_OPTIONS_PUBLIC: -std=c++11;-ftemplate-depth-512;-Wall;-Wextra;-Wno-unknown-pragmas;-Wno-unused-parameter;-Wno-unused-local-typedefs;-Wno-attributes;-Wno-reorder;-Wno-sign-compare
-- _cupla_COMPILE_DEFINITIONS_PUBLIC: ALPAKA_ACC_GPU_CUDA_ONLY_MODE;ALPAKA_ACC_GPU_CUDA_ENABLED;ALPAKA_DEBUG=0;CUPLA_STREAM_ASYNC_ENABLED=1
-- _cupla_INCLUDE_DIRECTORIES_PUBLIC: /sw/installed/CUDA/9.2.88/include;/scratch/.../picongpu2/thirdParty/alpaka/include;/scratch/.../picongpu2/thirdParty/cupla/include
-- _cupla_LINK_LIBRARIES_PUBLIC: Boost::boost;general;/sw/installed/CUDA/9.2.88/lib64/libcudart.so;general;pthread;general;rt
-- Found cupla: /scratch/.../picongpu2/thirdParty/cupla/include (found version "0.1.0") 
-- Found MPI_C: /software/haswell/OpenMPI/2.1.2-GCC-6.4.0-2.28/lib/libmpi.so (found version "3.1") 
-- Found MPI_CXX: /software/haswell/OpenMPI/2.1.2-GCC-6.4.0-2.28/lib/libmpi_cxx.so (found version "3.1") 
-- Found MPI: TRUE (found version "3.1")  
CMake Warning at /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
  /scratch/.../picongpu2/include/pmacc/PMaccConfig.cmake:276 (find_package)
  CMakeLists.txt:83 (find_package)


CMake Warning at /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
  /scratch/.../picongpu2/include/pmacc/PMaccConfig.cmake:276 (find_package)
  CMakeLists.txt:83 (find_package)


CMake Warning at /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
  /scratch/.../picongpu2/include/pmacc/PMaccConfig.cmake:276 (find_package)
  CMakeLists.txt:83 (find_package)


-- Boost version: 1.66.0
-- Found the following Boost libraries:
--   filesystem
--   system
--   math_tr1
-- Boost: result_of with TR1 style and decltype fallback
-- Boost: Disable template aliases
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Found CUDA: /sw/installed/CUDA/9.2.88 (found suitable version "9.2", minimum required is "5.0") 
-- Boost version: 1.66.0
-- Using mallocMC from thirdParty/ directory
-- Boost version: 1.66.0
-- Found mallocMC: /scratch/.../picongpu2/thirdParty/mallocMC/src (found suitable version "2.3.0", minimum required is "2.3.0")  
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Boost version: 1.66.0
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.7") 
CMake Warning at /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
  CMakeLists.txt:156 (find_package)


CMake Warning at /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
  CMakeLists.txt:156 (find_package)


CMake Warning at /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
  CMakeLists.txt:156 (find_package)


CMake Warning at /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
  CMakeLists.txt:156 (find_package)


CMake Warning at /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
  CMakeLists.txt:156 (find_package)


CMake Warning at /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
  CMakeLists.txt:156 (find_package)


-- Boost version: 1.66.0
-- Found the following Boost libraries:
--   program_options
--   regex
--   filesystem
--   system
--   math_tr1
--   serialization
-- Found CUDA: /sw/installed/CUDA/9.2.88 (found version "9.2") 
-- Could NOT find NVML (missing: NVML_LIBRARY) 
CMake Warning at /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
  /software/haswell/CMake/3.10.2-GCCcore-6.4.0/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
  /scratch/.../picongpu2/include/mpiInfo/CMakeLists.txt:83 (find_package)


-- Boost version: 1.66.0
-- Found the following Boost libraries:
--   program_options
-- Can NOT find 'adios_config' - set ADIOS_ROOT or check your PATH
-- Could NOT find ADIOS (missing: ADIOS_LIBRARIES ADIOS_INCLUDE_DIRS) (Required is at least version "1.13.1")
-- HDF5: Using hdf5 compiler wrapper to determine C configuration
-- Found HDF5: /sw/installed/HDF5/1.10.1-foss-2018a/lib/libhdf5.so;/sw/installed/Szip/2.1.1-GCCcore-6.4.0/lib/libsz.so;/sw/installed/zlib/1.2.11-GCCcore-6.4.0/lib/libz.so;/usr/lib64/libdl.so;/usr/lib64/libm.so;/usr/lib64/libpthread.so (found version "1.10.1")  
-- Found Splash: /scratch/.../libSCS5/lib/splash/lib/cmake/Splash
-- Could NOT find PNGwriter (missing: PNGwriter_DIR)
-- Could NOT find PNGwriter - set PNGwriter_DIR or check your CMAKE_PREFIX_PATH
-- Could NOT find ISAAC - set ISAAC_DIR or check your CMAKE_PREFIX_PATH
-- Configuring done
-- Generating done
-- Build files have been written to: /scratch/.../projects/_LWFA_default_SCS5/.build
Scanning dependencies of target picongpu-hostonly

final linking error:

/scratch/.../libSCS5/lib/splash/lib/libSplash.a(HandleMgr.cpp.o):HandleMgr.cpp:function splash::HandleMgr::get(splash::Dimensions): error: undefined reference to 'H5P_LST_FILE_CREATE_ID_g'
collect2: Fehler: ld gab 1 als Ende-Status zurück
make[2]: *** [picongpu] Fehler 1
make[1]: *** [CMakeFiles/picongpu.dir/all] Fehler 2
make: *** [all] Fehler 2

ERROR: Could not successfully run make install in build directory:
       .build

@PrometheusPi
Copy link
Member

PrometheusPi commented Sep 26, 2018

With regard to all these warnings - a newer boost, if needed even self compiled, might be a workaround.

@PrometheusPi
Copy link
Member

Self compiling boost 1.65.1 solved the issue.

A run time test showed, that hdf5 data is produced.

@PrometheusPi
Copy link
Member

PrometheusPi commented Sep 26, 2018

PIConGPU ran into a segmentation fault while writing hdf5 data.

UPDATE:
Not using the hdf5 plugin results in a running simulation - using the same binary.

UPDATE 2:
Only writing fields causes the same error.

UPDATE 3:
The linking issue mentioned previously is not solved by using boost 1.65.1 but seems to be related to compiling on the login node. The previous attempt worked because I was compiling PIConGPU on an interactive node.
Iterative node: taurusi6192 works
Login node: tauruslogin6 failed
This looks like we have a machine dependency here and are linking (?) against some local libraries.

@PrometheusPi
Copy link
Member

PrometheusPi commented Sep 26, 2018

offline discussion with @psychocoderHPC:
No local libraries are linked. A hard to check reason might be a wrong MPI in the hdf5 module.
Thus I will revert to the boost module and instead use a self-compiled hdf5 (with zlib via module).

@PrometheusPi
Copy link
Member

PrometheusPi commented Sep 26, 2018

Trying to build hdf5 from source caused the following MPI related error during configuration:

./configure --enable-parallel --enable-shared --prefix /scratch/.../libSCS5/lib/hdf5/

...
checking Threads support system scope... yes
checking for debug flags... none
checking whether function stack tracking is enabled... no
checking whether metadata trace file code is enabled... no
checking for API tracing... no
checking for instrumented library... no
checking whether to clear file buffers... yes
checking whether a memory checking tool will be used... no
checking for parallel support files... provided by compiler
checking whether a simple MPI-IO C program can be linked... no
configure: error: unable to link a simple MPI-IO C program

@psychocoderHPC
Copy link
Member

could you please test to set the mpi compiler instead compiling direct with the gnu compiler:

export CC=mpicc
export CXX=mpic++

@PrometheusPi
Copy link
Member

PrometheusPi commented Sep 27, 2018

with @psychocoderHPC trick, I could compile hdf5. Some test of hdf5 still failed with regard to MPI, bit I could build both splash and picongpu without compile errors.
PIConGPU is now running - let's see if it will crash again it writes hdf5 output like a charm.

@PrometheusPi
Copy link
Member

@ax3l Please rebase your branch against the current dev so that we can update the files easily.

@ax3l
Copy link
Member Author

ax3l commented Oct 5, 2018

@PrometheusPi updates, feel free to add further commits! :)

@ax3l
Copy link
Member Author

ax3l commented Oct 5, 2018

PNGwriter issue: you load libpng/1.6.34 but link against the system 1.5.13 libpng (or vice versa). Is it possible, the libpng module does not properly set the environment variable PNG_DIR or alternatively forgets to expand the CMAKE_PREFIX_PATH with its install location? If so, please report this the Taurs HPC support, ideally CMAKE_PREFIX_PATH should be expanded (prepend with install location of libpng on load).

@ax3l ax3l mentioned this pull request Oct 5, 2018
21 tasks
@PrometheusPi
Copy link
Member

@ax3l You were right, the CMAKE_PREFIX_PATH for libpng was not set. Setting it manually to the module path allows building the pngwriter.

I will open a ticket.

Furthermore, when linking picongpu with the pngwriterenabled an error occurs:

/sw/installed/libpng/1.6.34-GCCcore-6.4.0/lib/libpng.so: error: undefined reference to 'inflateValidate', version 'ZLIB_1.2.9'

libpng.so is linked to:

$ldd /sw/installed/libpng/1.6.34-GCCcore-6.4.0/lib/libpng.so
	linux-vdso.so.1 =>  (0x00007ffef53dc000)
	libz.so.1 => /sw/installed/zlib/1.2.11-GCCcore-6.4.0/lib/libz.so.1 (0x00007efda7239000)
	libm.so.6 => /usr/lib64/libm.so.6 (0x00007efda6f24000)
	libpthread.so.0 => /usr/lib64/libpthread.so.0 (0x00007efda6d08000)
	libc.so.6 => /usr/lib64/libc.so.6 (0x00007efda6945000)
	/lib64/ld-linux-x86-64.so.2 (0x000055c81a442000)

This looks like a mismatch - it links to the correctly loaded zlib module (1.2.11) but internally uses version 1.2.9.

I will open a ticket.

@ax3l
Copy link
Member Author

ax3l commented Oct 17, 2018

Yes, also make sure they expose the right zlib as a module again and that it itself sets CMAKE_PREFIX_PATH properly.

@PrometheusPi
Copy link
Member

Support ticket to taurus was submitted. ID: 2018101741002436

@PrometheusPi
Copy link
Member

@ComputationalRadiationPhysics/picongpu-developers Do we want to keep the Intel compiler for knl or do we switch to gcc?

@ax3l
Copy link
Member Author

ax3l commented Oct 17, 2018

In case ICC compiles for us, we should use it for KNL

@PrometheusPi
Copy link
Member

PrometheusPi commented Oct 17, 2018

reply from taurus support (in German):


Guten Tag @PrometheusPi,

aus meiner Sicht ist da, was unsere Module angeht, nichts falsch gebaut.

$ ml libpng/1.6.34-GCCcore-6.4.0
$ ldd $EBROOTLIBPNG/lib/libpng.so
[...]
libz.so.1 => /sw/installed/zlib/1.2.11-GCCcore-6.4.0/lib/libz.so.1 (0x00007f7f7ede4000)
[...]

Es wird also ganz normal die 1.2.11er gelinkt, die auch als Abhängigkeit für das libpng Modul gesetzt ist und somit mitgeladen wird.

Wenn Sie natürlich in Ihrer Anwendung nun aus irgendwelchen Gründen an anderer Stelle die libz.so aus /usr (1.2.9) gelinkt haben und zeitgleich auch libpng linken, kann so natürlich ein Konflikt entstehen, da ja nicht zwei Shared Libraries gleichen Namens im selben Prozess geladen werden können. D.h. da müssten Sie nochmal darauf schauen, was Ihr Buildsystem genau macht.

Was das CMAKE_PREFIX_PATH angeht: ansich setzen die Module alle CPATH, LIBRARY_PATH und PKG_CONFIG_PATH, das sollte dem Buildsystem doch eigentlich ausreichen, um nicht bevorzugt die Bibliotheken aus /usr zu benutzen...? Bei anderer Software funktioniert das schließlich auch.
Wir bauen unsere Software in letzter Zeit wenn möglich nur noch automatisiert über EasyBuild und wollen eigentlich jetzt nicht anfangen und manuell sämtliche Module oder auch nur die EasyConfigs von so Standardbibliotheken wie zlib/libpng oder auch Boost alle anpassen, da das z.B. bei neuen Toolchain Releases dann ja auch immer wieder mit immensem Aufwand verbunden ist und das den praktischen Nutzen von EasyBuild für uns, nämlich Arbeit zu ersparen, nicht unerheblich schmälern würde.
Wenn aus Ihrer Sicht also kein Weg um das Setzen von CMAKE_PREFIX_PATH herum führt und Sie das plausibel begründen können, wäre es vielleicht am sinnvollsten, sich damit mal an die EasyBuild Community (easybuild@lists.ugent.be) zu wenden. Ich denke, dort gibt es viele Leute, die sich intensiv mit der Thematik beschäftigt haben, und irgendwie kann ich mir nicht vorstellen, dass die das bislang einfach "übersehen" haben, bzw. vielleicht gibt es ja einen triftigen Grund, warum sie das eben nicht pauschal für alle Libraries mit setzen lassen.
Sollte das Anklang finden und in EasyBuild aufgenommen werden, würden wir dann automatisch auch davon profitieren.

Mit freundlichen Grüßen
[name of admin removed]


Therefore we need (for now) to set our CMAKE_PREFIX_PATH by ourself.

@ax3l
Copy link
Member Author

ax3l commented Oct 17, 2018

CPATH, LIBRARY_PATH und PKG_CONFIG_PATH,

Nope, we need a CMAKE_PREFIX_PATH, we are not in a pure Unix-systemtools nor autotools world. All modules should expand their CMAKE_PREFIX_PATH to their install prefixes... Please tell them, that all modern software projects based on CMake (C/C++/CUDA even Fortran) need this.

Regarding easybuild: it's just a config that ZIH did forget to set. Of course Easybuild supports setting arbitrary vars in modules. Do we have to explain them their own tools?

@ax3l
Copy link
Member Author

ax3l commented Oct 17, 2018

I wrote them an email explaining things. Nevertheless, did you find any <PACKAGENAME>_DIR or _ROOT variables we could as a fallback rely on?

@PrometheusPi
Copy link
Member

PrometheusPi commented Oct 17, 2018

As mentioned in the pr to your fork: the *.tplfiles do not need to be updated. @ax3l you can check the check boxes ✅

k20x and k80 run successful with the unchanged *.tpl- knl is still queuing.

@ax3l
Copy link
Member Author

ax3l commented Oct 17, 2018

Cool, thanks!

@@ -64,6 +67,6 @@ export PYTHONPATH=$PICSRC/lib/python:$PYTHONPATH

# "tbg" default options #######################################################
# - SLURM (sbatch)
# - "gpu1" queue
# - "gpu2" queue
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this comment should be gpu1

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and below it should read k20x.tpl :)

@@ -13,21 +13,37 @@ export MY_NAME="$(whoami) <$MY_MAIL>"
# - examples: "nano", "vim", "emacs -nw", "vi" or without terminal: "gedit"
#export EDITOR="nano"

# General modules #############################################################
# Modules #############################################################
#
module purge
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

purging here still a good idea? Works everywhere?

Copy link
Member

@PrometheusPi PrometheusPi Oct 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not hurt and there are no modules loaded before - however it throws a warning that tells you that it cannot unload the following modules - followed by an empty list.

To avoid confusion I vote for removing the "old" module purge 👴

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alright, I will take it out

@ax3l ax3l force-pushed the topic-taurusSCS5 branch 3 times, most recently from f0a2a77 to 9ff9d3b Compare October 17, 2018 22:59
@ax3l
Copy link
Member Author

ax3l commented Oct 18, 2018

@PrometheusPi can you check if this PR still contains typos or is everything in good shape? :)

export LD_LIBRARY_PATH=$HDF5_ROOT/lib:$LD_LIBRARY_PATH
export CMAKE_PREFIX_PATH=$HDF5_ROOT:$CMAKE_PREFIX_PATH

# pngwriter needs to be build by the user:
Copy link
Member

@sbastrakov sbastrakov Oct 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

be built, not be build; same on line 52 and in other profiles.
Also the whole construction feels weird to me, but I could not promptly come up with a clearly better alternative.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# pngwriter needs to be build by the user:
# pngwriter needs to be built by the user:

Copy link
Member

@PrometheusPi PrometheusPi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please apply the minor fixes - the rest looks fine

export LD_LIBRARY_PATH=$HDF5_ROOT/lib:$LD_LIBRARY_PATH
export CMAKE_PREFIX_PATH=$HDF5_ROOT:$CMAKE_PREFIX_PATH

# pngwriter needs to be build by the user:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# pngwriter needs to be build by the user:
# pngwriter needs to be built by the user:

export CMAKE_PREFIX_PATH=$PNGwriter_DIR:$CMAKE_PREFIX_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PNGwriter_DIR/lib/

# splash needs to be build by the user:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# splash needs to be build by the user:
# splash needs to be built by the user:

export LD_LIBRARY_PATH=$HDF5_ROOT/lib:$LD_LIBRARY_PATH
export CMAKE_PREFIX_PATH=$HDF5_ROOT:$CMAKE_PREFIX_PATH

# pngwriter needs to be build by the user:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# pngwriter needs to be build by the user:
# pngwriter needs to be built by the user:

export CMAKE_PREFIX_PATH=$PNGwriter_DIR:$CMAKE_PREFIX_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PNGwriter_DIR/lib/

# splash needs to be build by the user:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# splash needs to be build by the user:
# splash needs to be built by the user:

module load HDF5/1.10.1-intel-2018a
module load libpng/1.6.34-GCCcore-7.3.0

# module system does not expand cmake prefix path:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Imo this expand is not suitable here, as it does not mean expanding as for environment variable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

export

@ax3l ax3l force-pushed the topic-taurusSCS5 branch from 9ff9d3b to 5d6ef3d Compare October 18, 2018 08:22
@ax3l ax3l force-pushed the topic-taurusSCS5 branch from 5d6ef3d to 3c499ed Compare October 18, 2018 08:25
@PrometheusPi
Copy link
Member

PrometheusPi commented Oct 18, 2018

Regarding the knl partition:
The knl queue currently has 0 nodes - therefore no run time test can be performed.

I will open opened a support ticket.

@psychocoderHPC psychocoderHPC merged commit 4edb0bc into ComputationalRadiationPhysics:dev Oct 18, 2018
@ax3l ax3l deleted the topic-taurusSCS5 branch October 18, 2018 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation regarding documentation or wiki discussions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants