Problems setting up unit testing framework

[brhillman]

Not sure if this is the right place to put this, but I am having trouble setting up the unit testing framework, following the instructions here. I have installed pFUnit on a cluster that we have CIME otherwise configured for, adding the configuration to config_compilers.xml, and followed the instructions for running the existing unit tests, running

scripts/fortran_unit_testing/run_tests.py --build-dir `mktemp -d ./unit_tests.XXXXXXXX`

from the top-level CIME directory. This seems to fail somewhere short of actually running unit tests for me. The first errors that seem to show up are in reference to undefined references to operators in shr_vmath_mod.F90, e.g.,

CMakeFiles/vmath_test_exe.dir/__/__/__/util/shr_vmath_mod.F90.o: In function `shr_vmath_mod_mp_shr_vmath_sqrt_':
shr_vmath_mod.F90:(.text+0x3e): undefined reference to `vdsqrt_'

There's a handful of these errors, and then a lot more warnings...finally the script exits, with the following error:

ERROR: Command: 'make -j 8' failed with error 'None' from dir '/home/bhillma/codes/e3sm/branches/brhillman/cime/add-unit-tests-sems/cime/unit_tests.0DQu7Goq/__command_line_test__/__command_line_test__'

I don't know if any of this is helpful, but does anybody have any ideas as to what is going on here? I would really like to get this working so that I can use the framework to write unit tests for new development that I am doing.

[jedwards4b]

What compiler and compiler version are you using? Can you successfully build a simple case on this machine?

[brhillman]

I am building on skybridge with the Intel compiler, my .env_mach_specific.sh is as follows:

module purge 
module load sems-env
module load sems-git
module load sems-python/2.7.9
module load sems-cmake
module load gnu/4.9.2
module load sems-intel/16.0.2
module load mkl/16.0
module load sems-netcdf/4.4.1/exo
export NETCDFROOT=/projects/sems/install/toss3/sems/tpl/netcdf/4.4.1/intel/16.0.2/exo
export NETCDF_INCLUDES=/projects/sems/install/toss3/sems/tpl/netcdf/4.4.1/intel/16.0.2/exo/include
export NETCDF_LIBS=/projects/sems/install/toss3/sems/tpl/netcdf/4.4.1/intel/16.0.2/exo/lib
export OMP_STACKSIZE=64M
export COMPILER=intel
export MPILIB=mpi-serial
export DEBUG=FALSE
export OS=LINUX

I can build and run ACME/E3SM without issues.

[jedwards4b]

Send the entire output of scripts/fortran_unit_testing/run_tests.py --build-dir mktemp -d ./unit_tests.XXXXXXXX
maybe we'll spot something there.

[brhillman]

run_tests_output.txt

[mfdeakin-sandia]

It looks like it's not linking to libvml.
https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/306437
I haven't gone through enough of the cmake to determine why yet, but adding something like the following to ACME/cime/src/share/test/unit/shr_vmath_test/CMakeLists.txt should fix it for this executable:

target_link_libraries(vmath_test_exe vml)

This should be after the add_executable line

[brhillman]

@mfdeakin-sandia it didn't seem to like that...I get the following error:

checking for strings.h... ld: cannot find -lvml

See attached:
run_tests_output_2.txt

[mfdeakin-sandia]

I looked at Skybridge's libraries, can you try changing it to mkl_rt instead? Or maybe mkl_vml_avx

[brhillman]

@mfdeakin-sandia yes but not for a week...I'm on my way out of town at the moment and will have to revisit when I get back.

[mfdeakin-sandia]

Alright, I'll look into it

[billsacks]

I'd be surprised if adding a link line like this in the CMakeLists.txt is the right solution. The CMake build is supposed to pick up the link libraries specified in config_compilers.xml. For skybridge, I see:

  <ADD_SLIBS> $(shell $(NETCDF_PATH)/bin/nf-config --flibs) -L/projects/ccsm/BLAS-intel -lblas_LINUX</ADD_SLIBS>
  <PIO_FILESYSTEM_HINTS>lustre </PIO_FILESYSTEM_HINTS>
  <ADD_SLIBS MPILIB="openmpi"> -mkl=cluster </ADD_SLIBS>
  <ADD_SLIBS MPILIB="mpi-serial"> -mkl </ADD_SLIBS>

My first question is whether those SLIBS are being correctly added to the cmake build.

You can run run_tests.py with the -v argument to get the full build commands for each target. It would be helpful to compare the output from this with the output from a system build on this machine/compiler to determine what's missing from the build lines of the unit test builds.

[mfdeakin-sandia]

I agree it's not the correct solution, I just want to understand what needs to happen for this to work at this point, as I don't know how the cmake is configured at this point

[billsacks]

@mfdeakin ah, got it - thanks for the clarification.

[mfdeakin-sandia]

The issue is two-fold: it needs to link to mkl_rt, and the MKL library path isn't specified
@jgfouca Is it okay to just add these to the SLIBS variable for Skybridge, or should it only be for a subset of the things we build? Also, is the mkl library compiler specific?

[billsacks]

MKL is intel's math kernel library (which includes lapack and blas functionality), so yes, it's compiler-specific.

…

On Nov 20, 2017, at 1:52 PM, Michael Deakin ***@***.*** ***@***.***>> wrote: The issue is two-fold: it needs to link to mkl_rt, and the MKL library path isn't specified @jgfouca <https://github.com/jgfouca> Is it okay to just add these to the SLIBS variable for Skybridge, or should it only be for a subset of the things we build? Also, is the mkl library compiler specific? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2080 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AF-fdaacDMrPH-fZ2X1utGFE1yKFRMoWks5s4eaRgaJpZM4QhTwP>. <https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png> <https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png> <https://github.com/ESMCI/cime> <#2080 (comment)>

[jgfouca]

@mfdeakin-sandia , I think it would be OK to just add them.