Profile C++ code executed by Python script?

[mhieronymus]

I have some code in C++ instrumented with Score-P that is being called from a Python script.
I used

SCOREP_WRAPPER=off \
cmake ../src \
-DCMAKE_C_COMPILER=scorep-gcc \
-DCMAKE_CXX_COMPILER=scorep-g++ \
-DCMAKE_C_FLAGS=-finstrument-functions-exclude-file-list=intrin.h
make

to build and compile the files. When I use

python -m scorep my_script.py

I can only see the execution time for the calls in Python but I have no information about different methods that are called in C++. I use cube to take a look at profile.cubex. Am I missing something?

[mhieronymus]

Here a minimal example:
foo.cpp

#include <iostream>
#include <vector>

class Foo{
    public:
        void bar(){
            std::cout << "Hello" << std::endl;
            int i = calc(2000);
            i += calc(10000);
            i -= calc(1000);
            i *= calc(100);
            std::cout << i << std::endl;
        }
        
        int calc(int n) {
            std::vector<int> v(n);
            int i = 0;
            for(auto & e: v) {e = i; ++i;}
            for(i=0; i<n-1; ++i) v[i] += v[i+1];
            int ret = get(v, n);
            return ret;
        }
        
        int get(std::vector<int> & v, int n) {
            return v[n-1];
        }
};

extern "C" {
    Foo* Foo_new(){ return new Foo(); }
    void Foo_bar(Foo* foo){ foo->bar(); }
}

fooWrapper.py:

from ctypes import cdll
lib = cdll.LoadLibrary('./libfoo.so')

class Foo(object):
    def __init__(self):
        self.obj = lib.Foo_new()

    def bar(self):
        lib.Foo_bar(self.obj)

foo.py:

from fooWrapper import Foo

f = Foo() 
f.bar()

Compiled and linked with

/opt/scorep/bin/scorep-g++ -c -fPIC foo.cpp -o foo.o --std=c++14
/opt/scorep/bin/scorep-g++ -shared -Wl,-soname,libfoo.so -o libfoo.so foo.o --std=c++14

Executed with

python -m scorep foo.py 

Note: If I tried just using python foo.py, I'd get errors for undefined symbols such as

OSError: /opt/scorep/lib/libscorep_measurement.so.6: undefined symbol: scorep_ipc_group_world

I can fix that by using

from scorep.__main__ import global_trace

but no profiling will be done for the C++ code either.

Another note:
Using pure C++ I noticed, I cannot profile anything that comes from shared libraries (which have been instrumented with Score-P). I'll look into the documentation again and post whatever I find out.

[AndreasGocht]

Hey,

the example you posted worked for me.

However, it might not work as you might expected. The point is, that foo.cpp is more or less header only. This means that all functions of the class Foo are virtual. The Score-P compiler plugin for gcc and g++ is written in that way, that it ignores all virtual functions.
This has the consequence, that only the calls to Foo_new and Foo_bar are instrumented and therefore recorded.

If you change foo.cpp to

#include <iostream>
#include <vector>
#include "foo.h"

void Foo::bar(){
    std::cout << "Hello" << std::endl;
    int i = calc(2000);
    i += calc(10000);
    i -= calc(1000);
    i *= calc(100);
    std::cout << i << std::endl;
}
    
int Foo::calc(int n) {
    std::vector<int> v(n);
    int i = 0;
    for(auto & e: v) {e = i; ++i;}
        for(i=0; i<n-1; ++i) v[i] += v[i+1];
            int ret = get(v, n); 
    return ret;
}
    
int Foo::get(std::vector<int> & v, int n) {
    return v[n-1];
}

extern "C" {
    Foo* Foo_new(){ return new Foo(); }
    void Foo_bar(Foo* foo){ foo->bar(); }
}

and add a foo.h with the content of

class Foo{
    public:
        void bar();
        int calc(int n);
        int get(std::vector<int> & v, int n);
};  

compile and run as you have written, all calls are instrumented. You should see the calls to Foo::get and so on in the profile:

  |   |   |   |   + importlib._bootstrap:<genexpr>                                                                                          
  |   |   |   |   + importlib._bootstrap:_verbose_message                                                                                   
  |   + importlib._bootstrap:__exit__                                                                                                       
  |   |   + importlib._bootstrap:release                                                                                                    
  |   + importlib._bootstrap:cb                                                                                                             
  + importlib._bootstrap:_handle_fromlist                                                                                                   
  + fooWrapper:__init__                                                                                                                     
  |   + ctypes:__getattr__                                                                                                                  
  |   |   + ctypes:__getitem__                                                                                                              
  |   + Foo* Foo_new()                                                                                                                      
  + fooWrapper:bar                                                                                                                          
  |   + ctypes:__getattr__                                                                                                                  
  |   |   + ctypes:__getitem__                                                                                                              
  |   + void Foo_bar(Foo*)                                                                                                                  
  |   |   + void Foo::bar()                                                                                                                 
  |   |   |   + int Foo::calc(int)                                                                                                          
  |   |   |   |   + int Foo::get(std::vector<int>&, int)                                                                                    

[output of cube_calltree]

I hope this answers your question.

Best,

Andreas

[mhieronymus]

I still can't see the call to Foo_bar(Foo*) and everything below. Perhaps something is odd with my Score-P installation? Is there a way to check that? Or maybe I should just reinstall Score-P.
Here is my output when using --verbose=2 in case I missed something:

scorep --verbose=2 --dynamic g++ -fPIC -c foo.cpp -o foo.o --std=c++14

Enabled instrumentation: compiler
Selected paradigms:
    thread=none
    mpp=none
    mutex=none


Compiler name: g++
Flags before interposition lib:  -fPIC -c --std=c++14
Flags after interposition lib: 
Output file: foo.o
Input file(s): foo.cpp
g++ `/opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --cxxflags`  -I/home/mhierony/Documents/Uni/Masterthesis/tmp/should_work  -fPIC -c --std=c++14  -c /home/mhierony/Documents/Uni/Masterthesis/tmp/should_work/foo.cpp -o foo.o
 Executing: /opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --cxxflags
 Executing: g++ -g -finstrument-functions -I/opt/scorep/include -I/opt/scorep/include/scorep -I/home/mhierony/Documents/Uni/Masterthesis/tmp/should_work -fPIC -c --std=c++14 -c /home/mhierony/Documents/Uni/Masterthesis/tmp/should_work/foo.cpp -o foo.o
scorep --verbose=2 --dynamic g++ -shared  -Wl,-soname,libfoo.so -o libfoo.so foo.o --std=c++14

Enabled instrumentation: compiler
Selected paradigms:
    thread=none
    mpp=none
    mutex=none


Compiler name: g++
Flags before interposition lib:  -shared -Wl\,-soname\,libfoo.so --std=c++14
Flags after interposition lib: 
Output file: libfoo.so
Input file(s): foo.o
/usr/bin/nm -B foo.o 2>/dev/null | /bin/grep -E -l ' U (malloc|free|calloc|realloc|memalign|posix_memalign|valloc)$' >/dev/null 2>&1
/usr/bin/nm -B foo.o 2>/dev/null | /bin/grep -E -l ' U aligned_alloc$' >/dev/null 2>&1
/usr/bin/nm -B foo.o 2>/dev/null | /bin/grep -E -l ' U (_Znwj|_Znaj)$' >/dev/null 2>&1
/usr/bin/nm -B foo.o 2>/dev/null | /bin/grep -E -l ' U (_Znwm|_Znam)$' >/dev/null 2>&1
/usr/bin/nm -B foo.o 2>/dev/null | /bin/grep -E -l ' U (__nw__FUi|__nwa__FUi)$' >/dev/null 2>&1
/usr/bin/nm -B foo.o 2>/dev/null | /bin/grep -E -l ' U (__nw__FUl|__nwa__FUl)$' >/dev/null 2>&1
g++ foo.o `/opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --memory=c++L64 --dynamic --ldflags`  -shared -Wl\,-soname\,libfoo.so --std=c++14 `/opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --memory=c++L64 --dynamic --event-libs`  `/opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --memory=c++L64 --dynamic --mgmt-libs` -o libfoo.so
 Executing: /opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --memory=c++L64 --dynamic --ldflags
 Executing: /opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --memory=c++L64 --dynamic --event-libs
 Executing: /opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --memory=c++L64 --dynamic --mgmt-libs
 Executing: g++ foo.o -L/opt/otf2/lib -L/home/mhierony/lib/cubew/lib -L/opt/scorep/lib -Wl,-rpath -Wl,/opt/otf2/lib -Wl,-rpath -Wl,/home/mhierony/lib/cubew/lib -Wl,-rpath -Wl,/opt/scorep/lib -g -finstrument-functions -Wl,--undefined,__wrap__ZdlPv,-wrap,_ZdlPv,-wrap,_ZdaPv -Wl,--undefined,__wrap__Znwm,-wrap,_Znwm,-wrap,_Znam -shared -Wl,-soname,libfoo.so --std=c++14 /opt/scorep/lib/libscorep_adapter_compiler_event.so /opt/scorep/lib/libscorep_adapter_memory_event_cxx.so /opt/scorep/lib/libscorep_adapter_memory_event_cxx_L64.so /opt/scorep/lib/libscorep_measurement.so /opt/scorep/lib/libscorep_adapter_utils.so /home/mhierony/lib/cubew/lib/libcube4w.so /opt/scorep/lib/libscorep_adapter_compiler_mgmt.so /opt/scorep/lib/libscorep_adapter_memory_mgmt.so /opt/scorep/lib/libscorep_mpp_mockup.so /opt/scorep/lib/libscorep_online_access_mockup.so /opt/scorep/lib/libscorep_thread_mockup.so /opt/scorep/lib/libscorep_mutex_mockup.so /opt/scorep/lib/libscorep_alloc_metric.so -ldl -lz -lm -o libfoo.so
scorep --verbose=2 --dynamic g++    -c -o c.o c.cpp

Enabled instrumentation: compiler
Selected paradigms:
    thread=none
    mpp=none
    mutex=none


Compiler name: g++
Flags before interposition lib:  -c
Flags after interposition lib: 
Output file: c.o
Input file(s): c.cpp
g++ `/opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --cxxflags`  -I/home/mhierony/Documents/Uni/Masterthesis/tmp/should_work  -c  -c /home/mhierony/Documents/Uni/Masterthesis/tmp/should_work/c.cpp -o c.o
 Executing: /opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --cxxflags
 Executing: g++ -g -finstrument-functions -I/opt/scorep/include -I/opt/scorep/include/scorep -I/home/mhierony/Documents/Uni/Masterthesis/tmp/should_work -c -c /home/mhierony/Documents/Uni/Masterthesis/tmp/should_work/c.cpp -o c.o
scorep --verbose=2 --dynamic g++ -o c  c.o -L. -lfoo --std=c++14

Enabled instrumentation: compiler
Selected paradigms:
    thread=none
    mpp=none
    mutex=none


Compiler name: g++
Flags before interposition lib:  -L. -lfoo --std=c++14
Flags after interposition lib: 
Output file: c
Input file(s): c.o
/usr/bin/nm -B c.o  2>/dev/null | /bin/grep -E -l ' U (malloc|free|calloc|realloc|memalign|posix_memalign|valloc)$' >/dev/null 2>&1
/usr/bin/nm -B c.o  2>/dev/null | /bin/grep -E -l ' U aligned_alloc$' >/dev/null 2>&1
/usr/bin/nm -B c.o  2>/dev/null | /bin/grep -E -l ' U (_Znwj|_Znaj)$' >/dev/null 2>&1
/usr/bin/nm -B c.o  2>/dev/null | /bin/grep -E -l ' U (_Znwm|_Znam)$' >/dev/null 2>&1
/usr/bin/nm -B c.o  2>/dev/null | /bin/grep -E -l ' U (__nw__FUi|__nwa__FUi)$' >/dev/null 2>&1
/usr/bin/nm -B c.o  2>/dev/null | /bin/grep -E -l ' U (__nw__FUl|__nwa__FUl)$' >/dev/null 2>&1
/opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --adapter-init > c.scorep_init.c
 Executing: /opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --adapter-init
gcc -c c.scorep_init.c -o c.scorep_init.o
 Executing: gcc -c c.scorep_init.c -o c.scorep_init.o
g++ c.scorep_init.o c.o `/opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --constructor` `/opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --ldflags`  -L. -lfoo --std=c++14 `/opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --event-libs`  `/opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --mgmt-libs` -o c
 Executing: /opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --constructor
 Executing: /opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --ldflags
 Executing: /opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --event-libs
 Executing: /opt/scorep/bin/scorep-config --thread=none --mpp=none --mutex=none --nocuda --noonline-access --noopencl --noopenacc --nomemory --dynamic --mgmt-libs
 Executing: g++ c.scorep_init.o c.o /opt/scorep/lib/scorep/scorep_constructor.o -L/opt/otf2/lib -L/home/mhierony/lib/cubew/lib -L/opt/scorep/lib -Wl,-rpath -Wl,/opt/otf2/lib -Wl,-rpath -Wl,/home/mhierony/lib/cubew/lib -Wl,-rpath -Wl,/opt/scorep/lib -g -finstrument-functions -L. -lfoo --std=c++14 /opt/scorep/lib/libscorep_adapter_compiler_event.so /opt/scorep/lib/libscorep_measurement.so /opt/scorep/lib/libscorep_adapter_utils.so /home/mhierony/lib/cubew/lib/libcube4w.so /opt/scorep/lib/libscorep_adapter_compiler_mgmt.so /opt/scorep/lib/libscorep_mpp_mockup.so /opt/scorep/lib/libscorep_online_access_mockup.so /opt/scorep/lib/libscorep_thread_mockup.so /opt/scorep/lib/libscorep_mutex_mockup.so -ldl -lz -lm -o c
rm c.scorep_init.c c.scorep_init.o
 Executing: rm c.scorep_init.c c.scorep_init.o

I appreciate your help!

[AndreasGocht]

In theory this should work. However, I wonder which gcc version you are using?

Score-P uses -finstrument-functions instead of the gcc compiler plugin, which is recommended for modern gcc version.

Can you please post the output of

gcc --version

and

scorep-info config-summary

?

Best,

Andreas

[mhieronymus]

It works now. I noticed, I didn't have headers installed for gcc plugins, I missed libopenmpi-dev, libunwind and perf (I didn't have PAPI as an alternative either). After installing that and reinstalling Score-P, I get all the details I have dreamed of. Thanks again!

Just for completion, in case someone is looking at this issue, this is what I get now:

>> gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

and

>> scorep-info config-summary
Configure command:
  ../configure                  '--enable-shared'

Configuration summary:
  Score-P 4.0:
    Platform:                   linux (auto detected)
    Cross compiling:            no (auto detected)
    Machine name:               Linux
    otf2 support:               yes, using external via /opt/otf2/bin//otf2-config
    opari2 support:             yes, using external via /opt/opari2/bin//opari2-config
    cubew support:              yes, using external via /home/mhierony/bin/cubew-config
    cubelib support:            yes, using external via /home/mhierony/bin/cubelib-config

    Score-P (backend):
      C99 compiler used:        gcc
      Pthread support:          yes, using gcc -pthread 
      compiler constructor:     yes, using attribute syntax
      Platform:                 linux (provided)
      TLS support:              yes, using __thread and the initial-exec model
      PAPI support:             no
      metric perf support:      yes
      Unwinding support:        yes
        libunwind support:      yes, using -D_GNU_SOURCE -lunwind
      Sampling support:         yes, using -D_GNU_SOURCE, sa_sigaction
      getrusage support:        yes
      RUSAGE_THREAD support:    yes, using -D_GNU_SOURCE
      dlfcn support:            yes, using -ldl
      OpenCL support:           yes
        libOpenCL support:      yes, using -lOpenCL
      OTF2 features:
        SIONlib support:        no
      CUDA support:             no
        libcudart support:      no
        libcuda support:        no
        libcupti support:       no
        CUPTI async support:    no
        CUDA version >= 6.0:    no
      OpenACC support:          no
      OpenMP support:           yes, using -fopenmp
        OpenMP pomp_tpd:        yes
        OpenMP ancestry:        yes
      PDT support:              no
      Cobi support:             no
      Timer support:            yes, providing gettimeofday, clock_gettime(CLOCK_MONOTONIC_RAW), tsc (X86_64)
        Timer CPPFLAGS:         -D_POSIX_C_SOURCE=199309L
        Timer LDFLAGS:          -lm
      libbfd support:           no
      compiler instrumentation: yes, using GCC plug-in with support for compile-time filtering
      Online access support:    yes
      Memory tracking support:  yes
      Compiler wrappers:        scorep-gcc scorep-g++ scorep-gfortran scorep-mpicc scorep-mpiCC scorep-mpif77 scorep-oshcc scorep-oshfort 
      User library wrappers support:
                                no

    Score-P (GCC plug-in):
      GCC plug-in support:      yes, using the C++ compiler and -I/usr/lib/gcc/x86_64-linux-gnu/5/plugin/include
      Compiler used:            g++ -std=c++11

    Score-P (libwrap):
      Library wrapper support:  no, llvm-config not found

    Score-P (MPI backend):
      C99 compiler works:       yes, using mpicc
      C++ compiler works:       yes, using mpiCC
      F77 compiler works:       yes, using mpif77
      F90 compiler works:       yes, using mpif77
      Library used for MPI:     
      TLS support:              yes, using __thread and the initial-exec model
      PAPI support:             no
      metric perf support:      yes
      Unwinding support:        yes
        libunwind support:      yes, using -D_GNU_SOURCE -lunwind
      Sampling support:         yes, using -D_GNU_SOURCE, sa_sigaction
      getrusage support:        yes
      RUSAGE_THREAD support:    yes, using -D_GNU_SOURCE
      dlfcn support:            yes, using -ldl
      OpenCL support:           yes
        libOpenCL support:      yes, using -lOpenCL
      CUDA support:             no
        libcudart support:      no
        libcuda support:        no
        libcupti support:       no
        CUPTI async support:    no
        CUDA version >= 6.0:    no
      OpenACC support:          no
      OpenMP support:           yes, using -fopenmp
        OpenMP pomp_tpd:        yes
        OpenMP ancestry:        yes
      PDT MPI instrumentation:  yes, if PDT available
      Timer support:            yes, providing gettimeofday, clock_gettime(CLOCK_MONOTONIC_RAW), tsc (X86_64)
        Timer CPPFLAGS:         -D_POSIX_C_SOURCE=199309L
        Timer LDFLAGS:          -lm
      libbfd support:           no
      compiler instrumentation: yes, using GCC plug-in with support for compile-time filtering
      Online access support:    yes
      Memory tracking support:  yes
      User library wrappers support:
                                no

    Score-P (score):
      C compiler used:          gcc 
      C++ compiler used:        g++
      cube c++ library support: yes, using -I/home/mhierony/lib/cubew/include/cubelib  -L/home/mhierony/lib/cubew/lib -Wl,-rpath -Wl,/home/mhierony/lib/cubew/lib  -lcube4 -lz

    Score-P (SHMEM backend):
      C99 compiler used:        oshcc
      C99 compiler works:       yes, using oshcc
      C++ compiler works:       yes, using oshcc
      F77 compiler works:       yes, using oshfort
      F90 compiler works:       yes, using oshfort
      Library used for SHMEM:   
      TLS support:              yes, using __thread and the initial-exec model
      PAPI support:             no
      metric perf support:      yes
      Unwinding support:        yes
        libunwind support:      yes, using -D_GNU_SOURCE -lunwind
      Sampling support:         yes, using -D_GNU_SOURCE, sa_sigaction
      getrusage support:        yes
      RUSAGE_THREAD support:    yes, using -D_GNU_SOURCE
      dlfcn support:            yes, using -ldl
      OpenCL support:           yes
        libOpenCL support:      yes, using -lOpenCL
      CUDA support:             no
        libcudart support:      no
        libcuda support:        no
        libcupti support:       no
        CUPTI async support:    no
        CUDA version >= 6.0:    no
      OpenACC support:          no
      OpenMP support:           yes, using -fopenmp
        OpenMP pomp_tpd:        yes
        OpenMP ancestry:        yes
      PDT SHMEM instrumentation:
                                yes, if PDT available
      Timer support:            yes, providing gettimeofday, clock_gettime(CLOCK_MONOTONIC_RAW), tsc (X86_64)
        Timer CPPFLAGS:         -D_POSIX_C_SOURCE=199309L
        Timer LDFLAGS:          -lm
      libbfd support:           no
      compiler instrumentation: yes, using GCC plug-in with support for compile-time filtering
      Online access support:    yes
      Memory tracking support:  yes
      User library wrappers support:
                                no

    Score-P (score):
      C compiler used:          gcc 
      C++ compiler used:        g++
      cube c++ library support: yes, using -I/home/mhierony/lib/cubew/include/cubelib  -L/home/mhierony/lib/cubew/lib -Wl,-rpath -Wl,/home/mhierony/lib/cubew/lib  -lcube4 -lz

    Score-P (SHMEM backend):
      C99 compiler used:        oshcc
      C99 compiler works:       yes, using oshcc
      C++ compiler works:       yes, using oshcc
      F77 compiler works:       yes, using oshfort
      F90 compiler works:       yes, using oshfort
      Library used for SHMEM:   
      TLS support:              yes, using __thread and the initial-exec model
      PAPI support:             no
      metric perf support:      yes
      Unwinding support:        yes
        libunwind support:      yes, using -D_GNU_SOURCE -lunwind
      Sampling support:         yes, using -D_GNU_SOURCE, sa_sigaction
      getrusage support:        yes
      RUSAGE_THREAD support:    yes, using -D_GNU_SOURCE
      dlfcn support:            yes, using -ldl
      OpenCL support:           yes
        libOpenCL support:      yes, using -lOpenCL
      CUDA support:             no
        libcudart support:      no
        libcuda support:        no
        libcupti support:       no
        CUPTI async support:    no
        CUDA version >= 6.0:    no
      OpenACC support:          no
      OpenMP support:           yes, using -fopenmp
        OpenMP pomp_tpd:        yes
        OpenMP ancestry:        yes
      PDT SHMEM instrumentation:
                                yes, if PDT available
      intercepting SHMEM calls: yes, using SHMEM profiling interface
      libbfd support:           no
      compiler instrumentation: yes, using GCC plug-in with support for compile-time filtering
      Memory tracking support:  yes
      User library wrappers support:
                                no

[AndreasGocht]

It works now. I noticed, I didn't have headers installed for gcc plugins, I missed libopenmpi-dev, libunwind and perf (I didn't have PAPI as an alternative either). After installing that and reinstalling Score-P, I get all the details I have dreamed of. Thanks again!

I discussed your problem with the Score-P maintainer, and he told me that this is a regular problem 😉 .
Just for completeness: The needed gcc plugin header files are usually included in a package called gcc-8-plugin-dev or similar.

Best,

Andreas

[samcom12]

Hi,
I'm getting similar errors with score-p python bindings.

/usr/bin/ld: cannot find -lz
collect2: error: ld returned 1 exit status
Traceback (most recent call last):
  File "/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/python-3.9.9-j2qadxa25fkfwvn6t63w7xigwaolxgkp/lib/python3.9/distutils/unixccompiler.py", line 200, in link
/usr/bin/ld: cannot find -lz
collect2: error: ld returned 1 exit status
Traceback (most recent call last):
  File "/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/python-3.9.9-j2qadxa25fkfwvn6t63w7xigwaolxgkp/lib/python3.9/distutils/unixccompiler.py", line 200, in link
/usr/bin/ld: cannot find -lz
collect2: error: ld returned 1 exit status
/usr/bin/ld: cannot find -lz

GCC-version

 $gcc --version
gcc (Spack GCC) 11.2.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

and

$ scorep-info config-summary

Configure command:
  /tmp/samir/spack-stage/spack-stage-scorep-7.1-enjo72dn77yskd355ohixtxc5k4l6e44/spack-src/configure \
                                '--prefix=/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/scorep-7.1-enjo72dn77yskd355ohixtxc5k4l6e44' \
                                '--with-otf2=/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/otf2-2.3-eyyjpy7esf3jyclozbptm7m3lrryxjsw/bin' \
                                '--with-opari2=/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/opari2-2.0.6-ivz5oswqawsva3n43hwrdmvkrjidycgn/bin' \
                                '--enable-shared' \
                                '--with-nocross-compiler-suite=gcc' \
                                '--with-cubew=/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/cubew-4.6-xm2ruiidsnkoz3w7josjrsk27ztkckq4/bin' \
                                '--with-cubelib=/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/cubelib-4.6-gypjjtab7uyj376y46zbw5ha2v7ls47y/bin' \
                                '--with-papi-header=/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/papi-6.0.0.1-ji55i5ovyrydv6pjz5ghl5wubycjsf2e/include' \
                                '--with-papi-lib=/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/papi-6.0.0.1-ji55i5ovyrydv6pjz5ghl5wubycjsf2e/lib' \
                                '--without-shmem' \
                                '--with-mpi' \
                                '--with-mpi=mpich3' \
                                'CFLAGS=-fPIC' \
                                'CXXFLAGS=-fPIC' \
                                'MPICC=/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/mpich-3.4.2-sch55ydbzy3wkihpsjq3xhiwbtxddmlp/bin/mpicc' \
                                'MPICXX=/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/mpich-3.4.2-sch55ydbzy3wkihpsjq3xhiwbtxddmlp/bin/mpic++' \
                                'MPIF77=/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/mpich-3.4.2-sch55ydbzy3wkihpsjq3xhiwbtxddmlp/bin/mpif77' \
                                'MPIFC=/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/mpich-3.4.2-sch55ydbzy3wkihpsjq3xhiwbtxddmlp/bin/mpif90'

Loaded modules:
  module load                   autotools \
                                prun/1.3 \
                                gnu8/8.3.0 \
                                openmpi3/3.1.4 \
                                ohpc

Configuration summary:
  Score-P 7.1:
    Platform:                   linux (auto detected)
    Cross compiling:            no (auto detected)
    Machine name:               Linux
    otf2 support:               yes, using external via /home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/otf2-2.3-eyyjpy7esf3jyclozbptm7m3lrryxjsw/bin/otf2-config
    opari2 support:             yes, using external via /home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/opari2-2.0.6-ivz5oswqawsva3n43hwrdmvkrjidycgn/bin/opari2-config
    cubew support:              yes, using external via /home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/cubew-4.6-xm2ruiidsnkoz3w7josjrsk27ztkckq4/bin/cubew-config
    cubelib support:            yes, using external via /home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/cubelib-4.6-gypjjtab7uyj376y46zbw5ha2v7ls47y/bin/cubelib-config
    SHMEM support:              no, explicitly disabled via --without-shmem

    Score-P (backend):
      C99 compiler used:        gcc
      Link mode:                static=yes, shared=yes
      Pthread support:          yes, using gcc -pthread -lpthread
      compiler constructor:     yes, using attribute syntax without arguments
      TLS support:              yes, using __thread and the initial-exec model
      PAPI support:             yes
      metric perf support:      yes
      Unwinding support:        no, missing libunwind support
        libunwind found:        no
      Sampling support:         no
      getrusage support:        yes
      RUSAGE_THREAD support:    yes, using -D_GNU_SOURCE
      dlfcn support:            yes, using -ldl
      OpenCL support:           no
        libOpenCL found:        no
      I/O Recording features:
        POSIX I/O support:      yes
        POSIX asynchronous I/O support: \
                                yes, using -lrt
      OTF2 features:
        SIONlib support:        no
      CUDA support:             no
        libcudart found:        no
        libcuda found:          no
        libcupti found:         no
        libnvidia-ml found:     no
        CUPTI async support:    no
        CUDA version:           unknown
      OpenACC support:          no
      Kokkos support:           yes
      Mount point extraction:   yes
      Lustre stripe info support: \
                                yes
        liblustreapi found:     yes, using -llustreapi
      OpenMP support:           yes, using -fopenmp
        OpenMP pomp_tpd:        yes
        OpenMP ancestry:        yes
      PDT support:              no
      Cobi support:             no
      Timer support:            yes, providing gettimeofday, clock_gettime(CLOCK_MONOTONIC_RAW), tsc (X86_64)
        Timer LDFLAGS:          -lm
      libbfd found:             yes, using -lbfd and cplus_demangle
      compiler instrumentation: yes, using GCC plug-in with support for compile-time filtering
      Online access support:    yes
      Memory tracking support:  yes
      GCC atomic builtins:      yes
      Compiler wrappers:        scorep-gcc \
                                scorep-g++ \
                                scorep-gfortran \
                                scorep-mpicc \
                                scorep-mpic++ \
                                scorep-mpif77 \
                                scorep-mpif90
      User library wrappers support: \
                                no

    Score-P (GCC plug-in):
      GCC plug-in support:      yes, using the C++ compiler and -I/home/apps/spack/opt/spack/linux-centos7-skylake_avx512/gcc-8.3.0/gcc-11.2.0-fufsigm7go2apn3xzygaw3x266zuxav4/lib/gcc/x86_64-pc-linux-gnu/11.2.0/plugin/include
      Compiler used:            g++

    Score-P (libwrap):
      Library wrapper support:  no, llvm-config not found

    Score-P (MPI backend):
      C99 compiler used:        /home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/mpich-3.4.2-sch55ydbzy3wkihpsjq3xhiwbtxddmlp/bin/mpicc
      C99 compiler works:       yes, using /home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/mpich-3.4.2-sch55ydbzy3wkihpsjq3xhiwbtxddmlp/bin/mpicc
      C++ compiler works:       yes, using /home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/mpich-3.4.2-sch55ydbzy3wkihpsjq3xhiwbtxddmlp/bin/mpic++
      F77 compiler works:       yes, using /home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/mpich-3.4.2-sch55ydbzy3wkihpsjq3xhiwbtxddmlp/bin/mpif77
      F90 compiler works:       yes, using /home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/mpich-3.4.2-sch55ydbzy3wkihpsjq3xhiwbtxddmlp/bin/mpif90
      Library used for MPI:
      PDT MPI instrumentation:  yes, if PDT available

    Score-P (score):
      C compiler used:          gcc
      C++ compiler used:        g++
      cube c++ library support: yes, using -I/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/cubelib-4.6-gypjjtab7uyj376y46zbw5ha2v7ls47y/include/cubelib \
                                -L/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/cubelib-4.6-gypjjtab7uyj376y46zbw5ha2v7ls47y/lib \
                                -Wl,-rpath -Wl,/home/apps/spack/opt/spack/linux-centos7-cascadelake/gcc-11.2.0/cubelib-4.6-gypjjtab7uyj376y46zbw5ha2v7ls47y/lib \
                                -lcube4 -lz -lpthread

[samcom12]

And, my command looks like-

mpirun -np $SLURM_NTASKS -ppn 48 python -W ignore -m scorep --mpp=mpi run_model.py

I'm using this for ANUGA project at

https://github.com/anuga-community/anuga_core/commits/main

[AndreasGocht]

Hey,

/usr/bin/ld: cannot find -lz

I think this is a different Issue. You are missing libz. As you use Spack, I assume you do work on an HPC-System. Please ask your HPC-Admins to provide this lib in the environment you do use with Score-P.

Btw:

python -W ignore -m scorep --mpp=mpi run_model.py

I am not sure if the flag -W ignore is finally respected ( #18 might apply ).

Best,

Andreas