Skip to content

Simple Linux perf FAQ

Andi Kleen edited this page Apr 3, 2023 · 4 revisions

Common issues with Linux perf (independent of pmu-tools)

debian-fail ubuntu-fail

perf does not work after kernel update on Debian/Ubuntu

Debian and Debian derived distributions use a very broken setup where a shell wrapper looks for a perf binary tied to the specific kernel. Each shipped kernel has a special linux-tools package that contains the perf binary for this kernel. This works as long as only official kernel packages are used (which tend to pull in linux-tools updates), but breaks as soon as the kernel is updated.

(I think they added it because some ancient perf version had a backwards compatibility bug, which was fixed but instead of backporting the fix they added this broken setup, which now breaks kernel updates all the time. Unfortunately all the derivatives inherited this brain damage too)

The wrapper isn't needed at all. The perf tool is generally forward and backwards compatible with different kernel versions.

Solutions:

  1. Copy the newest perf out of /usr/lib/linux-tools/newest kernel version/perf over /usr/bin/perf.

  2. Build a new perf tool from [Linux perf] (or tools/perf in a kernel tree) and replace /usr/bin/perf with that.

This typically requires installing some extra development packages, otherwise the resulting perf will not support ELF symbols or interactive browsing, see the make output. However pmu-tools in counting mode normally works fine even with a bare boned perf without these packages. The extra support is normally only needed if you want to sample.

missing-minus-a

I run "perf stat -e events sleep 1" but get too low numbers?

Add -a. Perf defaults to only measuring the specified process, so you only measured "sleep"

tmp-symbols

I see "perf-....map" or [JIT] tid-... in perf report output

perf requires a special agent to resolve symbols in JITed code, such as in Java or JavaScript or similar VMs, code. For OpenJDK the perf source contains an suitable agent. In general the agent needs to be enabled in the JIT with suitable options, which then enables the JIT to write special symbol tables for perf to tmp.

perf-new-cpu

I want to measure a new CPU's events with perf stat/record and don't see them in "perf list"

Likely your perf version is too old. You can get a new version from [Linux perf] (or tools/perf in a kernel tree), or alternatively use the ocperf wrapper in pmu-tools. Note that some events may also need a kernel update.