Simpleperf: Different counters for different clusters?

[hosunhc]

Hi, I was wondering if it is possible to measure different counters on different clusters using Simpleperf stat? For example, the device I'm using is Pixel 6 which has three CPU clusters (A55, A76, and X1) and each core type has a different range of counters that can be measured.

The command, I am using is -a -e r73,r70,r41 --csv --interval 100 --per-core --interval-only-values --use-devfreq-counters but I was wondering if I could set events/counters for different clusters? Originally I thought it wasn't possible, at least on Simpleperf, but the Arm Mobile Studio software seems to allow it so I just wanted to be sure here.

[enh-google]

@yabinc

[ghost]

Oh and another question...
In the case of Pixel 6, the clusters can measure six 32-bit counters and one 64-bit counter for the CPU cycle (0x11), a total of 7 counters (Sources: A55, A76, X1).
But when I run with 7 counters:

simpleperf stat -e r11,r13,r12,r19,r1D,r1B,r8 -a --csv --interval 1000 --per-core --interval-only-values --duration 3 --use-devfreq-counters -o out.csv

it returns an error:

simpleperf W cmd_stat.cpp:944] It seems the number of hardware events are more than the number of
available CPU PMU hardware counters. That will trigger hardware counter
multiplexing. As a result, events are not counted all the time processes
running, and event counts are smaller than what really happen.
Use --print-hw-counter to show available hardware counters.
If on a rooted device, try --use-devfreq-counters to get more counters.

I was wondering why that is? On Arm Mobile Development, it does seem to work with 7 so just wanted to make sure.

[hosunhc]

@yabinc just a reminder, since it's been a while and Simpleperf is usually handled by you

[pirama-arumuga-nainar]

Yabin is OOO but will be back next week. Apologies for not posting about this while you waited.

[hosunhc]

Oh no worries, thanks for the response!

[yabinc]

Sorry for the long delay.
For the first question, the kernel supports setting different events/counters for different clusters. But currently simpleperf doesn't support it in the command line interface. I will extend the interface to make it possible.

For the second question, I agree pixel 6 clusters should support 7 counters (including 1 only for cpu-cycles). But in reality, I feel only 6 counters are available.

When using 6 counters:
oriole:/data/local/tmp # simpleperf stat --use-devfreq-counters --per-core --verbose -a --duration 1 --cpu 0 -e r11,r13,r12,r19,r1D,r1B
Performance counter statistics:

r11(tid -1, cpu 0): count 342598317, time_enabled 1001710528, time running 1001710528, id 303254
r13(tid -1, cpu 0): count 45188124, time_enabled 1002220741, time running 1002220741, id 303255
r12(tid -1, cpu 0): count 17029057, time_enabled 1002275266, time running 1002275266, id 303256
r19(tid -1, cpu 0): count 46920113, time_enabled 1002292396, time running 1002292396, id 303257
r1D(tid -1, cpu 0): count 342626767, time_enabled 1002288164, time running 1002288164, id 303258
r1B(tid -1, cpu 0): count 124866901, time_enabled 1002291866, time running 1002291866, id 303259

cpu count event_name # count / runtime

0 342,598,317 r11 # 342.013 M/sec
0 45,188,124 r13 # 45.088 M/sec
0 17,029,057 r12 # 16.990 M/sec
0 46,920,113 r19 # 46.813 M/sec
0 342,626,767 r1D # 341.845 M/sec
0 124,866,901 r1B # 124.581 M/sec

Total test time: 1.001508 seconds.

The time running and time_enabled are the same for all counters.

But when using 7 counters:
oriole:/data/local/tmp # simpleperf stat --use-devfreq-counters --per-core --verbose -a --duration 1 --cpu 0 -e r11,r13,r12,r19,r1D,r1B,r8
Performance counter statistics:

r11(tid -1, cpu 0): count 35847348, time_enabled 1002033037, time running 858269610, id 303358
r13(tid -1, cpu 0): count 4173289, time_enabled 1002537757, time running 858216793, id 303359
r12(tid -1, cpu 0): count 1724420, time_enabled 1002500200, time running 857149779, id 303360
r19(tid -1, cpu 0): count 5604894, time_enabled 1002456295, time running 859368809, id 303361
r1D(tid -1, cpu 0): count 32428905, time_enabled 1002374224, time running 861053140, id 303362
r1B(tid -1, cpu 0): count 12360521, time_enabled 1002300737, time running 861637404, id 303363
r8(tid -1, cpu 0): count 9888409, time_enabled 1002215369, time running 714787884, id 303364

cpu count event_name # count / runtime

0 35,847,348 r11 # 41.767 M/sec
0 4,173,289 r13 # 4.863 M/sec
0 1,724,420 r12 # 2.012 M/sec
0 5,604,894 r19 # 6.522 M/sec
0 32,428,905 r1D # 37.662 M/sec
0 12,360,521 r1B # 14.345 M/sec
0 9,888,409 r8 # 13.834 M/sec

Total test time: 1.001116 seconds.
simpleperf W cmd_stat.cpp:960] It seems the number of hardware events are more than the number of
available CPU PMU hardware counters. That will trigger hardware counter
multiplexing. As a result, events are not counted all the time processes
running, and event counts are smaller than what really happen.
Use --print-hw-counter to show available hardware counters.
If on a rooted device, try --use-devfreq-counters to get more counters.

The time running are smaller than time_enabled for all counters. It means there are counter multiplexing.
I guess it's because the kernel still reserves one counter when --use-devfreq-counters is used. I will check it out when having time.

[hosunhc]

No worries! I would very much like the function of having different counters for the various clusters. Thank you for taking the time to check this out.

[yabinc]

The support of monitoring different events on different cores was added in https://android-review.git.corp.google.com/c/platform/system/extras/+/2799485. The document was updated in https://android-review.git.corp.google.com/c/platform/system/extras/+/2810673.
You can build simpleperf executable as in https://android.googlesource.com/platform/system/extras/+/master/simpleperf/doc/README.md#bugs-and-contribution.
The executable in https://android.googlesource.com/platform/system/extras/+/master/simpleperf/scripts/bin/android/arm64 will be updated once https://android-review.git.corp.google.com/c/platform/system/extras/+/2811126 is submitted.

[hosunhc]

Since I am not a part of Google corporation, I think that means I cannot check any of the android-review.git.corp.google.com links. But just to be sure, does that mean I can still have access to the executable?

[enh-google]

Since I am not a part of Google corporation, I think that means I cannot check any of the android-review.git.corp.google.com links. But just to be sure, does that mean I can still have access to the executable?

sorry, that's our stupid "security" setup that rewrites URLs internally, making them useless for outsiders. any time you see a URL like "android-review.git.corp.google.com" you need to change it back to "android-review.googlesource.com".

[hosunhc]

Ahhh I see thanks!
I'm trying to compile it right now, but it seems the TARGET_RELEASE config has changed from the original aosp_arm-userdebug to <product>-<release>-<variant>. Do you know which release I should be using? There's next, staging, trunk, trunk_food, and trunk_staging, and I can't seem to find any documentation on which is which

[yabinc]

Any release is fine. Because the simpleperf changes above aren't controlled by any flags.