Small cpu_comparison "pythonic" improvements

[makslevental]

I was gonna start working on #581, which is downstream of one of the comparisons, and I started using the script and then got distracted.

[newling]

Thanks for the improvements. n_repeats vs num_repeats, not clear what's going on is num-repeats a global default?

Please run black if you haven't already

[makslevental]

Thanks for the improvements. n_repeats vs num_repeats, not clear what's going on is num-repeats a global default?

err I'm blind - I didn't notice n_repeats.

Please run black if you haven't already

I have black bound to ctrl+shift+b and I run it compulsively after every edit lol

[makslevental]

not clear what's going on is num-repeats a global default?

Since I didn't realize n_repeats existed, I patched in a different semantic for num_repeats (repeating only the run). But actually the current/existing semantic you have for n_repeats is the right one for this test. So I'm gonna remove the changes associated with num_repeats.

[newling]

not clear what's going on is num-repeats a global default?

Since I didn't realize n_repeats existed, I patched in a different semantic for num_repeats (repeating only the run). But actually the current/existing semantic you have for n_repeats is the right one for this test. So I'm gonna remove the changes associated with num_repeats.

I was wondering about this actually, repeating the compile does hurt test time quite badly and I'm not sure it's worth it. Ideally we could have the option for both, and if we're repeating the compile it'd be nice if we could actually diff the artifacts (xclbin?) to see if compilation is deterministic. xclbin might contain timestamps or something which this not possible though.

TL;DR: my current thinking is that if we have to choose one or the other, we shouldn't be repeating the compilation, just the run... FFS.

[newling]

n_repeats is still confusing: run_test takes as 2 arguments for the number of repeats, one via config (config.n_repeats) and one via n_repeats.

In this PR, can you please remove all changes to n_repeats, and we can get back to it later? Everything else looks good to me

[makslevental]

@newling all set

[newling]

Thanks!