linter: make our codspeed benchmarks more representative of the real wold

[camc314]

based on #13690

In this PR, our codspeed benchmarks showed perf improvements of ~50%, however when benmarking on real-world repos, this diff improvement was reduced to ~15%.

The goal here should be to introduce benchmarks that are more representitive of the real world, so we can accurately track the perf improvements in real world examples

[overlookmotel]

I think part of the reason for the difference between CodSpeed and real world is that CodSpeed doesn't measure walltime - its measure is closer to an instruction counter. In particular, CodSpeed doesn't take branch prediction misses into account, which can have a large effect on perf. See: oxc-project/backlog#5

That said, I agree with you that our benchmarks (not just for the linter, but also parser, transformer etc) are not very representative of real world. In particular, it'd be much more realistic to measure processing many smaller files in succession than processing 1 massive file.

As a starting point, perhaps cal.com repo would be a good example (without squashing all the files into 1 big file as we have now).

Now that CodSpeed (finally) has ability to skip some benchmarks, we could set up CI so that e.g. linter benchmarks are skipped on PRs related to transformer or formatter. It may then be viable to have some benchmarks which take longer to run, because they'll run less often.