Via IR bytecode comparison

[cameel]

Depends on #14354. Merged.
Depends on #14374.
Fixes #13583.

This is the final PR for --via-ir bytecode comparison.

[cameel]

Unfortunately even after all that upfront work to get things working in parallel, it's pretty slow. Optimized via IR preset takes as long as 25 min on windows and on emscripten.

[nikola-matic]

What does no_output_timeout do? Fail the step if the runtime exceeds 30 minutes? If so, then a 5 minute buffer for a job that takes 25 minutes is a little tight, no? Would it make sense to bump this a little bit; maybe 35 minutes?

[cameel]

Yeah. But the default timeout is 10 min so I'm increasing it, not decreasing it :)

I think 30 min is fine. If it fails we can still increase it but we'll get a clear signal that things are getting out of hand. 25 min is actually way more than I expected. I'd prefer this to finish much sooner. If not for the fact that we're working on IR performance already, I'd suggest parallelizing things even more to improve it, rather than keep bumping the limit.

[cameel]

Timing, from the most recent run:

Job	Preset	Legacy	Via IR
b_bytecode_win	optimize	6m 52s	24m 24s
b_bytecode_win	no-optimize	5m 5s	10m 12s
b_bytecode_osx	optimize	3m 56s	10m 42s
b_bytecode_osx	no-optimize	3m 38s	5m 55s
b_bytecode_ems	optimize	5m 31s	26m 47s
b_bytecode_ems	no-optimize	2m 8s	13m 25s
b_bytecode_ubu	optimize	3m 48s	9m 18s
b_bytecode_ubu	no-optimize	3m 2s	5m 9s
b_bytecode_ubu_static	optimize	2m 12s	7m 51s
b_bytecode_ubu_static	no-optimize	1m 13s	3m 35s

[cameel]

Also, we have a problem: t_bytecode_compare does not pass via IR. And it looks like it's due to actual differences in generated bytecode. Weirdly, this happens only for unoptimized code. Optimized is fine.

I will investigate that on Monday.

[cameel]

Current status: blocked by #14361, which I'll be fixing.

If need be, I could add a temporary workaround to delete test cases that do not pass and merge this. I would then remove that workaround in the PR fixing #14361.

[cameel]

With #14374 and one additional tweak the bytecode comparison finally passes.

The tweak is basically that one of the tests triggers a YulException (actually stack too deep, but for some reason it's not StackTooDeep) and CLI just barfs while Standard JSON can still successfully return some outputs like metadata. This is basically a hack to ignore those outputs. The proper way to deal with it would be to fix #13925.

The PR is now reviewable but I'll keep it as a draft until #14374 is merged because PRs with dependencies always confuse people :)

[matheusaaguiar]

LGTM

[cameel]

Base PR was merged. This is now reviewable.

@matheusaaguiar But since the PR is simple and you already reviewed and approved it, we can also just merge if you reapprove.

[cameel]

@ekpyron dropped #13496 from the focus board recently so I guess he does not want use to focus on fixing in this. For that reason it'll stay like this for the time being. I.e. we're not really running the comparison on all test cases from the repo.

[nikola-matic]

What does no_output_timeout do? Fail the step if the runtime exceeds 30 minutes? If so, then a 5 minute buffer for a job that takes 25 minutes is a little tight, no? Would it make sense to bump this a little bit; maybe 35 minutes?