Interleaving of code hints with function bodies

[eqrion]

The code section can be really large in modules. In SpiderMonkey, we compile functions in the code section in parallel and kick off the tasks as soon as we get them off the network. So minimizing the information needed before being able to compile the first function is helpful.

If we expect many hints to be provided for the functions, then downloading and decoding the whole branch hint section before getting to the first function in the code section could delay compilation. It could be useful if the branch hints were inside the code section so that we only need to decode the hints for each function immediately before the function is compiled.

That would involve changing the code section, making this not a custom section anymore. I'm not sure if this was discussed previously and I forgot about it.

[yuri91]

This concern has been raised before.

My opinion is that this is not really an issue, because branch hints are quite rare in actual codebases. You can search on github for [[likely]]/[[unlikely]] and __builtin_expect to get an idea.

The situation is different for JIT-compiled code, but in that case the code is not downloaded at all, so it's not a problem.

An intermediate case could be hints generated from PGO.

I don't really know how many branches would be annotated as a result of PGO on an average, but even assuming a worst case scenario, the numbers are not too bad:

I just did some quick math using two wasm modules i have laying around: a 5.5MB one compiled from C++, and a 18MB one compiled from Go.

Counting the number of branch instructions with wasm-opcodecnt, and assuming that each one of them is hinted, I get to a size for the custom section that is around 6-7.5% that of the code section.

A more realistic ratio is probably 1 in 10 hinted branches or even less, so it seems to me that the overhead is quite negligible.

(For my estimation I considered the size of a hint to be 4 bytes, plus 4 bytes for each hinted function)

[titzer]

I generally agree and I'm not too worried about this section's size. I'm not sure branch hints would be useful for baseline compilation, because, at least Wizard's baseline compiler will always emit machine code in bytecode order, since it's single pass. So I see branch hinting as info primarily useful to the optimizing compiler.