Cheaper decoding of scripts

[effectfully]

@colll78 reports on Slack:

People don’t really care much for compression onchain. The cost of bytes is so low that developers would much rather pay a large min Ada UTxO cost once and have deserialization be cheap, then to have min Ada small and deserialization cost high.

and on Twitter:

A week or so ago I did some benchmarking of Flat against a Data.Binary serialization implementation I did, I can post the results where ever is useful.

@colll78 we're very much interested in those numbers! Though I'm really surprised to hear that binary might be better than flat given the benchmarks done in haskell-perf clearly pointing out that flat is several times more performant than binary. Maybe the issue is not with the format chosen but merely our own implementation? Or maybe it's the CBOR-encoded Data that is problematic rather than flat? Or the additional CBOR compression done on top of flat.

[effectfully]

I've looked at our Flat code and it isn't efficient. In particular, there's plain recursion instead of recursive join points. I don't think anybody has ever optimized the decoding machinery properly? So we should do that.

[effectfully]

Why are we not caching builtinsAvailableIn? Do we actually recompute it for each script? That sounds wasteful.

Can it explain the puzzling results of #6502 together with the fact that we don't benchmark what we think we benchmark (PR incoming)? Maybe.

[kwxm]

Why are we not caching builtinsAvailableIn? Do we actually recompute it for each script? That sounds wasteful.

We should definitely look into this. How would we test it though? If we replace it with something better it could be tricky to tell what impact it's having in the real world.