Frontend is slower than necessary

[jfecher]

Description

Aim

Trying to run nargo test on a reasonably small code sample with 4 tests.

Expected behavior

The frontend should finish checking the code quickly and the backend should take a majority of the time as it goes through each test individually.

Bug

Instead, the frontend takes a relatively consistent ~2 seconds on my machine (Macbook M1 Pro) despite the small source code size. We should investigate the reason(s) for this and fix them since the front end of a compiler should be able to achieve ~100k lines of code per second for a similar feature set to noir (citation needed). It is unclear how much of these 2 seconds are startup time or time that would scale with program size.

[jfecher]

Some preliminary testing reveals the source of the slowness is almost entirely within collect_defs and not the subsequent resolution steps or type checking, nor the steps of adding the stdlib beforehand.

[jfecher]

Here's a flamegraph of nargo prove p running on a short program. You can see that almost the entire middle plateau is from collect_defs. Further inspection then reveals that the majority of the time within collect_defs is actually spent during parsing. A bit above half of this time is spent in expression_with_precedence, and a bit below half seems to be spent in Drop::drop for parser errors. The former can be fixed with a more efficient parser, but the later may be more inherent to chumsky. We may have to wait for an update in chumsky to fix the drop performance impact, or change our parsing library.

[jfecher]

New flamegraph after the optimization in #869:

Now only the middle tower is the parsing-related portion. Our runtime is now dominated by calls to common::crs::read_crs for short programs at least.

[jfecher]

Just for completeness, here is a flamegraph of nargo prove p running on a somewhat larger program (8_integration):

Here our runtime is dominated almost entirely by barretenberg. All the little hills that can be seen are from barretenberg::scalar_multiplication::reduce_buckets. The taller spike on the right is from prove_cmd::run.