fix: Account for missing indices in flattened witness map

[charlielye]

Description

Fixes the flat witness format to include 32 0's in place of witness index map holes.
Includes all compiled test vectors json output, and flat witness output.

Problem*

Barretenberg wants to run a bunch of acir test vectors through it's bb and bb.js impls.

Documentation

• This PR requires documentation updates when merged.

  • I will submit a noir-lang/docs PR.
  • I will request for and support Dev Rel's help in documenting this PR.

Additional Context

PR Checklist*

• I have tested the changes locally.
• I have formatted the changes with Prettier and/or cargo fmt on default settings.

[TomAFrench]

What's the reasoning about committing bberg flattened witnesses here rather than using the ACVM format and flattening inside the bberg repo like we do for ACIR? We could have a binary similar to acir-to-bberg-circuit quite easily and would reduce some hardcoding between noir and bberg (this test suite could then be reused by other backends).

[TomAFrench]

On second thought there's still some bberg dependency through the black box function support but it's much reduced.

[kevaundray]

On second thought there's still some bberg dependency through the black box function support but it's much reduced.

Yeah I think we can reduce the bberg dependency as we go along, with this change, we can confidently start removing bberg from Noir itself as bberg will be running these compiled tests on master.

Long term, we want these to be uploaded to github storage and then bberg can be changed to download from that instead

[charlielye]

I currently have no idea how to parse the witness format output by Noir.
@kobyhallx thinks flat witness should maybe become the default output.
I'd say unless there is good reason otherwise, the simpler flattened witness is trivial for anyone to parse.
It depends on if these "holes" in the witness data can in normal circumstances become arbitrarily large, but that's easily enough handled with compression.

If the more complicated format is desired its serialization format needs to be documented (or I need to be pointed to it).

IIRC it was also using DEFLATE but in a pretty non standard way. I was unable to trivially get a unix cmd line tool to decompress it. Can we not at least use GzEncoder so its trivially simple to decompress (if compression is needed at all)? 1m witnesses is a 32mb file which doesn't seem too bad.

[kevaundray]

se GzEncoder so its trivially simple to

We can switch to using GzEncoder for compression and do something like:

    let file = File::create("witness.gz")?;
    let mut gz = GzEncoder::new(file, Compression::default());