feat(dsl)!: DSL recursion changes w/ bb.js #485
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
recursive verification key no longer encapsulates a reference string (neither were fundamentally needed!)
Added passing serialization tests Fixed format of RecursionConstraint to be compatible with existing ACIR formatting
Can be used by backend to force recursive verification key to be a specific value (it is represented via witnesses and therefore is not by-default constraint to represent a specific circuit)
* merge master in zw/noir-recursion * add dsl files to commit * namespace stuff and debugging * c bind functions * constraint test and comment removal * revert some changes to RecursionConstraint while debugging serialization issues * dispaly error when trying to use create_circuit_with_witness * is_recursive flag as part of new_proof and verify_proof, new RecursiveProver type in dsl, and serialization changes for recursion acir_proofs methods * remove debug output from TestSerializationWithRecursion
…and env_load_verifier_crs
…er generation tests
…ook so not caught earlier, but need more clarity on ubuntu
* merge conflicts and small updates to get acir_proofs test passing with arbitrary depth cahnges * inline with mv/noir-recursion and working double recursion acir_proofs serialization test * cleanup and working towards supporting full nested proofs, still some bugs in acir_proofs test * full recursive composition test working in acir_proofs * use two public inptus for inner circuit * delete commented out unnecessary acir proofs method * made dummy transcript points unique to prevent point collisions * handle nested proofs when exporting dummy transcript, export recursive proof indices when init vkey, fix full recursive comp test in acir_proofs * update bindings on verify_recursive_proof to accept num public inputs * missing acir_format field in tests * missing one more correct acir_format struct in recursion constraint test * cleanup and additional comment for recursion_constraint * fix up comment in recursion_constraint * remove unnecesary comments when we were including proof outputs as public inputs in the recursion constraint --------- Co-authored-by: zac-williamson <blorktronics@gmail.com>
…ying to invert zero in the field for dummy transcript and key
charlielye
added a commit
that referenced
this pull request
Jun 1, 2023
…t. (#471) * Initial * Separate thread build dir. * Large vector slab allocator. * format msgpack serialization and excldue msgpack-c from clang-format (#467) * patch: temporarily remove broken solidity ci (#470) * progress. * Sumcheck improvements (#455) * convert partially evaluated polynomials from vectors to Polynomials and rename * rename fold method to partially_evaluate * static constexpr barycentric arrays * change purported evaluations to claimed evaluations * specify relations in Flavor * working. but cant compute pk independently :/ * Fixed. * Better key construction separation. * Clean * tests pass * Commander * Fix. * Fix * Fix serve. * Fix * More commands. * Fixed a bug in biggroup tests (#478) * Add back custom libc++ build. * Fix. * write_vk * DSL: Add KeccakVar opcode (#476) * add initial KeccakVar code * add result field * add keccak_var_constraints to fields * Multi-constraint Relations (#444) Allow for correct and efficient batching over identities in the Sumcheck relation * Clean up srs package. Add a global crs that the wasm can init once. * Add back vk serialization for a3 * Clean * Exclude env from lib. * feat(dsl)!: DSL recursion changes w/ bb.js (#485) * Added recursion constraint into dsl + tests * fix dsl ecdsa constraint. Added ecdsa dsl tests * changed dsl recursion to pass proof/key as witnesses * recursive verification key no longer encapsulates a native key recursive verification key no longer encapsulates a reference string (neither were fundamentally needed!) * added serialization test for acir_proofs * Added serialization methods into dsl for recursive proof composition Added passing serialization tests Fixed format of RecursionConstraint to be compatible with existing ACIR formatting * added verificaiton key hash into RecursionConstraint Can be used by backend to force recursive verification key to be a specific value (it is represented via witnesses and therefore is not by-default constraint to represent a specific circuit) * fixed compiler errors in stdlib_recursion_tests * feat(dsl)!: Noir recursion updates (#379) * merge master in zw/noir-recursion * add dsl files to commit * namespace stuff and debugging * c bind functions * constraint test and comment removal * revert some changes to RecursionConstraint while debugging serialization issues * dispaly error when trying to use create_circuit_with_witness * is_recursive flag as part of new_proof and verify_proof, new RecursiveProver type in dsl, and serialization changes for recursion acir_proofs methods * remove debug output from TestSerializationWithRecursion * master merge conflicts * EnvReferenceString undefined reference to error workaround * add missing recursion_constraint field from acir_format tests * add recursion_constraints field back in acir_proofs.test inner circuit * fix inner circuit in RecursionConstraint.TestRecursionConstraint * Empty-Commit * add verify_recursive_proof method for simulating recursive verification in the ACVM * add ecc module to env package to fix linking of ennv_load_prover_crs and env_load_verifier_crs * add back reference strinng to stdlib recursion key to pass sol verifier generation tests * remove prints from recursive verifier test * fix dirty free for when serializing vk to fields, was working on macbook so not caught earlier, but need more clarity on ubuntu * fix ecdsa tests after master merge * missing keccak constraints fields in acir format and proofs tests * one more missing keccak constraint * mismatched acir format structs for gcc build * missing block constraints in acir tests * feat(dsl)!: Arbitrary depth recursion (#433) * merge conflicts and small updates to get acir_proofs test passing with arbitrary depth cahnges * inline with mv/noir-recursion and working double recursion acir_proofs serialization test * cleanup and working towards supporting full nested proofs, still some bugs in acir_proofs test * full recursive composition test working in acir_proofs * use two public inptus for inner circuit * delete commented out unnecessary acir proofs method * made dummy transcript points unique to prevent point collisions * handle nested proofs when exporting dummy transcript, export recursive proof indices when init vkey, fix full recursive comp test in acir_proofs * update bindings on verify_recursive_proof to accept num public inputs * missing acir_format field in tests * missing one more correct acir_format struct in recursion constraint test * cleanup and additional comment for recursion_constraint * fix up comment in recursion_constraint * remove unnecesary comments when we were including proof outputs as public inputs in the recursion constraint --------- Co-authored-by: zac-williamson <blorktronics@gmail.com> * move export key into dsl package * chore: remove unused export key in recursion format from main proof system classes * pr review: moved from_witness comment and renamed from_field_pt_vector * moved export transcript in recursion format to DSL package * move order of acir functions * remove ecc bb_module declaration in env package * chore: remove usage of magic numbers when slicing g1::affine_element into barretenberg::fr * introduce NUM_AGGREGATION_ELEMENTS constant and use it through recursion constraint * nit ASSERT(result != -1) in get_public_input_index * remove unused found var * cast -1 * fix up verify_recursive_proof and add a test for it * moved from tempalte for has_valid_witness_assignments to a flag * chore: add comments to AcirProofs.TestVerifyRecursiveProofPass * update prove and verify exports to handle is_recursive * remove install state * remove debug output and use total_circuit_size * remove verify_recursive_proof as we might not need it for simulation * switch how we do is_recursive * remove acir recursive simulation method * changed wasi stubs and removed cout from acir_init_proving_key * removed cout outs getting runtime error for func definition * working recursion format funcs, but strangely getting errors about trying to invert zero in the field for dummy transcript and key * most recent debug output, hitting bad alloc in compute_proving_key_base * delete old comment * bb.js cmd changes * Lint and default fixes. * cleanup info and debug comments * add comment to recursion serialization methods * match package.json on cl/acir_cbinds * don't remove EventEmitter from bb_wasm.ts * delete empty exports file * use process.exit * log * add comment for why CIRCUIT_SIZE is 2**18 * mising keccak var constraints in new acir format tests * Cleanup some serialization. * Cleanup for publishing. README. --------- Co-authored-by: zac-williamson <blorktronics@gmail.com> Co-authored-by: Charlie Lye <karl.lye@gmail.com> --------- Co-authored-by: Maxim Vezenov <mvezenov@gmail.com> Co-authored-by: Maddiaa <47148561+Maddiaa0@users.noreply.github.com> Co-authored-by: ledwards2225 <98505400+ledwards2225@users.noreply.github.com> Co-authored-by: Innokentii Sennovskii <isennovskiy@gmail.com> Co-authored-by: kevaundray <kevtheappdev@gmail.com> Co-authored-by: zac-williamson <blorktronics@gmail.com>
This was referenced Jun 12, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR enables the starting skeleton for recursion in Noir and supersedes PR #383 following the work done in PR #471.
This PR enables full recursive composition in the DSL package and exposes two new methods for fetching recursive circuit information.
proof_as_fieldsandvk_as_fieldsboth return the relevant circuit information in the recursion format needed for recursive verification. There is also a new flagisRecursivethat is part of thecreate_proofandverifymethods. This flag distinguishes whether to use a prover and verifier that has a more recursive friendly hash type for the fiat shamir. We want to still expose a keccak prover in order to allow proof verification in Solidity, however, with recursive verification we need a more snark-friendly hash to keep recursive verification cheap.Checklist:
/markdown/specshave been updated.@briefdescribing the intended functionality.