Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zw/noir recursion 2 #414

Merged
merged 18 commits into from
May 17, 2023
Merged

Zw/noir recursion 2 #414

merged 18 commits into from
May 17, 2023

Conversation

zac-williamson
Copy link
Contributor

@zac-williamson zac-williamson commented May 3, 2023

Description

PR reduces cost of recursive proof composition in UltraPlonk to under 2^18 constraints.

The cost of "maximal" double recursion is 523,506; under the 2^19 threshold. "Maximal" means the circuit verifies 2 UltraPlonk proofs, where both inner proofs are themselves recursive and contain an aggregation object that must be accumulated by the outer circuit. (old cost pre zw/noir-recursion was 738,038)

Additionally, several security flaws have been noticed and fixed.

Security Fixes

  • pedersen_lookup::hash_single . Fixed bug where we were not validating the sum of the decomposed bit slices matched the input scalar.

  • stdlib::recursion::verify_proof . When recovering an aggregation object from public inputs, we now apply range constraints to validate the bigfield limbs have the intended size

  • stdlib::recursion::verify_proof . When extracting group elements from the proof, we were constructing biggroup elements directly from the raw proof data. These witnesses were not being linked to the group elements being hashed by stdlib::recursion::transcript . This is now fixed.

  • stdlib::recursion::transcript . Previously when splitting field elements into 2 slices to better format the hash preimage buffer, we were not validating that the sum of the slices was less than the field modulus (over the integers). This meant the hash preimage was not unique per proof. This is now fixed.

Changes

  • UltraComposer::decompose_into_default_range does not create extra range tables to handle "leftover" bits if the range table size is large (e.g. a 27-bit range check will not be decomposed into a 14-bit and a 13-bit check. Instead, 2 14-bit checks will be used in combination with an arithmetic gate to prevent overflow).

  • pedersen_lookup::merkle_damgard_compress . The "size" parameter of the Merkle-Damgard construction has been moved to the start of the hash block (not the end). This removes one call to hash_single in the case where the IV is constant.

  • UltraComposer evaluate_non_native_field_multiplication is no longer cached. evaluate_partial_non_native_field_multiplication is cached. Saves ~6,000+ gates

  • pedersen_lookup::compress_with_relaxed_range_constraints . New method that does not apply extra range checks when calling pedersen_lookup::hash_single. Used when compressing the recursive transcript and the verification key. The extra range checks are required to ensure the decomposed bit slices that represent the input scalars sum to a value that is < the field modulus (over the integers). We know these values are 31 bytes from previous range constraints and therefore this cannot happen.

  • pedersen_lookup::hash_single . Reduced constraints required when performing the above mentioned range check

  • bigfield::operator- . Fixed bug where we were not using the UltraPlonk code path to save an addition gate. Fixed bug where UltraPlonk code path computed the wrong output.

  • biggroup::validate_on_curve uses mult_madd to save on 1 field reduction

  • big group::create_group_element_rom_tables / read_group_element_rom_tables now tracks the maximum value of bigfield limbs added into / read from the table (tracking is worst-case. When reading a limb, its maximum value is the max of every limb in the table).

  • biggroup::create_group_element_rom_tables will generate size-6 lookup tables if doing so reduces the total number of tables required

  • biggroup::add_sub new method, computes A + B and A - B. Used in biggroup::create_group_element_rom_tables. Prevents a redundant bigfield addition, subtraction and assert_is_not_equal call.

  • biggroup::quadruple_and_add will perform 2 doublings and add a vector of points into the result. Used in biggroup_batch_mul to save 3 field reductions per iteration

  • biggroup::multiple_mongtomery_ladder replaces several montgomery_ladder implementations (now deleted). Will perform repeated iterations of the Montgomery ladder (A + B + A). Each successive iteration saves on 1 field reduction at the cost of increased additions.

  • biggroup_nafs replaced calls to field_t::create_range_constraint with direct calls to the Composer's method. For UltraPlonk this will prevent an unnecessary addition gate being constructed for each bit.

  • biggroup_tables : removed unnecessary reduce() calls whenever a negated point was written into the table

  • stdlib::recursion::aggregation_state no longer calls normalize on the aggregation object, instead calling reduce (the top-level verifier can validate the final outputs are in the correct field

  • stdlib::recursion::aggregation_state only calls normalize on an aggregation object being added into public inputs

  • stdlib::recursion::verification_key::compress now optimally packs inputs into a minimal number of field elements prior to calling pedersen::compress. Saves >10,000 constraints (previously we were splitting each group element into 16 68-bit limbs and hashing them as individual field elements)

Checklist:

  • I have reviewed my diff in github, line by line.
  • Every change is related to the PR description.
  • I have linked this pull request to the issue(s) that it resolves.
  • There are no unexpected formatting changes, superfluous debug logs, or commented-out code.
  • There are no circuit changes, OR specifications in /markdown/specs have been updated.
  • There are no circuit changes, OR a cryptographer has been assigned for review.
  • I've updated any terraform that needs updating (e.g. environment variables) for deployment.
  • The branch has been rebased against the head of its merge target.
  • I'm happy for the PR to be merged at the reviewer's next convenience.
  • New functions, classes, etc. have been documented according to the doxygen comment format. Classes and structs must have @brief describing the intended functionality.
  • If existing code has been modified, such documentation has been added or updated.

simplified hash input structure when hashing transcripts

cached partial non native field multiplications

reverted how native transcript computes hash buffers

pedersen_plookup can be configured to skip the hash_single range check under limited conditions

fixed the range check in pedersen_plookup::hash_single

pedersen_plookup::hash_single now validates the low and high scalar slice values match the  original scalar

bigfield::operator- now correctly uses the UltraPlonk code path if able to

added biggroup::multiple_montgomery_ladder to reduce required field multiplications

added biggroup::quadruple_and_add to reduce required field multiplications

biggroup_nafs now directly calls the Composer range constraint methods to avoid creating redundant arithmetic gates when using the PlookupComposer

biggroup plookup ROM tables now track the maximum size of any field element recovered from the table (i.e. the maximum of the input maximum sizes)

biggroup batch tables prefer to create size-6 lookup tables if doing so reduces the number of individual tables required for a given MSM

recursion::transcript no longer performs redundant range constraints when adding buffer elements
recursion::transcript correctly checks that, when slicing field elements , the slice values are correct over the integers (i.e. slice_sum != original + p)

recursion::verification_key now optimally packs key data into minimum required number of field elements before hashing

recursion::verifier proof and key data is now correctly extracted from the transcript/key instead of being generated directly as witnesses.

cleaned up code + comments

code tidy, added more comments

cleaned up how aggregation object handles public inputs

native verification_key::compress matches circuit output

fixed compile errors + failing tests

compiler error

join_split.test.cpp passing

Note: not changing any upstream .js verification keys. I don't think we need to as bberg is now decoupled from aztec connect
@zac-williamson zac-williamson force-pushed the zw/noir-recursion-2 branch from 675a7ee to 87d2da3 Compare May 4, 2023 09:37
barretenberg::fr compressed_key;
if (proof_system::ComposerType(composer_type) == proof_system::ComposerType::PLOOKUP) {
compressed_key = crypto::pedersen_commitment::lookup::compress_native(preimage_data, hash_index);
if (proof_system::ComposerType(composer_type) == ComposerType::PLOOKUP) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is hash_index removed? I think we had added it so as to not use the default generators (hash_index = 0) since we'd end up using overlapping generators for Merkle hashing and VK. Is that okay?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the methods to compress uint8_t vectors did not take in a hash index. Will fix the proper way but adding this support

Copy link
Collaborator

@codygunton codygunton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked over what's in stdlib/recursion. As discussed elsewhere, I'm not doing a thorough review here. This is is a preliminary PR introducing complex circuit optimizations, and, as such, recursion in Barretenberg should not be put into production anywhere before a deep audit is conducted (note this is not the code that was put into production as part of Aztec Connect).

rhs = rhs + PI_Z;
rhs = (-rhs).normalize();

std::vector<uint32_t> proof_witness_indices{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is moved to a member function on the aggregation_state.

ASSERT(proof_witness_indices.size() > 0);
ASSERT(proof_witness_indices.size() == 0);

P0 = P0.reduce();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like these reductions were missing before ✔️


field_pt compress(const size_t hash_index)
{
// we can only use relaxed range checks in pedersen::compress iff bits_per_element < modulus bits
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not verifying whether this condition is sharp in the current review.

}
}

std::vector<field_pt> work_element;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be helpful to explain what these data members are.

slice_element(element, num_bits);
}

void slice_element(const field_pt& element, const size_t num_bits)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a refactor of the lambda function split that was defined in apply_fiat_shamir. That function used to split into bytes whereas this one splits into bits.

field_pt working_element(context);

size_t byte_counter = 0;
PedersenPreimageBuilder<Composer, 31 * 8> preimage_buffer(context);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Magic number 31 * 8 = 248? What's the significance of this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

31 bytes = largest number of bytes we can fit into a field element. Will update w. named constant

for (const auto& [tag, value] : input_key->commitments) {
key->commitments.insert({ tag, Curve::g1_ct::from_witness(ctx, value) });
// We do not perform on_curve() circuit checks when constructing the Curve::g1_ct element.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're saying this hash check would be done in a circuit? Maybe I misunderstand, but it seems like even if the circuit creator (who constructed the input_key) was honest, we're still relying on witnesses being constructed in the default way. Anyway, you make it clear in your comment that there's ambiguity around the security model, something that we can address in the audit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's ok because the Prover does not have the ability to influence the witness. We can assume that some higher-level logic is going to check the correctness of the verification key hash (e.g. it comes from a valid leaf in a Merkle tree). If circuit creator points are on curve and the Prover-supplied witness is not, the vkey hash won't ma

domain.generator.create_range_constraint(16, "domain.generator");
domain.domain.create_range_constraint(32, "domain.generator");
num_public_inputs.create_range_constraint(32, "num_public_inputs");
preimage_buffer.add_element_with_existing_range_constraint(composer_type, 8);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this with_existing interface.

@@ -320,8 +307,7 @@ aggregation_state<Curve> verify_proof(typename Curve::Composer* context,
opening_elements.push_back(previous_output.P0);
opening_scalars.push_back(random_separator);

rhs_elements.push_back(
(-(previous_output.P1)).reduce()); // TODO: use .normalize() instead? (As per defi bridge project)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reduce call is now in add_proof_outputs_as_public_inputs

@@ -369,7 +359,7 @@ aggregation_state<Curve> verify_proof(typename Curve::Composer* context,
opening_elements.push_back(g1_ct(x0, y0));
opening_scalars.push_back(recursion_separator_challenge);

rhs_elements.push_back((-g1_ct(x1, y1)).normalize());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not immediately clear why this isn't needed, similarly for opening_result below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The multiscalar multiplication algorithm can handle unreduced points. Similar to how stdlib methods that use field_ct do not need to call normalize constantly (...anymore)

Copy link
Contributor

@Rumata888 Rumata888 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At a glance everything seems ok, except for the soundness issue in range constraint

.c_scaling = 1,
.const_scaling = -barretenberg::fr(last_limb_range),
});
create_new_range_constraint(diff_idx, sublimb_mask);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to range constrain the limb itself,too. Otherwise it is possible to underflow.

return std::array<uint32_t, 2>{ lo_0_idx, hi_1_idx };
}
/**
* @brief The following tests FAILED:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are they still failing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch. No. That was debugging info from when tests were failing. have removed.

lambda1_left = previous_y.mul_left;
lambda1_right = previous_y.mul_right;
lambda1_add = previous_y.add;
// std::copy(previous_y.mul_left.begin(), previous_y.mul_left.end(), std::back_inserter(lambda1_left));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need these commented out lines?

Copy link
Contributor

@suyash67 suyash67 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pedersen stuff looks okay to me, just left one comment about hash index.

zac-williamson and others added 5 commits May 15, 2023 12:14
fixed composer bug where `decompose_into_default_range` was sometimes not range-constraining last limb

removed commented-out code

added more descriptive comments to PedersenPreimageBuilder
Copy link
Contributor

@Rumata888 Rumata888 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bug is fixed

@zac-williamson zac-williamson merged commit 5425693 into master May 17, 2023
@zac-williamson zac-williamson deleted the zw/noir-recursion-2 branch May 17, 2023 17:47
ludamad added a commit to ludamad/barretenberg that referenced this pull request May 29, 2023
* Added dynamic array abstraction into standard library (AztecProtocol#112)

Implements RAM/ROM stuff and dynamic arrays as well as separated all fixed_base operation in standard plonk into a separate file, so that it is no longer part of composer

* fix: Store lagrange forms of selector polys w/ Ultra (AztecProtocol#255)

* store lagrange forms of selector polynomials when serializing pk for Ultra

* added comment to ultra_selector_properties

* feat(ts): allow passing srs via env functions (AztecProtocol#260)

* feat(ts): switch to node-modules linker

* feat(ts): add new env for SRS objects

* feat(ts): test srs bindings

* fix: proper uint8_t include

* feat(ts): revert unneeded changes

* feat(ts): revert unneeded changes

* feat(ts): unify writeMemory arg order

* Update barretenberg_wasm.ts

* feat(ts): fix srs comments

* Update data_store.hpp

---------

Co-authored-by: Adam Domurad <adam@aztecprotocol.com>

* Lde/transcript (AztecProtocol#248)

* adding adrians new transcript classes
* tests added for transcript and new manifest concept

---------

Co-authored-by: codygunton <codygunton@gmail.com>

* fix(build): git add -f .yalc (AztecProtocol#265)

* feat(ts): switch to node-modules linker

* feat(ts): add new env for SRS objects

* feat(ts): test srs bindings

* fix: proper uint8_t include

* feat(ts): revert unneeded changes

* feat(ts): revert unneeded changes

* feat(ts): unify writeMemory arg order

* Update barretenberg_wasm.ts

* feat(ts): fix srs comments

* Fix deps

* Fix comments

* fix(build): git add -f .yalc

* Merge

---------

Co-authored-by: Adam Domurad <adam@aztecprotocol.com>

* chore: modularize bb (AztecProtocol#271)

* chore: modularize ts

* chore: reformat

* Adding foundation to bb.js (AztecProtocol#274)

* cleaning up bb.js deps

* update bb structure to use workspaces

* remove foundation .yarn/cache

* chore: don't bundle .yalc

* Update readme

* chore: modularize bb (AztecProtocol#271)

* chore: modularize ts

* chore: reformat

* merge

* remove yalc

* Unbundle tsbuildinfo

---------

Co-authored-by: ludamad <domuradical@gmail.com>
Co-authored-by: ludamad <adam.domurad@gmail.com>

* Fix build of ts package (AztecProtocol#276)

* Splitting turbo composer (AztecProtocol#266)

* Turbo Circuit Constructor working

* Turbo!! And also fixed some of the fuzzer compilation issues

* Luke: Addressing my own comments and adding minor TODOs where necessary

---------

Co-authored-by: ledwards2225 <l.edwards.d@gmail.com>

* Cg/move to shared (AztecProtocol#294)

* Move circuit constructors to shared.

* Move helper lib and perm helper.

* Move tmp composers and helpers for plonk.

* Fix namespace and red herring comment.

* Remove pointless namespace declaration.

* Fix more namespaces.

* Split flavor

* Rename tests to avoid ambiguity.

* Remove redundant macro defs.

* Fix comment formatting.

* StandardArithmetization is not shared with plonk.

* Lde/split gemini (AztecProtocol#256)

* adding adrians new transcript classes

* building with some failing tests

* tests passing

* tests added for transcript and new manifest concept

* improvements to the manifest concept

* prover now operating on split gemini fuctionality

* make shplonk test independent of Gemini

* gemini and kzg tests updated; reduce prove removed from gemini

* general cleanup

* woops, fix gcc build

* minor rebase fix

* make gemini method return fold polys per Adrians suggestion

* fix bad move

* Lde/lookup grand product (AztecProtocol#286)

* moving perm grand product to prover lib and fleshing out lookup grand product

* cleaning up perm grand product test

* lookup grand product test in place

* cleaning up lookup grand prod test and adding sorted list accum method and test

* rename prover tests to prover library tests

* general cleanup

* improve naming for gamma and beta constants

* rabse fix

* Cg/arithmetization (AztecProtocol#296)

* Move gate data to better location.
* Add basic arithmetization class.
* CircuitConstructor takes Arithmetization.
* Remove FooSelector enums from split composers.

* feat: Working UltraPlonk for Noir (AztecProtocol#299)

* Make dsl composer agnostic.

* change SYSTEM_COMPOSER under stdlib::types to ultra composer type

* use ultra logic constraints

* in process of debugging, move to using ultra logic constraints

* add get_total_circuit_size method

* acir format tests showing failures with range constraints of different bit sizes

* remove unnecessary comment

* (fix) Temporarily add a redundant add-gate for variables that need range constraint < 8 bits.

* rename functions

* Implement get_solidity_verifier function

* Fix no longer available properties

* remove constraint system

* logic gate changes using plookup

* logic gate debugging

* test for logic gates passing

* last debug things XOR and AND returnign correct results, XOR still failing

* cleanup

* pedersen_plookup

* plookup funcs

* add to header

* fixed error in pedersen hash when RHS is a circuit constant

* added ACIR test for XOR gate

pedersen hash test now checks y coordinate

* temp disable wasm-opt

* Making everything compile with any composer & add a cmake flag to switch on turbo

* enable wasm-opt for asyncify but disable optimizations

* remove using in header

* fixed work queue bug with wasm

wasm code path was not correctly storing fft outputs in proving key

* added bitwise logic operations into stdlib

stdlib method is utility method to provide Composer-agnostic interface due to the use of plookup tables if enabled

* updated acir_format to use new stdlib logic class

Updated ReadMe to include wasm example that supports gtest filtering

* reenable tests

* linting fixes

* disable binaryen with comment

* write instead of read

* remove random

* WIP

* cleanup the debug logging

* restore the randomness

* only add a zero/one test instead of replacing

* remove unused change

* changes to make solgen work correctly in bindings

* fix join_split_tests.test_deposit_construct_proof

* working serialized proving key size and circuit change test for ultra (AztecProtocol#307)

* USE_TURBO for join_split

* Empty-Commit

* Don't default one function; tweak comments.

* Empty-Commit

---------

Co-authored-by: Suyash Bagad <suyashnbagad1997@gmail.com>
Co-authored-by: vezenovm <mvezenov@gmail.com>
Co-authored-by: Maxim Vezenov <maximvezenov@Maxims-MacBook-Pro.local>
Co-authored-by: zac-williamson <blorktronics@gmail.com>
Co-authored-by: kevaundray <kevtheappdev@gmail.com>
Co-authored-by: codygunton <codygunton@gmail.com>

* Add debugging CMake preset & update code-workspace (AztecProtocol#308)

* Add debugging CMake preset & update code-workspace

---------

Co-authored-by: Blaine Bublitz <blaine.bublitz@gmail.com>

* Lde/ultra composer (AztecProtocol#302)

* duplicate ultra composer with tests passing

* instantiating a circuit constructor in composer but not using it yet

* directory updates after rebase plus finalize circuit function added

* WiP almost have composer helper proving key computation building

* WiP still debugging linker error

* linker issue seemingly resolved

* create prover building and running with new composer

* proving key polys match old composer for simple circuit

* circuit with no lookups is verifying

* all composer tests passing with split ultra composer

* kill poly store debug code

* cleanup

* fix arithmetization rebase issues

* WiP new test

* fix bad circuit size bug

* cleanup

* fix(nix): Use wasi-sdk 12 to provide barretenberg-wasm in overlay (AztecProtocol#315)

* fix(nix): Use wasi-sdk 12 to provide barretenberg-wasm in overlay

* chore: Remove the wasm stuff from main package

* chore(nix): Switch the default llvm to 11

* chore(nix): Add transcript00 to the overlay

chore(nix): Cleanup for nix flake check

* Use hash for each platform

* avoid symlinks

* try wasi-sdk that someone wrote on github

* fix hash for linux

* try to ignore libstdc++

* need the whole name

* try to include std lib instead of ignore

* cleanup and nix flake check

* chore(ci): Check the nix flake in CI

* run default build instead of llvm12

* Prep: move composer type, proving key and verification key. (AztecProtocol#303)

* Move composer type from plonk to bonk.
* Move pk & vk into plonk.
* bonk ~>  proof_system; nest plonk and honk in it.
* proof_system independent of plonk.

* fix(dsl): Use info instead of std::cout to log (AztecProtocol#323)

* fix(dsl): Use info instead of std::cout to log

* Empty-Commit

---------

Co-authored-by: Maxim Vezenov <mvezenov@gmail.com>

* fix(nix): Disable ASM & ADX when building in Nix (AztecProtocol#327)

* fix(nix): Disable ASM & ADX when building in Nix

* Empty-Commit

---------

Co-authored-by: kevaundray <kevtheappdev@gmail.com>

* Aztec3 Specific Work in Barretenberg (AztecProtocol#142)

* Split Pedersen Hash & Commitment Gadgets (AztecProtocol#95)

* [SQUASHED] Pedersen refactor into hash and commitment.

Use lookup pedersen for merkle tree, fixed-base pedersen for commitments.
---------
Co-authored-by: Suyash Bagad <suyashnbagad1997@gmail.com>

Port `copy_as_new_witness`.

Port `must_imply`.

`operator++`.

Port changes from `common`.

Port `ecc/groups`.

* [CPM] add missing dependencies to libbarretenberg.a (AztecProtocol#154)
---------

* Increase Pedersen Generator indices and subindices. (AztecProtocol#169)

* Remove a3 specific types. (AztecProtocol#252)

* Address Luke's Comments on `aztec3 -> master` (AztecProtocol#263)

* Add must_imply tests.

* Added a test for `field_t::copy_as_new_witness`

* add test for `conditional_assign`

* Added `infinity` test.

* Add `add_affine_test`.

* Tests for Array Object in `stdlib` (AztecProtocol#262)

* basic array tests.

* Add `composer_type` while hashing/compressing a vkey.

* Add `contains_recursive_proof` to Recursive VK (AztecProtocol#268)

* feat: debug utility for serialization (AztecProtocol#290)

* feat: enable asan config

* `array_push` for Generic Type (AztecProtocol#291)

* Add Indexed Merkle Tree  (AztecProtocol#281)

* remove ts (consulted with Adam and we're good to go). (AztecProtocol#292)

* Add cout for verification_key struct (AztecProtocol#295)

* compute tree (AztecProtocol#298)

* [SQUASHED] fixing `push_array_to_array` method. (AztecProtocol#304)

* feat(memory_tree|a3): add sibling path calculations (AztecProtocol#301)

* feat(memory_tree): frontier paths

* fix array and resolve merge conflicts (AztecProtocol#305)

* Mc/hash vk (AztecProtocol#306)

* Increase number of sub-generators to 128.

* Build a3crypto.wasm (AztecProtocol#311)

* More Tests on A3 `stdlib` methods (AztecProtocol#316)

* test: more vk tests to compare circuit/native/vk_data (AztecProtocol#310)

* Mc/hash vk (AztecProtocol#306)

* inc num_generators_per_hash_index to 128. (AztecProtocol#309)

* fix. (AztecProtocol#318)

* Added test for `compute_tree_native`. (AztecProtocol#319)

* Install instructions for apt on ubuntu (AztecProtocol#312)

* Fix address compilation. (AztecProtocol#329)

---------

Co-authored-by: David Banks <47112877+dbanks12@users.noreply.github.com>
Co-authored-by: Michael Connor <mike@aztecprotocol.com>
Co-authored-by: dbanks12 <david@aztecprotocol.com>
Co-authored-by: Santiago Palladino <spalladino@gmail.com>
Co-authored-by: ludamad <adam.domurad@gmail.com>
Co-authored-by: Maddiaa <47148561+cheethas@users.noreply.github.com>
Co-authored-by: Santiago Palladino <santiago@aztecprotocol.com>
Co-authored-by: ludamad <domuradical@gmail.com>
Co-authored-by: cheethas <urmasurda@gmail.com>

* Split shplonk in prep for work queue (AztecProtocol#321)

* Consolidate permutation mapping computation into one method (AztecProtocol#330)

* Lde/reinstate work queue (AztecProtocol#324)

* make MSM size in work queue more flexible

* new work queue hooked up everywhere excluding shplonk

* improve interface and remove commitment key from prover

* move old work queue to plonk namespace

* fix(cmake): Remove leveldb dependency that was accidentally re-added (AztecProtocol#335)

* fix(cmake): Remove leveldb dep d that was accidentally re-added

* Empty-Commit

---------

Co-authored-by: kevaundray <kevtheappdev@gmail.com>

* change to get_num_gates() inside get_total_circuit_size() (AztecProtocol#332)

* Mm/ensure all stdlib_primitives_tests are run using all four composers

* UltraHonk Composer (Split) (AztecProtocol#339)

* add split UltraHonk composer and checks for consistency with UltraPlonk

* adding issue number to some TODOs

* fix: Revert generator changes that cause memory OOB access (AztecProtocol#338)

* fix: Revert generator changes that cause memory OOB access

* Empty-Commit

* Fix cci (temporarily).

* comment out one more test.

---------

Co-authored-by: kevaundray <kevtheappdev@gmail.com>
Co-authored-by: Suyash Bagad <suyashnbagad1997@gmail.com>

* doc: Document more thoroughly why fields don't 0-init (AztecProtocol#349)

* Update field.hpp

* Update field.hpp

* Update field.hpp

* 32-Byte Keccak256 challenges for UltraPlonK (AztecProtocol#350)

* Add WithKeccak variants.

* Update SYSTEM_COMPOSER dependents.

* Ultra Honk arithmetic and grand product relations (AztecProtocol#351)

* add width 4 perm grand prod construction relation

* make grand prod construction use id polys, relation correctness passing

* reorganize relation testing suites

* primary ultra arithmetic relation with passing tests

* secondary arith relation and grand prod init relation plus tests

* add modified consistency check for selectors (AztecProtocol#354)

* No `SYSTEM_COMPOSER` (AztecProtocol#352)

* Get rid of system composer.
* Remove USE_TURBO

* Lde/lookup grand prod relation (AztecProtocol#359)

* lookup grand product relation tests passing

* ignore final entry in table polys for consistency chec, same reason as for selectors

* adding eta and lookup gp delta to relation params

* incorporate lookup gp init relation into tests

* correcting the degree of lookup relation

* fixed bug where range constraining connected witnesses threw an error (AztecProtocol#369)

* fixed bug where range constraining connected witnesses threw an error
* can now apply multiple overlapping ranges to the same witness (or copies of the witness)

---------

Co-authored-by: codygunton <codygunton@gmail.com>

* Add Mutex to global initialisation of generators (AztecProtocol#371)

* Mutex lock initialization call

* add comment on mutex

---------

Co-authored-by: kevaundray <kevtheappdev@gmail.com>

* chore: Add cachix action for uploading binary cache artifacts (AztecProtocol#373)

* chore: Add cachix action for uploading binary cache artifacts

* Only run the nix action on master

* run on my branch and remove todos

* Remove running on my branch

* Zw/recursion constraint reduction (AztecProtocol#377)

* removed blake3s hash from ultraplonk recursive prover
* UltraComposer will now not create duplicate non-native field multiplication constraints
* Propagate new stuff to Honk and splitting_tmp.
* Clean up and add comments.
---------

Co-authored-by: codygunton <codygunton@gmail.com>

* add constraints to ensure that the result of the logic constraint is
indeed produced from the left and right operands of the functions and
improve testings.

* fixed error where applying copy constraints to range-constrained indices would create an unbalanced set membership check

* improve documentation and fix issue in non-ultra variants of the
composer

* [SQUASHED] ecdsa key recovery with test(s). (AztecProtocol#346)

init commit.

Recover pubkey works!

Make gcc happy.

Make gcc happy (again)

gcc fix.

don't use y = 0 as error condition. instead use (x = 0, y = 0) as failure return.

* feat(nullifier_tree): make empty nullifier tree leaves hash be 0 (AztecProtocol#360) (AztecProtocol#382)

* feat(nullifier_tree): make empty nullifier tree leaves be 0

* fix: add append zero behaviour and test

* fix: explicit type name

* clean: update class semantics

---------

Co-authored-by: Maddiaa <47148561+cheethas@users.noreply.github.com>
Co-authored-by: cheethas <urmasurda@gmail.com>
Co-authored-by: cheethas <addaboy@gmail.com>

* Solidity Ultra Verifier with tests (AztecProtocol#363)

* initial setup

* feat: add test setup

* fix: revert unneeded cmakelist changes + typos

* fix: solidity helper binaries in docker

* chore: Use `-assert` variant instead of plain

* chore: config.yml missing whitespace

* fix: alpine base dockerfile

* chore: add sol to build_manifest

* Dockerfile: add string utils

* fix: use different base, add bash

* chore: remove stale misc

* circle-ci fiddling

* more circle fiddling

* fiddle

* Circle-ci fiddling for verifiers (AztecProtocol#365)

* build_manifest update

* fiddling

* fiddling

* skip ensure_repo

* skipping more stuff

* fiddling

* get docker version

* force change in key_gen

* fiddle

* docker version in cond_spot

* fiddle

* update path

* fiddle

* build in init

* fiddle

* fiddle

* fiddle

* fiddle

* alpine

* package naming

* add apt-repository

* add apt rep key

* lld-15

* fiddle with docker

* chore: cleanups

* chore: remove log

---------

Co-authored-by: cheethas <addaboy@gmail.com>

* fix: throw -> throw_or_abort in sol gen (AztecProtocol#388)

* fix: throw -> throw_or_abort in sol gen

* toggle nix build

* fix: toggle nix build

* chore: revert toggle

---------

Co-authored-by: cheethas <addaboy@gmail.com>

* feat!: replace `MerkleMembershipConstraint` with`ComputeMerkleRootConstraint` (AztecProtocol#385)

* feat: replace `MerkleMembershipConstraint` with`ComputeMerkleRootConstraint`

* Update acir_format.cpp

* Ultraplonk check_circuit (AztecProtocol#366)

* Add check_circuit with mid-construction introspection

* updating IPA to use transcript (AztecProtocol#367)

* hack: introduce BARRETENBERG_CRYPTO_GENERATOR_PARAMETERS_HACK (AztecProtocol#409)

* hack: introduce BARRETENBERG_CRYPTO_GENERATOR_PARAMETERS_HACK

* doc: concise

* Update generator_data.cpp

* Update generator_data.cpp

* Add Keccak constraints to acir_format (AztecProtocol#393)

* exp: fix alpine versioning (AztecProtocol#415)

Co-authored-by: cheethas <addaboy@gmail.com>

* Small change that was left out of construct_addition_chains fix (AztecProtocol#404)

Fixes the intermittent construct_addition_chains bugs (the previous fix was incomplete) and cleans the test up a bit

* Pending bb work for aztec3 (AztecProtocol#368)

* update js vk (because we now use UP for merkle hashing)

* Helpers for ECDSA in A3 (AztecProtocol#364)

* Add `stdlib_keccak` in cmake.

Correct an assertion in `to_byte_array` in bigfield.

* Add `random_element` to affine element.

* negate y conditionally.

* Change pedersen hash c_bind to use `pedersen_hash::lookup`.

* c_binds and other ECDSA related fixes (AztecProtocol#407)

* Add v to stdlib ecdsa.

* create an engine if its empty.

* Add ecdsa c_bind.

* print v as a uint32.

* Add secp256k1 cbind.

add c_bind.hpp

Change hpp to h.

remove hpp.

* Add ecdsa in cmakelists.

remove stdlib_ecdsa from build.

* chore: align BARRETENBERG_CRYPTO_GENERATOR_PARAMETERS_HACK usage (AztecProtocol#411)

* Update join_split test

* Tweaks to comments

* Add comment for the assertion in bigfield.

* Expanded on ecdsa comment.

---------

Co-authored-by: ludamad <adam.domurad@gmail.com>
Co-authored-by: ludamad <domuradical@gmail.com>
Co-authored-by: codygunton <codygunton@gmail.com>

* feat: CI to test aztec circuits with current commit of bberg (AztecProtocol#418)

* More generators for aztec3.

* update js vk (because we now use UP for merkle hashing)

* Helpers for ECDSA in A3 (AztecProtocol#364)

* Add `stdlib_keccak` in cmake.

Correct an assertion in `to_byte_array` in bigfield.

* Add `random_element` to affine element.

* negate y conditionally.

* Change pedersen hash c_bind to use `pedersen_hash::lookup`.

* c_binds and other ECDSA related fixes (AztecProtocol#407)

* Add v to stdlib ecdsa.

* create an engine if its empty.

* Add ecdsa c_bind.

* print v as a uint32.

* Add secp256k1 cbind.

add c_bind.hpp

Change hpp to h.

remove hpp.

* Add ecdsa in cmakelists.

remove stdlib_ecdsa from build.

* hack: (aztec3) introduce barretenberg crypto generator parameters hack (AztecProtocol#408)

* hack: introduce BARRETENBERG_CRYPTO_GENERATOR_PARAMETERS_HACK

* doc: concise

* chore: align BARRETENBERG_CRYPTO_GENERATOR_PARAMETERS_HACK usage

* CI to test aztec circuits with current commit of bberg

* build manifest

* chore: align BARRETENBERG_CRYPTO_GENERATOR_PARAMETERS_HACK usage (AztecProtocol#411)

* try other branch of aztec packages

* ci rename script

* Update join_split test

* bump aztec version and merge in aztec3-temporary fixes

* aztec commit switched to branch

* bump aztec commit and document

* typo README.md

* Update README.md

---------

Co-authored-by: Suyash Bagad <suyashnbagad1997@gmail.com>
Co-authored-by: Suyash Bagad <suyash@aztecprotocol.com>
Co-authored-by: ludamad <adam.domurad@gmail.com>
Co-authored-by: ludamad <domuradical@gmail.com>

* chore: use build-system submodule (AztecProtocol#419)

* attempt to remove old CI files and replace with build-system submodule

* project and version files for CCI

* slack orb context

* slack bberg channel (AztecProtocol#422)

* Cg/flavor (AztecProtocol#326)

- Introducing the flavor classes (mainly honk, splash of plonk)
---------

Co-authored-by: ledwards2225 <l.edwards.d@gmail.com>

* Add external benchmarks (AztecProtocol#401)

Adds an external_bench file with benchmarks we use for external benchmarking projects

* Reduce occurence of using namespace syntax in header files (AztecProtocol#387)

Co-authored-by: maramihali <mara@aztecprotocol.com>

* ensure all operand sizes are tested (AztecProtocol#432)

Co-authored-by: maramihali <mara@aztecprotocol.com>

* Add ECDSA test for ACIR and fix (AztecProtocol#435)


---------

Co-authored-by: zac-williamson <blorktronics@gmail.com>

* chore(ci): Add Noir CI that runs with a github label (AztecProtocol#430)

* Adds prehashed message variant of EcDSA (AztecProtocol#437)

* verification takes a pre-hashed message : Note: if len(hash) > 32 bytes, then bigfield will fail

* use hashed_message when generating signature

* modify acir structure and function to now use prehashed variant

* message -> hashed_message

* Ultra Honk (AztecProtocol#412)

* DSL: Add valid dummy data for ecdsa constraints when verifier is creating circuit (AztecProtocol#438)

* Add way to make verifiers data valid by replacing zeroes with valid public keys and signatures

Co-authored-by: Zachary James Williamson <zac-williamson@users.noreply.github.com>

* Update cpp/src/barretenberg/dsl/acir_format/ecdsa_secp256k1.test.cpp

* replace templates with concrete methods

* add comment

* PR review

* add comments

* change to use boolean flag, so dummy_ecdsa method lives in ecdsa

* ad true as default

---------

Co-authored-by: Zachary James Williamson <zac-williamson@users.noreply.github.com>

* chore(ci): Add explicit ref for Noir checkout (AztecProtocol#440)

* feat!: add support for ROM and RAM ACVM opcodes (AztecProtocol#417)

* *WIP* do not push

* Generate constraints for dynamic memory

* fix unit test: add missing block_constraint

* add unit test for dynamic memory

* missed one block constraint in ecdsa unit test

* trying a rebase

* remove comments

* fix the build (AztecProtocol#442)

* msgpack: initial support for friendly binary serialization format (AztecProtocol#374)

* Regenerate pedersen lookup tables if they're empty

* re-init generator tables if they're empty.

* feat(nullifier_tree): make empty nullifier tree leaves hash be 0 (AztecProtocol#360)

* feat(nullifier_tree): make empty nullifier tree leaves be 0

* fix: add append zero behaviour and test

* fix: explicit type name

* clean: update class semantics

---------

Co-authored-by: cheethas <urmasurda@gmail.com>
Co-authored-by: cheethas <addaboy@gmail.com>

* More generators for aztec3.

* update js vk (because we now use UP for merkle hashing)

* Helpers for ECDSA in A3 (AztecProtocol#364)

* Add `stdlib_keccak` in cmake.

Correct an assertion in `to_byte_array` in bigfield.

* Add `random_element` to affine element.

* negate y conditionally.

* feat(nullifier_tree): make empty nullifier tree leaves hash be 0 (AztecProtocol#360)

* feat(nullifier_tree): make empty nullifier tree leaves be 0

* fix: add append zero behaviour and test

* fix: explicit type name

* clean: update class semantics

---------

Co-authored-by: cheethas <urmasurda@gmail.com>
Co-authored-by: cheethas <addaboy@gmail.com>

* Change pedersen hash c_bind to use `pedersen_hash::lookup`.

* feat: add msgpack-c submodule

* Give up on msgpack c_master

* Working hacky msgpack test

* Interim work

* Interim work

* Getting rid of memory hacks

* fix: memory leaks

* Start of demoing cbinds

* Align with other methods

* chore: Remove need to return from msgpack method

* Iterate example

* fix: Hack around generator issues

* feat: iterate on msgpack in bb

* fix: fork msgpack for greater checks

* Refactor

* cleanup

* Update turbo_circuit_constructor.cpp

* chore: continued cleanup

* chore: continued cleanup

* chore: continued cleanup

* Refactor

* Refactor

* fix: ci

* feat(wasm): hacks to make work in a fno-exceptions wasm environment

* feat(wasm): bump msgpack-c

* feat(msgpack): first 'complex' object bound

* More wasm fixes. Was breaking throw() declaration

* Fix field serialization

* refactoring

* Update CMakeLists.txt

* Remove // TODO redundant with msgpack

* Refactor to use macro

* Refactor to use macro

* fix printing bug

* fix: fieldd msgpack endianness fix

* fix: remove shared ptr reference

* doc

* Add static checking for MSGPACK usage

* Revert log.hpp change

* Update struct_map_impl.hpp

* Revert

* remote_build fix

* Keep trying to init submodules

* Keep trying to init submodules

* Bump

* Add missing init_submodules

* Msgpack test fix

* Msgpack test fix

* Msgpack test fix

* Msgpack test fix

* Update polynomial_store.test.cpp

* Merge master

* Update msgpack error

* Better abort distinguishing

* fix: join split VK hash

* Serialization updates

* Fix circuits build

* Try to make circuits test work again

* Try to make circuits test work again

* Try to make circuits test work again

* fix: initialization warning

* fix: prefer default constructor for field, related cleanup

* Grand rename

* chore: remove unused funcs

* Revert fields constructor change for now

* chore: Revert .circleci changes

* chore: Revert foundation removal

* Revert .gitmodules

* Update affine_element.hpp

* Update element.hpp

* Revert header optimizations

* Revert init line

* Update polynomial_store.test.cpp

* Revert header optimization

* Update raw_pointer.hpp

* Update raw_pointer.hpp

* Update func_traits.hpp documentation

* Document msgpack methods in field_impl.hpp

* Update msgpack.hpp

* Update cbind.hpp

* Update msgpack.hpp

* Update msgpack.hpp

* Update schema_impl.hpp

* Update g1.hpp

---------

Co-authored-by: Suyash Bagad <suyashnbagad1997@gmail.com>
Co-authored-by: Maddiaa <47148561+cheethas@users.noreply.github.com>
Co-authored-by: cheethas <urmasurda@gmail.com>
Co-authored-by: cheethas <addaboy@gmail.com>
Co-authored-by: Suyash Bagad <suyash@aztecprotocol.com>

* Zw/noir recursion 2 (AztecProtocol#414)

* removed redundant `reduce` operations after negating biggroup elements

simplified hash input structure when hashing transcripts

cached partial non native field multiplications

reverted how native transcript computes hash buffers

pedersen_plookup can be configured to skip the hash_single range check under limited conditions

fixed the range check in pedersen_plookup::hash_single

pedersen_plookup::hash_single now validates the low and high scalar slice values match the  original scalar

bigfield::operator- now correctly uses the UltraPlonk code path if able to

added biggroup::multiple_montgomery_ladder to reduce required field multiplications

added biggroup::quadruple_and_add to reduce required field multiplications

biggroup_nafs now directly calls the Composer range constraint methods to avoid creating redundant arithmetic gates when using the PlookupComposer

biggroup plookup ROM tables now track the maximum size of any field element recovered from the table (i.e. the maximum of the input maximum sizes)

biggroup batch tables prefer to create size-6 lookup tables if doing so reduces the number of individual tables required for a given MSM

recursion::transcript no longer performs redundant range constraints when adding buffer elements
recursion::transcript correctly checks that, when slicing field elements , the slice values are correct over the integers (i.e. slice_sum != original + p)

recursion::verification_key now optimally packs key data into minimum required number of field elements before hashing

recursion::verifier proof and key data is now correctly extracted from the transcript/key instead of being generated directly as witnesses.

cleaned up code + comments

code tidy, added more comments

cleaned up how aggregation object handles public inputs

native verification_key::compress matches circuit output

fixed compile errors + failing tests

compiler error

join_split.test.cpp passing

Note: not changing any upstream .js verification keys. I don't think we need to as bberg is now decoupled from aztec connect

* compiler fix

* more compiler fix

* attempt to fix .js and .sol tests

* revert keccak transcript to original functionality

* added hash_index back into verification_key::compress

fixed composer bug where `decompose_into_default_range` was sometimes not range-constraining last limb

removed commented-out code

added more descriptive comments to PedersenPreimageBuilder

* changed join-split vkey

* temporarily point to branch of aztec that updates aggregation state usage until fix is in aztec master

* revert .aztec-packages-commit

* header brittleness fix

* compiler fix

* compiler fix w. aggregation object

* reverting changes to `assign_object_to_proof_outputs` to preserve backwards-compatibility with a3-packages

* more backwards compatibility fixes

* wip

---------

Co-authored-by: dbanks12 <david@aztecprotocol.com>
Co-authored-by: David Banks <47112877+dbanks12@users.noreply.github.com>

* Chore: bundle msgpack to fix nix-build (AztecProtocol#450)

* Revert msgpack submodule

* Bundle msgpack to avoid issues with submodules

* variable-length keccak (AztecProtocol#441)

* updated stdlib::keccak to be able to hash variable-length inputs (where input size not known at circuit-compile time, only a  maximum possible input size)

* compile error

* compile fils

* compiler fix

* more fix

* compiler fix

* compile fix

* Update cpp/src/barretenberg/stdlib/hash/keccak/keccak.test.cpp

Co-authored-by: kevaundray <kevtheappdev@gmail.com>

* Update cpp/src/barretenberg/stdlib/hash/keccak/keccak.test.cpp

Co-authored-by: kevaundray <kevtheappdev@gmail.com>

* Update cpp/src/barretenberg/stdlib/hash/keccak/keccak.test.cpp

Co-authored-by: kevaundray <kevtheappdev@gmail.com>

* Update cpp/src/barretenberg/stdlib/hash/keccak/keccak.test.cpp

Co-authored-by: kevaundray <kevtheappdev@gmail.com>

* Update cpp/src/barretenberg/stdlib/primitives/field/field.test.cpp

Co-authored-by: kevaundray <kevtheappdev@gmail.com>

* improved readability of stdlib test

* replaced magic numbers in keccak with constants + comments

---------

Co-authored-by: kevaundray <kevtheappdev@gmail.com>

* chore: disable circuits tests in master (AztecProtocol#454)

* fix: msgpack error (AztecProtocol#456)

* Add missing `hash_index` while compressing vk. (AztecProtocol#457)

* Add missing `hash_index` while compressing vk.

* comment back vk tests with hash index > 0.

* Adam/fix allow explicit field init (AztecProtocol#460)

* fix: msgpack error

* fix: allow explicit field init

* fix: msgpack variant_impl.hpp (AztecProtocol#462)

Previous version accidentally created a packer<packer<Stream>>

* fix: bbmalloc linker error (AztecProtocol#459)

* format msgpack serialization and excldue msgpack-c from clang-format (AztecProtocol#467)

* patch: temporarily remove broken solidity ci (AztecProtocol#470)

* Sumcheck improvements (AztecProtocol#455)

* convert partially evaluated polynomials from vectors to Polynomials and rename

* rename fold method to partially_evaluate

* static constexpr barycentric arrays

* change purported evaluations to claimed evaluations

* specify relations in Flavor

* Fixed a bug in biggroup tests (AztecProtocol#478)

* DSL: Add KeccakVar opcode (AztecProtocol#476)

* add initial KeccakVar code

* add result field

* add keccak_var_constraints to fields

* Multi-constraint Relations (AztecProtocol#444)

Allow for correct and efficient batching over identities in the Sumcheck relation

---------

Co-authored-by: Zachary James Williamson <blorktronics@gmail.com>
Co-authored-by: Maxim Vezenov <mvezenov@gmail.com>
Co-authored-by: Adam Domurad <adam@aztecprotocol.com>
Co-authored-by: ledwards2225 <98505400+ledwards2225@users.noreply.github.com>
Co-authored-by: codygunton <codygunton@gmail.com>
Co-authored-by: spypsy <spypsy@users.noreply.github.com>
Co-authored-by: Santiago Palladino <spalladino@gmail.com>
Co-authored-by: Innokentii Sennovskii <isennovskiy@gmail.com>
Co-authored-by: ledwards2225 <l.edwards.d@gmail.com>
Co-authored-by: Blaine Bublitz <blaine.bublitz@gmail.com>
Co-authored-by: Suyash Bagad <suyashnbagad1997@gmail.com>
Co-authored-by: Maxim Vezenov <maximvezenov@Maxims-MacBook-Pro.local>
Co-authored-by: kevaundray <kevtheappdev@gmail.com>
Co-authored-by: Suyash Bagad <suyash@aztecprotocol.com>
Co-authored-by: David Banks <47112877+dbanks12@users.noreply.github.com>
Co-authored-by: Michael Connor <mike@aztecprotocol.com>
Co-authored-by: dbanks12 <david@aztecprotocol.com>
Co-authored-by: Maddiaa <47148561+cheethas@users.noreply.github.com>
Co-authored-by: Santiago Palladino <santiago@aztecprotocol.com>
Co-authored-by: cheethas <urmasurda@gmail.com>
Co-authored-by: maramihali <mara@aztecprotocol.com>
Co-authored-by: Max Hora <maxhora.ua@gmail.com>
Co-authored-by: cheethas <addaboy@gmail.com>
Co-authored-by: Lasse Herskind <16536249+LHerskind@users.noreply.github.com>
Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>
Co-authored-by: guipublic <47281315+guipublic@users.noreply.github.com>
Co-authored-by: maramihali <maramihali@google.com>
Co-authored-by: Zachary James Williamson <zac-williamson@users.noreply.github.com>
Co-authored-by: Maddiaa <47148561+Maddiaa0@users.noreply.github.com>
ludamad pushed a commit to AztecProtocol/aztec-packages that referenced this pull request Jul 22, 2023
* removed redundant `reduce` operations after negating biggroup elements

simplified hash input structure when hashing transcripts

cached partial non native field multiplications

reverted how native transcript computes hash buffers

pedersen_plookup can be configured to skip the hash_single range check under limited conditions

fixed the range check in pedersen_plookup::hash_single

pedersen_plookup::hash_single now validates the low and high scalar slice values match the  original scalar

bigfield::operator- now correctly uses the UltraPlonk code path if able to

added biggroup::multiple_montgomery_ladder to reduce required field multiplications

added biggroup::quadruple_and_add to reduce required field multiplications

biggroup_nafs now directly calls the Composer range constraint methods to avoid creating redundant arithmetic gates when using the PlookupComposer

biggroup plookup ROM tables now track the maximum size of any field element recovered from the table (i.e. the maximum of the input maximum sizes)

biggroup batch tables prefer to create size-6 lookup tables if doing so reduces the number of individual tables required for a given MSM

recursion::transcript no longer performs redundant range constraints when adding buffer elements
recursion::transcript correctly checks that, when slicing field elements , the slice values are correct over the integers (i.e. slice_sum != original + p)

recursion::verification_key now optimally packs key data into minimum required number of field elements before hashing

recursion::verifier proof and key data is now correctly extracted from the transcript/key instead of being generated directly as witnesses.

cleaned up code + comments

code tidy, added more comments

cleaned up how aggregation object handles public inputs

native verification_key::compress matches circuit output

fixed compile errors + failing tests

compiler error

join_split.test.cpp passing

Note: not changing any upstream .js verification keys. I don't think we need to as bberg is now decoupled from aztec connect

* compiler fix

* more compiler fix

* attempt to fix .js and .sol tests

* revert keccak transcript to original functionality

* added hash_index back into verification_key::compress

fixed composer bug where `decompose_into_default_range` was sometimes not range-constraining last limb

removed commented-out code

added more descriptive comments to PedersenPreimageBuilder

* changed join-split vkey

* temporarily point to branch of aztec that updates aggregation state usage until fix is in aztec master

* revert .aztec-packages-commit

* header brittleness fix

* compiler fix

* compiler fix w. aggregation object

* reverting changes to `assign_object_to_proof_outputs` to preserve backwards-compatibility with a3-packages

* more backwards compatibility fixes

* wip

---------

Co-authored-by: dbanks12 <david@aztecprotocol.com>
Co-authored-by: David Banks <47112877+dbanks12@users.noreply.github.com>
ludamad pushed a commit to AztecProtocol/aztec-packages that referenced this pull request Jul 24, 2023
* removed redundant `reduce` operations after negating biggroup elements

simplified hash input structure when hashing transcripts

cached partial non native field multiplications

reverted how native transcript computes hash buffers

pedersen_plookup can be configured to skip the hash_single range check under limited conditions

fixed the range check in pedersen_plookup::hash_single

pedersen_plookup::hash_single now validates the low and high scalar slice values match the  original scalar

bigfield::operator- now correctly uses the UltraPlonk code path if able to

added biggroup::multiple_montgomery_ladder to reduce required field multiplications

added biggroup::quadruple_and_add to reduce required field multiplications

biggroup_nafs now directly calls the Composer range constraint methods to avoid creating redundant arithmetic gates when using the PlookupComposer

biggroup plookup ROM tables now track the maximum size of any field element recovered from the table (i.e. the maximum of the input maximum sizes)

biggroup batch tables prefer to create size-6 lookup tables if doing so reduces the number of individual tables required for a given MSM

recursion::transcript no longer performs redundant range constraints when adding buffer elements
recursion::transcript correctly checks that, when slicing field elements , the slice values are correct over the integers (i.e. slice_sum != original + p)

recursion::verification_key now optimally packs key data into minimum required number of field elements before hashing

recursion::verifier proof and key data is now correctly extracted from the transcript/key instead of being generated directly as witnesses.

cleaned up code + comments

code tidy, added more comments

cleaned up how aggregation object handles public inputs

native verification_key::compress matches circuit output

fixed compile errors + failing tests

compiler error

join_split.test.cpp passing

Note: not changing any upstream .js verification keys. I don't think we need to as bberg is now decoupled from aztec connect

* compiler fix

* more compiler fix

* attempt to fix .js and .sol tests

* revert keccak transcript to original functionality

* added hash_index back into verification_key::compress

fixed composer bug where `decompose_into_default_range` was sometimes not range-constraining last limb

removed commented-out code

added more descriptive comments to PedersenPreimageBuilder

* changed join-split vkey

* temporarily point to branch of aztec that updates aggregation state usage until fix is in aztec master

* revert .aztec-packages-commit

* header brittleness fix

* compiler fix

* compiler fix w. aggregation object

* reverting changes to `assign_object_to_proof_outputs` to preserve backwards-compatibility with a3-packages

* more backwards compatibility fixes

* wip

---------

Co-authored-by: dbanks12 <david@aztecprotocol.com>
Co-authored-by: David Banks <47112877+dbanks12@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants