Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add poseidon2 hashing to native transcript #3718

Merged
merged 43 commits into from
Jan 30, 2024
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
29a0d28
added poseidon2 to native transcript
lucasxia01 Dec 14, 2023
61c9eca
wip
lucasxia01 Jan 4, 2024
a8204fb
removed grumpkin poseidon2 params
lucasxia01 Jan 10, 2024
05085c5
removed transcript template param,
lucasxia01 Jan 10, 2024
99bd01d
removing poseidon2 grumpkin
lucasxia01 Jan 10, 2024
029350c
removed template param from honk::proof
lucasxia01 Jan 10, 2024
a045b6d
linker error be gone among other merge errors
lucasxia01 Jan 14, 2024
86065bc
conversions
lucasxia01 Jan 16, 2024
d50ac92
updated calc_num_frs() with all transcript types
lucasxia01 Jan 17, 2024
5f46e2d
completed convert_to_bn254_frs for various transcript types
lucasxia01 Jan 17, 2024
d157087
convert_from_bn254_frs initial impl updated
lucasxia01 Jan 17, 2024
f769528
fixing bugs (wip)
lucasxia01 Jan 17, 2024
e8702c7
hacky solution to templating to get things to compile
lucasxia01 Jan 17, 2024
86eae85
checking if CI tests still fail
lucasxia01 Jan 18, 2024
4a04326
fixed bugs in conversions
lucasxia01 Jan 18, 2024
2d16e5d
added (partial) unit tests
lucasxia01 Jan 18, 2024
2c6406d
updated structuring and tests
lucasxia01 Jan 18, 2024
c0c715c
temporary "fix" to ci gcc compile error
lucasxia01 Jan 18, 2024
dd3366c
cleaning up field conversion calc_num_frs style
lucasxia01 Jan 18, 2024
b37bed8
undo challenge fix
lucasxia01 Jan 18, 2024
09f7ea2
pushing linker error
lucasxia01 Jan 19, 2024
7346629
Merge branch 'master' into lx/transcript-native-field-refactor
lucasxia01 Jan 19, 2024
1484030
changed to bb::
lucasxia01 Jan 19, 2024
97c2f88
inline + comments + cleanup
lucasxia01 Jan 19, 2024
e2f7526
completed tests, small updates
lucasxia01 Jan 19, 2024
f67148d
Merge branch 'master' into lx/transcript-native-field-refactor
lucasxia01 Jan 19, 2024
c0b5fb6
reomving unnecessary cmake stuff
lucasxia01 Jan 22, 2024
d3e110b
Merge branch 'master' into lx/transcript-native-field-refactor
lucasxia01 Jan 22, 2024
330c0c8
Merge branch 'master' into lx/transcript-native-field-refactor
lucasxia01 Jan 23, 2024
7fd7553
fix?
lucasxia01 Jan 23, 2024
bbfb92b
cbind fix fix
lucasxia01 Jan 23, 2024
a53a144
undo
lucasxia01 Jan 23, 2024
e12cb61
maybe cbind fix
lucasxia01 Jan 23, 2024
fe43adf
cbind fix for real this time
lucasxia01 Jan 24, 2024
b1d5678
fixed error caused by overfixing
lucasxia01 Jan 24, 2024
cab1312
Merge branch 'master' into lx/transcript-native-field-refactor
lucasxia01 Jan 24, 2024
54bb864
fixed merge conflicts with protogalaxy, decider recursive verifiers
lucasxia01 Jan 24, 2024
9cc55d1
updated conversion for grumpkin fr
lucasxia01 Jan 29, 2024
87078b3
responded to PR comments, refactored, added comments
lucasxia01 Jan 29, 2024
850ec4d
small style updates
lucasxia01 Jan 30, 2024
9f6293c
Merge branch 'master' into lx/transcript-native-field-refactor
lucasxia01 Jan 30, 2024
777e8b1
updated merged in code
lucasxia01 Jan 30, 2024
de5b619
updated protogalaxy cmakelists
lucasxia01 Jan 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions barretenberg/build-system
Submodule build-system added at a109f3
2 changes: 2 additions & 0 deletions barretenberg/cpp/src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ set(BARRETENBERG_TARGET_OBJECTS
$<TARGET_OBJECTS:crypto_keccak_objects>
$<TARGET_OBJECTS:crypto_pedersen_commitment_objects>
$<TARGET_OBJECTS:crypto_pedersen_hash_objects>
$<TARGET_OBJECTS:crypto_poseidon2_objects>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure what this is doing but this helped me resolve some annoying linker errors, presumably introduced because goblin was introduced to main

Copy link
Contributor Author

@lucasxia01 lucasxia01 Jan 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is necessary from building without it/with it since transcript uses it, which is used by main.cpp in goblin_verify

$<TARGET_OBJECTS:crypto_schnorr_objects>
$<TARGET_OBJECTS:crypto_sha256_objects>
$<TARGET_OBJECTS:dsl_objects>
Expand All @@ -130,6 +131,7 @@ set(BARRETENBERG_TARGET_OBJECTS
$<TARGET_OBJECTS:stdlib_merkle_tree_objects>
$<TARGET_OBJECTS:stdlib_pedersen_commitment_objects>
$<TARGET_OBJECTS:stdlib_pedersen_hash_objects>
$<TARGET_OBJECTS:stdlib_poseidon2_objects>
$<TARGET_OBJECTS:stdlib_primitives_objects>
$<TARGET_OBJECTS:stdlib_recursion_objects>
$<TARGET_OBJECTS:stdlib_schnorr_objects>
Expand Down
2 changes: 2 additions & 0 deletions barretenberg/cpp/src/barretenberg/barretenberg.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
#include "crypto/keccak/keccak.hpp"
#include "crypto/pedersen_commitment/pedersen.hpp"
#include "crypto/pedersen_hash/pedersen.hpp"
#include "crypto/poseidon2/poseidon2.hpp"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added in an attempt to solve linker error, but probably still needed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like this file is unused so adding this is just for documentation about what is in barretenberg

#include "crypto/schnorr/schnorr.hpp"
#include "crypto/sha256/sha256.hpp"
#include "ecc/curves/bn254/fq.hpp"
Expand All @@ -41,6 +42,7 @@
#include "stdlib/hash/blake2s/blake2s.hpp"
#include "stdlib/hash/blake3s/blake3s.hpp"
#include "stdlib/hash/pedersen/pedersen.hpp"
#include "stdlib/hash/poseidon2/poseidon2.hpp"
#include "stdlib/merkle_tree/hash.hpp"
#include "stdlib/merkle_tree/membership.hpp"
#include "stdlib/merkle_tree/memory_store.hpp"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,8 @@ template <typename Params> class Poseidon2Permutation {
using MatrixDiagonal = std::array<FF, t>;
using RoundConstantsContainer = std::array<RoundConstants, NUM_ROUNDS>;

static constexpr MatrixDiagonal internal_matrix_diagonal =
Poseidon2Bn254ScalarFieldParams::internal_matrix_diagonal;
static constexpr RoundConstantsContainer round_constants = Poseidon2Bn254ScalarFieldParams::round_constants;
static constexpr MatrixDiagonal internal_matrix_diagonal = Params::internal_matrix_diagonal;
static constexpr RoundConstantsContainer round_constants = Params::round_constants;

static constexpr void matrix_multiplication_4x4(State& input)
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ void AcirComposer::create_goblin_circuit(acir_format::acir_format& constraint_sy
GoblinMockCircuits::construct_goblin_ecc_op_circuit(goblin_builder_);
}

std::vector<uint8_t> AcirComposer::create_goblin_proof()
std::vector<barretenberg::fr> AcirComposer::create_goblin_proof()
{
return goblin.construct_proof(goblin_builder_);
}
Expand Down Expand Up @@ -160,7 +160,7 @@ bool AcirComposer::verify_proof(std::vector<uint8_t> const& proof, bool is_recur
}
}

bool AcirComposer::verify_goblin_proof(std::vector<uint8_t> const& proof)
bool AcirComposer::verify_goblin_proof(std::vector<barretenberg::fr> const& proof)
{
return goblin.verify_proof({ proof });
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ class AcirComposer {

// Goblin specific methods
void create_goblin_circuit(acir_format::acir_format& constraint_system, acir_format::WitnessVector& witness);
std::vector<uint8_t> create_goblin_proof();
bool verify_goblin_proof(std::vector<uint8_t> const& proof);
std::vector<barretenberg::fr> create_goblin_proof();
bool verify_goblin_proof(std::vector<barretenberg::fr> const& proof);

private:
acir_format::Builder builder_;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ WASM_EXPORT void acir_get_proving_key(in_ptr acir_composer_ptr, uint8_t const* a
WASM_EXPORT void acir_verify_goblin_proof(in_ptr acir_composer_ptr, uint8_t const* proof_buf, bool* result)
{
auto acir_composer = reinterpret_cast<acir_proofs::AcirComposer*>(*acir_composer_ptr);
auto proof = from_buffer<std::vector<uint8_t>>(proof_buf);
auto proof = from_buffer<std::vector<barretenberg::fr>>(proof_buf);
*result = acir_composer->verify_goblin_proof(proof);
}

Expand Down
5 changes: 4 additions & 1 deletion barretenberg/cpp/src/barretenberg/ecc/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
barretenberg_module(ecc numeric crypto_keccak crypto_sha256)
barretenberg_module(ecc numeric crypto_keccak crypto_sha256 ecc_fields)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, an attempt to solve linker errors but not sure if this is needed at all

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing this caused no problems


if(DISABLE_ADX)
message(STATUS "Disabling ADX assembly variant.")
Expand All @@ -14,7 +14,10 @@ target_precompile_headers(
$<$<COMPILE_LANGUAGE:CXX>:"${CMAKE_CURRENT_SOURCE_DIR}/fields/asm_macros.hpp">
$<$<COMPILE_LANGUAGE:CXX>:"${CMAKE_CURRENT_SOURCE_DIR}/fields/field_declarations.hpp">
$<$<COMPILE_LANGUAGE:CXX>:"${CMAKE_CURRENT_SOURCE_DIR}/fields/field_impl.hpp">
$<$<COMPILE_LANGUAGE:CXX>:"${CMAKE_CURRENT_SOURCE_DIR}/fields/field_conversion_utils.hpp">
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assume this is necessary?

$<$<COMPILE_LANGUAGE:CXX>:"${CMAKE_CURRENT_SOURCE_DIR}/fields/field_impl_generic.hpp">
$<$<COMPILE_LANGUAGE:CXX>:"${CMAKE_CURRENT_SOURCE_DIR}/fields/field_impl_x64.hpp">
$<$<COMPILE_LANGUAGE:CXX>:"${CMAKE_CURRENT_SOURCE_DIR}/fields/field.hpp">
)

add_subdirectory(fields)
lucasxia01 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
barretenberg_module(ecc_fields)
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@

#include "barretenberg/ecc/fields/field_conversion_utils.hpp"
lucasxia01 marked this conversation as resolved.
Show resolved Hide resolved

namespace barretenberg::field_conversion_utils {

lucasxia01 marked this conversation as resolved.
Show resolved Hide resolved
static constexpr uint64_t NUM_CONVERSION_LIMB_BITS = 64;

/**
* @brief Decomposes a barretenberg::fr into two 64-bit limbs. Helper function for
* convert_barretenberg_frs_to_grumpkin_fr.
*
* @param field_val
* @return std::array<uint64_t, 2>
*/
std::array<uint64_t, 2> decompose_bn254_fr_to_two_limbs(const barretenberg::fr& field_val)
{
ASSERT(uint256_t(field_val) < (uint256_t(1) << (2 * NUM_CONVERSION_LIMB_BITS))); // should be 128 bits or less
constexpr uint256_t LIMB_MASK =
(uint256_t(1) << NUM_CONVERSION_LIMB_BITS) - 1; // split bn254_fr into two 64 bit limbs
const uint256_t value = field_val;
const uint64_t low = static_cast<uint64_t>(value & LIMB_MASK);
const uint64_t hi = static_cast<uint64_t>(value >> NUM_CONVERSION_LIMB_BITS);
ASSERT(static_cast<uint256_t>(low) + (static_cast<uint256_t>(hi) << NUM_CONVERSION_LIMB_BITS) == value);

return std::array<uint64_t, 2>{ low, hi };
}

/**
* @brief Converts 2 barretenberg::fr elements to grumpkin::fr
* @details Checks that each barretenberg::fr must be at most 128 bits (to ensure no overflow), and decomposes each
* barretenberg::fr into two 64-bit limbs, and the 4 64-bit limbs form the grumpkin::fr
* @param low_bits_in
* @param high_bits_in
* @return grumpkin::fr
*/
grumpkin::fr convert_barretenberg_frs_to_grumpkin_fr(const barretenberg::fr& low_bits_in,
const barretenberg::fr& high_bits_in)
{
// TODO: figure out can_overflow, maximum_bitlength in stdlib version
ASSERT(uint256_t(low_bits_in) < (uint256_t(1) << (NUM_CONVERSION_LIMB_BITS * 2)));
ASSERT(uint256_t(high_bits_in) < (uint256_t(1) << (NUM_CONVERSION_LIMB_BITS * 2)));
auto low_bit_decomp = decompose_bn254_fr_to_two_limbs(low_bits_in);
uint256_t tmp;
tmp.data[0] = low_bit_decomp[0];
tmp.data[1] = low_bit_decomp[1];
auto high_bit_decomp = decompose_bn254_fr_to_two_limbs(high_bits_in);
tmp.data[2] = high_bit_decomp[0];
tmp.data[3] = high_bit_decomp[1];
grumpkin::fr result(tmp);
return result;
}

/**
* @brief Converts grumpkin::fr to 2 barretenberg::fr elements
* @details Does the reverse of convert_barretenberg_frs_to_grumpkin_fr, by merging the two pairs of limbs back into the
* 2 barretenberg::fr elements.
* @param input
* @return std::array<barretenberg::fr, 2>
*/
std::array<barretenberg::fr, 2> convert_grumpkin_fr_to_barretenberg_frs(const grumpkin::fr& input)
{
auto tmp = static_cast<uint256_t>(input);
std::array<barretenberg::fr, 2> result;
result[0] = static_cast<uint256_t>(tmp.data[0]) + (static_cast<uint256_t>(tmp.data[1]) << NUM_CONVERSION_LIMB_BITS);
result[1] = static_cast<uint256_t>(tmp.data[2]) + (static_cast<uint256_t>(tmp.data[3]) << NUM_CONVERSION_LIMB_BITS);
return result;
}

} // namespace barretenberg::field_conversion_utils
Loading