Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: avoid superfluous clones in ProverKey and VerifierKey for Nova & SuperNova #285

Merged
merged 4 commits into from
Jan 31, 2024
Merged

refactor: avoid superfluous clones in ProverKey and VerifierKey for Nova & SuperNova #285

merged 4 commits into from
Jan 31, 2024

Conversation

huitseeker
Copy link
Contributor

@huitseeker huitseeker commented Jan 29, 2024 •
edited
Loading

This PR removes extra copies of the CommitmentKey in the ProverKey and VerifierKey.

Using the results of this PR, we hope to remove the ProverKey and VerifierKey in Lurk's PublicParams, leaving only the nova::PublicParams field there, from which it's cheap to derive the removed fields as needed,

In Detail

This notices that it is fundamentally cheap to derive prover and verifier key from a set of universal public parameters (in nova parlance, the CommitmentKey), and takes that into motion:

  • the setup functions for instances of EvaluationEgine are deterministic and near-free,
  • by composition, so is the setup of CompressedSNARK,
  • those functions do not need to allocate a significant amount of data,
  • we consequently drop the Serialize, Deserialize and Abomonate bounds on all ProverKeys,
  • we consequently drop the Deserialize and Abomonate bounds on all VerifierKeys,
  • the Serialize bound on the VerifierKey is momentarily kept in order to compute its Digestible instance using SimpleDigestible (it's not essential to keep, but a bit involved for this short PR)

In More Detail: repairing the Abomonate functionality

This introduces Arc inside the fields of the PublicParams data structures, which is at odds with Abomonate-ing them in the course of a proof. Since it is highly non trivial (if at all possible) to Abomonate something under an Arc, we bypass the problem in the following way:

  • we implement FlatPublicParams, a facade with no Arc, which we know how to Abomonate
  • TryFrom<PublicParams> for FlatPublicParams and
  • From<FlatPublicParams> for <PublicParams>,

The TryFrom<PublicParams> for FlatPublicParams relies on Arc::try_unwrap for each of the two problematic fields, and the only thing we'd have to take care of at runtime is that we haven't left a trailing ProverKey or VerifierKey in scope before doing the conversion.

Then instead of trying to have Abomonate on the PublicParams, what the caching logic does is FlatPublicParams::try_from(nova_public_params)?.encode() and PublicParams::from(decode::<FlatPublicParams>(bytes).clone()). We demonstrate that in the minroot example.

Note

Lurk is not expected to compile on top of this. Companion PR at lurk-lab/lurk-beta#1085

@huitseeker
refactor: Refactor codebase to use Arc for commitment keys
23faed5 
- Major changes in the handling of the Commitment Key object across several files, moving from direct references to using the Arc (Atomic Reference Counting) mechanism for improved thread-safety and efficient sharing of data.
- Removed the `Abomonation` trait from several structures, such as `ProverKey` and `VerifierKey`.
- Introduced the `SimpleDigestible` trait to VerifierKey in several files, allowing them to be converted into a digestible form for hashing or storage.
- Numerous updates in test cases and instances to reflect the shift from direct Commitment Key references to Arc.
- Refactoring of the `setup` and `prove` methods in multiple files to handle the Arc type for Commitment Key.
- Removal of several `abomonate_with` and `abomonation_skip` attribute annotations.
- Enhancement of error details in multiple `assert` statements.
- Added "rc" feature to the serde dependency in the Cargo.toml file.
- Made comments on minimizing unnecessary data copies and keeping `setup` methods inexpensive.
@huitseeker
refactor: Serialization with Abomonation
78741eb 
- Extended functionality in `lib.rs` with new `PublicParams` structure, efficient abomonationwith `FlatPublicParams`,
- Updated `mod.rs` for conditional feature `abomonate`, added `AuxParams` struct, and implemented conversions between `AuxParams` and `FlatAuxParams`.
- demoed the new use of Abomonation in the minroot example
huitseeker added a commit to lurk-lab/lurk-beta that referenced this pull request Jan 30, 2024
@huitseeker
refactor: Refactor public parameters handling and caching
4a40f93 
This adapts to upstream lurk-lang/arecibo#285, which removes the need for separate `ProverKey` and `VerifierKey` instances (the former containing `CommitmentKey` copies) in Public Parameters.

- Implemented structural changes in `nova.rs` for improvements in key handling for the prover and verifier keys, which generated as `OnceCell`,

- Revamped `disk_cache.rs` with added `Arc` imports, elimination of lifetime parameter `'a` from `DiskCache`, modifications of `read` and `write` functions, and updates in the serialization and deserialization steps.
- Altered `public_params` and `supernova_public_params` functions in `mod.rs` to deal more capably with errors and modified various methods and tests to align with the changes in `disk_cache.rs`.
- Removed complete `mem_cache.rs` file, which was unsafe, and unused.

- Moved initialization of `store` to an earlier point in `fibonacci_prove` function and made minor formatting changes in `fibonacci.rs`.
@huitseeker huitseeker mentioned this pull request Jan 30, 2024
Merged
huitseeker added a commit to lurk-lab/lurk-beta that referenced this pull request Jan 30, 2024
@huitseeker
refactor: Refactor public parameters handling and caching
9e8c203 
This adapts to upstream lurk-lang/arecibo#285, which removes the need for separate `ProverKey` and `VerifierKey` instances (the former containing `CommitmentKey` copies) in Public Parameters.

- Implemented structural changes in `nova.rs` for improvements in key handling for the prover and verifier keys, which generated as `OnceCell`,

- Revamped `disk_cache.rs` with added `Arc` imports, elimination of lifetime parameter `'a` from `DiskCache`, modifications of `read` and `write` functions, and updates in the serialization and deserialization steps.
- Altered `public_params` and `supernova_public_params` functions in `mod.rs` to deal more capably with errors and modified various methods and tests to align with the changes in `disk_cache.rs`.
- Removed complete `mem_cache.rs` file, which was unsafe, and unused.

- Moved initialization of `store` to an earlier point in `fibonacci_prove` function and made minor formatting changes in `fibonacci.rs`.
@huitseeker huitseeker mentioned this pull request Jan 31, 2024
Merged
@winston-h-zhang winston-h-zhang mentioned this pull request Jan 31, 2024
Draft
@huitseeker huitseeker added this pull request to the merge queue Jan 31, 2024
Merged via the queue into dev with commit 6846dd5 Jan 31, 2024
8 of 9 checks passed
@huitseeker huitseeker deleted the killing_params branch January 31, 2024 21:26
github-merge-queue bot pushed a commit to lurk-lab/lurk-beta that referenced this pull request Jan 31, 2024
@huitseeker
Refactor public parameters based on Arecibo changes (#1085)
3648868 
* refactor: Refactor public parameters handling and caching

This adapts to upstream lurk-lang/arecibo#285, which removes the need for separate `ProverKey` and `VerifierKey` instances (the former containing `CommitmentKey` copies) in Public Parameters.

- Implemented structural changes in `nova.rs` for improvements in key handling for the prover and verifier keys, which generated as `OnceCell`,

- Revamped `disk_cache.rs` with added `Arc` imports, elimination of lifetime parameter `'a` from `DiskCache`, modifications of `read` and `write` functions, and updates in the serialization and deserialization steps.
- Altered `public_params` and `supernova_public_params` functions in `mod.rs` to deal more capably with errors and modified various methods and tests to align with the changes in `disk_cache.rs`.
- Removed complete `mem_cache.rs` file, which was unsafe, and unused.

- Moved initialization of `store` to an earlier point in `fibonacci_prove` function and made minor formatting changes in `fibonacci.rs`.

* refactor: Refactor code to remove lifetime on Instance + Abomonation dependency

- Removed `Abomonation` trait and `PrimeField` imports from `multiframe.rs`, also simplified the `NonUniformCircuit` access in the same file.
- Removed `Abomonation` trait bounds from the `circuit_cache_key` functions in `supernova.rs`.
- Refactored `Instance<>` struct in `instance.rs`, removing the usage of `Abomonation` and `PhantomData`, and eliminated need for lifetime declaration `'a`.
- Enhanced `open()` method in `instance.rs` to include metadata comparison to prevent instance mismatches.

* refactor: Adapt SuperNova parameters and remove superfluous Abomonation bounds

- Improved `supernova.rs` by using `OnceCell` to store `ProverKey` and `VerifierKey` and updating related methods,
- removed uneeded Abomonation bounds in a few places.

* chore: slience clippy

* refactor: Refactor PublicParams cache to remove obsolete std::sync::Arc usage

- reduced complexity by removing `std::sync::Arc` in `src/public_parameters/disk_cache.rs`.
- Modified `read` and `write` functions in `DiskCache` to work directly with `PublicParams`.

* chore: remove anymap dependency

* chore: point the dependency back to Arecibo's dev
@huitseeker huitseeker mentioned this pull request Feb 1, 2024
Closed
gabriel-barrett pushed a commit to gabriel-barrett/arecibo that referenced this pull request Feb 9, 2024
@huitseeker @jbearer
Developer quality of life improvements (Nova forward ports) (lurk-lan…
221c002 
…g#226)

* Expose the last outputs and number of steps from RecursiveSNARK (lurk-lang#285)

Both of these data are easily accessible, and could be very useful
to clients:
* Exposing the last outputs allows us to get the current state of
  the computation on the prover side without wasting energy
  recomputing it
* Exposing the number of steps makes it easier to eventually pass
  `num_steps` into `CompressedSNARK::verify`

* Improve error handling (lurk-lang#286)

* When a function already returns a `Result`, propagate errors
  instead of panicking with `expect`
* For `NovaError::SynthesisError`, retain information about the
  original bellpepper error. Since `NovaError` implements `Clone`
  but bellpepper's `SynthesisError` does not, we keep the error
  information as a `String`.

This commit only fixes low-hanging fruit in lib.rs, for functions
that already return a `Result` and can easily propagate errors just
by replacing `expect(...)` with `?`. There are still many `unwrap()`
calls in functions returning `Result` in other modules, particularly
gadgets. But I don't understand the code well in those parts, and I
suspect some of those `unwrap()`s actually can't fail based on
invariants of the code, so it makes perfect sense to leave them as is.

Co-authored-by: Francois Garillot <francois@lurk-lab.com>

---------

Co-authored-by: Jeb Bearer <jeb.bearer@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@winston-h-zhang winston-h-zhang winston-h-zhang approved these changes

@adr1anh adr1anh Awaiting requested review from adr1anh

@porcuquine porcuquine Awaiting requested review from porcuquine

@mpenciak mpenciak Awaiting requested review from mpenciak

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@huitseeker @winston-h-zhang