Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Update Sphinx, replace Groth16 with Plonk #22

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions aptos/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,10 @@ sha2 = "0.9"
thiserror = "1.0.58"
tiny-keccak = "2.0.2"
url = "2.5.0"
sphinx-derive = { git = "ssh://git@github.com/lurk-lab/sphinx", branch = "dev" }
sphinx-sdk = { git = "ssh://git@github.com/lurk-lab/sphinx", branch = "dev" }
sphinx-zkvm = { git = "ssh://git@github.com/lurk-lab/sphinx", branch = "dev" }
sphinx-helper = { git = "ssh://git@github.com/lurk-lab/sphinx", branch = "dev" }
sphinx-derive = { git = "ssh://git@github.com/lurk-lab/sphinx", branch = "plonk" }
sphinx-sdk = { git = "ssh://git@github.com/lurk-lab/sphinx", branch = "plonk", features = ["plonk"] }
sphinx-zkvm = { git = "ssh://git@github.com/lurk-lab/sphinx", branch = "plonk" }
sphinx-helper = { git = "ssh://git@github.com/lurk-lab/sphinx", branch = "plonk" }
tokio = "1.37"
tokio-stream = "0.1"

Expand Down
16 changes: 8 additions & 8 deletions aptos/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,21 +225,21 @@ cargo +nightly nextest run --verbose --release --profile ci --features aptos --p

> Note: The `--no-capture` flag is necessary to see the logs generated by the tests.

### Groth16 proofs
### SNARK proofs (Plonk BN254)

When running any tests or benchmarks that makes Groth16 proofs, it's necessary to build the correct circuit artifacts.
When running any tests or benchmarks that makes Plonk proofs, it's necessary to build the correct circuit artifacts.

Currently, if you don't manually build them, it will lead to a proof generation failure (unsatisfied constraint) due to
circuit differences.
circuit differences in Sphinx.

To build the Groth16 artifacts, do the following:
To build the Plonk artifacts, do the following:

```shell
unset FRI_QUERIES
cd sphinx/prover
make build-groth16
mkdir -p ~/.sp1/circuits/groth16/9f43e920/
cp build/* ~/.sp1/circuits/groth16/9f43e920/
make build-plonk-bn254
mkdir -p ~/.sp1/circuits/plonk_bn254/57ad9e7d/
cp build/* ~/.sp1/circuits/plonk_bn254/57ad9e7d/
```

The trailing commit identifier after `~/.sp1/circuits/groth16/` depends on the value of `GROTH16_ARTIFACTS_COMMIT` defined [here](https://github.com/lurk-lab/sphinx/blob/3f60558d3465c51d7261c33aa8e63d7c7356ca25/prover/src/install.rs#L13), make sure to use the most up-to-date value.
The trailing commit identifier after `~/.sp1/circuits/plonk_bn254/` depends on the value of `PLONK_ARTIFACTS_COMMIT` defined [here](https://github.com/lurk-lab/sphinx/blob/df866f8872c108283dc4d0f63abb336de97a3216/prover/src/install.rs#L18), make sure to use the most up-to-date value.
Binary file not shown.
Binary file modified aptos/aptos-programs/artifacts/epoch-change-program
Binary file not shown.
Binary file modified aptos/aptos-programs/artifacts/inclusion-program
Binary file not shown.
6 changes: 3 additions & 3 deletions aptos/light-client/src/epoch_change.rs
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ mod test {

#[test]
#[ignore = "This test is too slow for CI"]
fn test_groth16_prove_epoch_change() {
fn test_snark_prove_epoch_change() {
use super::*;
use aptos_lc_core::aptos_test_utils::wrapper::AptosWrapper;
use aptos_lc_core::crypto::hash::CryptoHash;
Expand Down Expand Up @@ -201,12 +201,12 @@ mod test {

let start = Instant::now();
println!("Starting generation of prove_epoch_change proof...");
let groth16proof = client.prove_groth16(&pk, stdin).unwrap();
let snark_proof = client.prove_plonk(&pk, stdin).unwrap();
println!("Proving took {:?}", start.elapsed());

let start = Instant::now();
println!("Starting verification of prove_epoch_change proof...");
client.verify_groth16(&groth16proof, &vk).unwrap();
client.verify_plonk(&snark_proof, &vk).unwrap();
println!("Verification took {:?}", start.elapsed());
}
}
6 changes: 3 additions & 3 deletions aptos/light-client/src/inclusion.rs
Original file line number Diff line number Diff line change
Expand Up @@ -298,7 +298,7 @@ mod test {

#[test]
#[ignore = "This test is too slow for CI"]
fn test_groth16_prove_inclusion() {
fn test_snark_prove_inclusion() {
use super::*;
use sphinx_sdk::ProverClient;
use std::time::Instant;
Expand All @@ -319,12 +319,12 @@ mod test {

let start = Instant::now();
println!("Starting generation of inclusion proof...");
let groth16proof = client.prove_groth16(&pk, stdin).unwrap();
let snark_proof = client.prove_plonk(&pk, stdin).unwrap();
println!("Proving took {:?}", start.elapsed());

let start = Instant::now();
println!("Starting verification of inclusion proof...");
client.verify_groth16(&groth16proof, &vk).unwrap();
client.verify_plonk(&snark_proof, &vk).unwrap();
println!("Verification took {:?}", start.elapsed());
}
}
6 changes: 3 additions & 3 deletions aptos/light-client/src/sig.rs
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ mod test {

#[test]
#[ignore = "This test is too slow for CI"]
fn test_groth16_prove_sig() {
fn test_snark_prove_sig() {
use super::*;
use aptos_lc_core::aptos_test_utils::wrapper::AptosWrapper;
use sphinx_sdk::ProverClient;
Expand All @@ -141,12 +141,12 @@ mod test {

let start = Instant::now();
println!("Starting generation of signature verification proof...");
let groth16proof = client.prove_groth16(&pk, stdin).unwrap();
let snark_proof = client.prove_plonk(&pk, stdin).unwrap();
println!("Proving took {:?}", start.elapsed());

let start = Instant::now();
println!("Starting verification of signature verification proof...");
client.verify_groth16(&groth16proof, &vk).unwrap();
client.verify_plonk(&snark_proof, &vk).unwrap();
println!("Verification took {:?}", start.elapsed());
}
}
34 changes: 17 additions & 17 deletions aptos/proof-server/benches/proof_server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,28 +30,28 @@ const ACCOUNT_INCLUSION_DATA_PATH: &str = "./benches/assets/account_inclusion_da
const EPOCH_CHANGE_DATA_PATH: &str = "./benches/assets/epoch_change_data.bcs";

fn main() -> Result<(), anyhow::Error> {
let groth16: bool = env::var("GROTH16").unwrap_or_else(|_| "0".into()) == "1";
let final_snark: bool = env::var("SNARK").unwrap_or_else(|_| "0".into()) == "1";
let run_serially: bool = env::var("RUN_SERIAL").unwrap_or_else(|_| "0".into()) == "1";

let rt = Runtime::new().unwrap();

// Start secondary server
let mut secondary_server_process = rt.block_on(start_secondary_server(groth16))?;
let mut secondary_server_process = rt.block_on(start_secondary_server(final_snark))?;

// Start primary server
let mut primary_server_process = rt.block_on(start_primary_server(groth16))?;
let mut primary_server_process = rt.block_on(start_primary_server(final_snark))?;

// Join the benchmark tasks and block until they are done
let (inclusion_proof, epoch_change_proof) = if run_serially {
rt.block_on(async {
let inclusion_proof = bench_proving_inclusion(groth16).await;
let epoch_change_proof = bench_proving_epoch_change(groth16).await;
let inclusion_proof = bench_proving_inclusion(final_snark).await;
let epoch_change_proof = bench_proving_epoch_change(final_snark).await;
(Ok(inclusion_proof), Ok(epoch_change_proof))
})
} else {
rt.block_on(async {
let inclusion_proof_task = tokio::spawn(bench_proving_inclusion(groth16));
let epoch_change_proof_task = tokio::spawn(bench_proving_epoch_change(groth16));
let inclusion_proof_task = tokio::spawn(bench_proving_inclusion(final_snark));
let epoch_change_proof_task = tokio::spawn(bench_proving_epoch_change(final_snark));

let inclusion_proof = inclusion_proof_task.await.map_err(|e| anyhow!(e));
let epoch_change_proof = epoch_change_proof_task.await.map_err(|e| anyhow!(e));
Expand All @@ -74,13 +74,13 @@ fn main() -> Result<(), anyhow::Error> {
Ok(())
}

async fn start_primary_server(groth16: bool) -> Result<Child, anyhow::Error> {
async fn start_primary_server(final_snark: bool) -> Result<Child, anyhow::Error> {
let primary_addr =
env::var("PRIMARY_ADDR").map_err(|_| anyhow::anyhow!("PRIMARY_ADDR not set"))?;
let secondary_addr =
env::var("SECONDARY_ADDR").map_err(|_| anyhow::anyhow!("SECONDARY_ADDR not set"))?;

let shard_size = if groth16 { "4194304" } else { "1048576" };
let shard_size = if final_snark { "4194304" } else { "1048576" };

let process = Command::new("cargo")
.args([
Expand Down Expand Up @@ -122,11 +122,11 @@ async fn start_primary_server(groth16: bool) -> Result<Child, anyhow::Error> {
}
}

async fn start_secondary_server(groth16: bool) -> Result<Child, anyhow::Error> {
async fn start_secondary_server(final_snark: bool) -> Result<Child, anyhow::Error> {
let secondary_addr =
env::var("SECONDARY_ADDR").map_err(|_| anyhow::anyhow!("SECONDARY_ADDR not set"))?;

let shard_size = if groth16 { "4194304" } else { "1048576" };
let shard_size = if final_snark { "4194304" } else { "1048576" };

let process = Command::new("cargo")
.args([
Expand Down Expand Up @@ -166,7 +166,7 @@ async fn start_secondary_server(groth16: bool) -> Result<Child, anyhow::Error> {
}
}

async fn bench_proving_inclusion(groth16: bool) -> Result<ProofData, anyhow::Error> {
async fn bench_proving_inclusion(final_snark: bool) -> Result<ProofData, anyhow::Error> {
// Connect to primary server
let primary_address =
env::var("PRIMARY_ADDR").map_err(|_| anyhow::anyhow!("PRIMARY_ADDR not set"))?;
Expand All @@ -187,8 +187,8 @@ async fn bench_proving_inclusion(groth16: bool) -> Result<ProofData, anyhow::Err
let inclusion_data: InclusionData = account_inclusion_proof_response.into();

// Send the InclusionData as a request payload to the primary server
let request_bytes = if groth16 {
bcs::to_bytes(&Request::Groth16ProveInclusion(inclusion_data)).map_err(|e| anyhow!(e))?
let request_bytes = if final_snark {
bcs::to_bytes(&Request::SnarkProveInclusion(inclusion_data)).map_err(|e| anyhow!(e))?
} else {
bcs::to_bytes(&Request::ProveInclusion(inclusion_data)).map_err(|e| anyhow!(e))?
};
Expand All @@ -209,7 +209,7 @@ async fn bench_proving_inclusion(groth16: bool) -> Result<ProofData, anyhow::Err
})
}

async fn bench_proving_epoch_change(groth16: bool) -> Result<ProofData, anyhow::Error> {
async fn bench_proving_epoch_change(final_snark: bool) -> Result<ProofData, anyhow::Error> {
// Connect to primary server
let primary_address =
env::var("PRIMARY_ADDR").map_err(|_| anyhow::anyhow!("PRIMARY_ADDR not set"))?;
Expand All @@ -230,8 +230,8 @@ async fn bench_proving_epoch_change(groth16: bool) -> Result<ProofData, anyhow::
let inclusion_data: EpochChangeData = account_inclusion_proof_response.into();

// Send the InclusionData as a request payload to the primary server
let request_bytes = if groth16 {
bcs::to_bytes(&Request::Groth16ProveEpochChange(inclusion_data)).map_err(|e| anyhow!(e))?
let request_bytes = if final_snark {
bcs::to_bytes(&Request::SnarkProveEpochChange(inclusion_data)).map_err(|e| anyhow!(e))?
} else {
bcs::to_bytes(&Request::ProveEpochChange(inclusion_data)).map_err(|e| anyhow!(e))?
};
Expand Down
16 changes: 8 additions & 8 deletions aptos/proof-server/src/bin/server_primary.rs
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ async fn main() -> Result<()> {
write_bytes(&mut client_stream, &verified).await?;
info!("Verification result sent");
}
Ok(Request::Groth16ProveInclusion(inclusion_data)) => {
Ok(Request::SnarkProveInclusion(inclusion_data)) => {
let InclusionData {
sparse_merkle_proof_assets,
transaction_proof_assets,
Expand All @@ -154,25 +154,25 @@ async fn main() -> Result<()> {
);
info!("Start proving");
let proof_handle =
spawn_blocking(move || prover_client.prove_groth16(&pk, stdin));
spawn_blocking(move || prover_client.prove_plonk(&pk, stdin));
let proof = proof_handle.await??;
info!("Proof generated. Serializing");
let proof_bytes = bcs::to_bytes(&proof)?;
info!("Sending proof");
write_bytes(&mut client_stream, &proof_bytes).await?;
info!("Proof sent");
}
Ok(Request::Groth16VerifyInclusion(proof)) => {
Ok(Request::SnarkVerifyInclusion(proof)) => {
write_bytes(
&mut client_stream,
&bcs::to_bytes(&prover_client.verify_groth16(&proof, &vk).is_ok())?,
&bcs::to_bytes(&prover_client.verify_plonk(&proof, &vk).is_ok())?,
)
.await?;
}
Ok(Request::Groth16ProveEpochChange(epoch_change_data)) => {
Ok(Request::SnarkProveEpochChange(epoch_change_data)) => {
info!("Connecting to the secondary server");
let mut secondary_stream = TcpStream::connect(&*snd_addr).await?;
let secondary_request = SecondaryRequest::Groth16Prove(epoch_change_data);
let secondary_request = SecondaryRequest::SnarkProve(epoch_change_data);
info!("Serializing secondary request");
let secondary_request_bytes = bcs::to_bytes(&secondary_request)?;
info!("Sending secondary request");
Expand All @@ -183,10 +183,10 @@ async fn main() -> Result<()> {
write_bytes(&mut client_stream, &proof_bytes).await?;
info!("Proof sent");
}
Ok(Request::Groth16VerifyEpochChange(proof)) => {
Ok(Request::SnarkVerifyEpochChange(proof)) => {
info!("Connecting to the secondary server");
let mut secondary_stream = TcpStream::connect(&*snd_addr).await?;
let secondary_request = SecondaryRequest::Groth16Verify(proof);
let secondary_request = SecondaryRequest::SnarkVerify(proof);
info!("Serializing secondary request");
let secondary_request_bytes = bcs::to_bytes(&secondary_request)?;
info!("Sending secondary request");
Expand Down
8 changes: 4 additions & 4 deletions aptos/proof-server/src/bin/server_secondary.rs
Original file line number Diff line number Diff line change
Expand Up @@ -85,25 +85,25 @@ async fn main() -> Result<()> {
)
.await?;
}
SecondaryRequest::Groth16Prove(EpochChangeData {
SecondaryRequest::SnarkProve(EpochChangeData {
trusted_state,
epoch_change_proof,
}) => {
let stdin = epoch_change::generate_stdin(&trusted_state, &epoch_change_proof);
info!("Start proving");
let proof_handle =
spawn_blocking(move || prover_client.prove_groth16(&pk, stdin));
spawn_blocking(move || prover_client.prove_plonk(&pk, stdin));
let proof = proof_handle.await??;
info!("Proof generated. Serializing");
let proof_bytes = bcs::to_bytes(&proof)?;
info!("Sending proof to the primary server");
write_bytes(&mut primary_stream, &proof_bytes).await?;
info!("Proof sent");
}
SecondaryRequest::Groth16Verify(proof) => {
SecondaryRequest::SnarkVerify(proof) => {
write_bytes(
&mut primary_stream,
&bcs::to_bytes(&prover_client.verify_groth16(&proof, &vk).is_ok())?,
&bcs::to_bytes(&prover_client.verify_plonk(&proof, &vk).is_ok())?,
)
.await?;
}
Expand Down
22 changes: 11 additions & 11 deletions aptos/proof-server/src/types/proof_server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ use aptos_lc::inclusion::{
SparseMerkleProofAssets, TransactionProofAssets, ValidatorVerifierAssets,
};
use serde::{Deserialize, Serialize};
use sphinx_sdk::{SphinxGroth16Proof, SphinxProof};
use sphinx_sdk::{SphinxPlonkBn254Proof, SphinxProof};
use std::fmt::Display;

#[derive(Serialize, Deserialize)]
Expand All @@ -27,10 +27,10 @@ pub enum Request {
ProveEpochChange(EpochChangeData),
VerifyInclusion(SphinxProof),
VerifyEpochChange(SphinxProof),
Groth16ProveInclusion(InclusionData),
Groth16ProveEpochChange(EpochChangeData),
Groth16VerifyInclusion(SphinxGroth16Proof),
Groth16VerifyEpochChange(SphinxGroth16Proof),
SnarkProveInclusion(InclusionData),
SnarkProveEpochChange(EpochChangeData),
SnarkVerifyInclusion(SphinxPlonkBn254Proof),
SnarkVerifyEpochChange(SphinxPlonkBn254Proof),
}

impl Display for &Request {
Expand All @@ -40,10 +40,10 @@ impl Display for &Request {
Request::ProveEpochChange(_) => write!(f, "ProveEpochChange"),
Request::VerifyInclusion(_) => write!(f, "VerifyInclusion"),
Request::VerifyEpochChange(_) => write!(f, "VerifyEpochChange"),
Request::Groth16ProveInclusion(_) => write!(f, "Groth16ProveInclusion"),
Request::Groth16ProveEpochChange(_) => write!(f, "Groth16ProveEpochChange"),
Request::Groth16VerifyInclusion(_) => write!(f, "Groth16VerifyInclusion"),
Request::Groth16VerifyEpochChange(_) => write!(f, "Groth16VerifyEpochChange"),
Request::SnarkProveInclusion(_) => write!(f, "SnarkProveInclusion"),
Request::SnarkProveEpochChange(_) => write!(f, "SnarkProveEpochChange"),
Request::SnarkVerifyInclusion(_) => write!(f, "SnarkVerifyInclusion"),
Request::SnarkVerifyEpochChange(_) => write!(f, "SnarkVerifyEpochChange"),
}
}
}
Expand All @@ -52,6 +52,6 @@ impl Display for &Request {
pub enum SecondaryRequest {
Prove(EpochChangeData),
Verify(SphinxProof),
Groth16Prove(EpochChangeData),
Groth16Verify(SphinxGroth16Proof),
SnarkProve(EpochChangeData),
SnarkVerify(SphinxPlonkBn254Proof),
}