basic benchmark #41

lispc · 2021-02-14T14:43:37Z

Choose a circuit ( either spotTrade or Block including spotTrade ), get a table with following schema:

Both one core and multi core ( 4 core or 8 core ) ( or real time / user time ) results are useful :

time	groth16-zkutil	groth16-rapidsnark	plonk-plonkit
rescue hash	??	??	??
poseidon hash	??	??	??

plonkit should be upgraded to bellman beta branch ( If fully upgrading including solidity stuff costs much time, we can create a branch rather than push unfinished code to plonkit master )

The bench script code may be executed later so it is needed to be commited into the repo. ( We don't need a full auto script. Some manual operations are ok )

https://github.com/Fluidex/circuits/blob/master/tools/replace_hash.sh may be useful.

rapidsnark need a zkey, we may have to generate it.

The text was updated successfully, but these errors were encountered:

0xmountaintop · 2021-02-19T06:29:48Z

Block(4, 1, 1) with poseidon_hash:

ubuntu@ip-172-31-11-58:~/repos/circuits/tools/benchmark/data/poseidon/block$ head *.time
==> plonkit.time <==
++ plonkit prove --srs_monomial_form /home/ubuntu/repos/circuits/tools/benchmark/keys/plonk/2pow20.key --circuit circuit.r1cs --witness witness.wtns --proof proof.bin

real    0m5.191s
user    1m21.696s
sys     0m3.429s

==> plonkit_lagrange.time <==
++ plonkit prove -m /home/ubuntu/repos/circuits/tools/benchmark/keys/plonk/2pow20.key -l /home/ubuntu/repos/circuits/tools/benchmark/keys/plonk/2pow20_lagrange.key -c circuit.r1cs -w witness.wtns -p proof.bin

real    0m5.074s
user    1m14.342s
sys     0m3.484s

==> rapidsnark.time <==
++ /home/ubuntu/repos/circuits/tools/benchmark/../../node_modules/rapidsnark/build/prover zkey.zkey witness.wtns proof.json public.json

real    0m1.316s
user    1m44.137s
sys     0m7.348s

==> zkutil.time <==
++ zkutil prove

real    0m1.197s
user    0m12.163s
sys     0m0.803s

0xmountaintop · 2021-02-19T06:30:01Z

Block(4, 1, 1) with rescue_hash:

ubuntu@ip-172-31-11-58:~/repos/circuits/tools/benchmark/data/rescue/block$ head *.time
==> plonkit.time <==
++ plonkit prove --srs_monomial_form /home/ubuntu/repos/circuits/tools/benchmark/keys/plonk/2pow20.key --circuit circuit.r1cs --witness witness.wtns --proof proof.bin

real    0m5.412s
user    1m23.291s
sys     0m3.742s

==> plonkit_lagrange.time <==
++ plonkit prove -m /home/ubuntu/repos/circuits/tools/benchmark/keys/plonk/2pow20.key -l /home/ubuntu/repos/circuits/tools/benchmark/keys/plonk/2pow20_lagrange.key -c circuit.r1cs -w witness.wtns -p proof.bin

real    0m5.270s
user    1m16.324s
sys     0m3.357s

==> rapidsnark.time <==
++ /home/ubuntu/repos/circuits/tools/benchmark/../../node_modules/rapidsnark/build/prover zkey.zkey witness.wtns proof.json public.json

real    0m1.288s
user    1m41.400s
sys     0m7.051s

==> zkutil.time <==
++ zkutil prove

real    0m1.805s
user    0m15.699s
sys     0m0.556s

lispc · 2021-02-19T06:57:03Z

better to post circuit here like 'Block(2,2,2,)'

0xmountaintop · 2021-02-19T07:13:03Z

I see.

lispc · 2021-03-23T12:50:13Z

==> ./data/rescue/massive/plonkit.time <==
real 2m34.194s
user 57m6.142s
sys 0m36.691s

==> ./data/rescue/massive/plonkit_lagrange.time <==
real 2m36.156s
user 53m46.198s
sys 0m30.630s

==> ./data/rescue/massive/zkutil.time <==
real 0m50.003s
user 10m31.481s
sys 0m8.330s

==> ./data/poseidon/massive/plonkit.time <==
real 2m36.773s
user 57m27.337s
sys 0m46.737s

==> ./data/poseidon/massive/plonkit_lagrange.time <==
real 2m39.482s
user 56m2.942s
sys 0m41.280s

==> ./data/poseidon/massive/zkutil.time <==
real 0m38.163s
user 9m37.281s
sys 0m6.639s
++ find ./ -name circuit.circom

tail -n 3 ./data/rescue/massive/circuit.circom ./data/poseidon/massive/circuit.circom
==> ./data/rescue/massive/circuit.circom <==
include "/home/ubuntu/repos/circuits/src/block.circom";
component main = Block(10, 20, 20, 20);

lispc · 2021-03-24T06:42:56Z

rapidsnark zkey generation is too slow; zkutil only supports json witness, but nodejs seems cannot write huge json file (> 512MB ). We may need to cherry-pick wtns reading to zkutil

lispc · 2021-03-24T07:00:52Z

component main = Block(50, 20, 20, 20);

==> ./data/rescue/massive/plonkit.time <==
real 10m23.963s
user 222m25.871s
sys 3m27.515s

We will use this circuit for a while

lispc · 2021-04-07T06:13:53Z

Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz

component main = Block(100, 10, 10, 10); // it takes about 24h to process the circuit

real 18m50.111s
user 425m52.298s
sys 7m50.207s

component main = Block(100, 10, 10, 10);

real 0m14.529s
user 4m52.056s
sys 0m4.144s

lispc · 2021-04-07T06:19:55Z

Conclusion:

So the proving time is basically linear as expected.

100 tps needs 300 24core machine ( real core? ht core? )

Assume EC2 c5.12xlarge ( 48 vcpus $2.04 per Hour ). 100 tps costs 500-600$/h ? ( c5 instance has 3.5G freq ) , much cheaper than ETH GAS. Huawei cloud: C6s ( 48 vcpus 8.19CNY/h )

Latency about 15-20min~

lispc · 2021-05-15T09:53:25Z

==> /home/ubuntu/repos/circuits/tools/benchmark/data/massive/plonkit_lagrange_gen.time <==
real 232m25.302s
user 7166m45.125s
sys 6m10.302s

==> /home/ubuntu/repos/circuits/tools/benchmark/data/massive/plonkit.time <==
real 18m22.700s
user 421m24.006s
sys 5m55.358s

==> /home/ubuntu/repos/circuits/tools/benchmark/data/massive/plonkit_lagrange.time <==
real 18m31.070s
user 371m51.932s
sys 10m21.829s

==> /home/ubuntu/repos/circuits/tools/benchmark/data/massive/zkutil.time <==
real 1m51.334s
user 38m11.670s
sys 0m50.919s

==> /home/ubuntu/repos/circuits/tools/benchmark/data/massive/circuit.circom <==
include "/home/ubuntu/repos/circuits/src/block.circom";
component main = Block(100, 20, 20, 20);

lispc · 2021-05-28T08:05:23Z

[2021-05-15T05:08:07Z INFO  plonkit::plonk] transpile done, gates_count 42176699 hints size 11403099
[2021-05-15T05:11:29Z INFO  plonkit::plonk] circuit setup_polynomials.n 67108863 size 26

so 421766.99 gates per tx. 2**28 == 636 * 421766.99. more than 600 layer2 tx in one l2 block

lispc · 2021-07-01T09:35:52Z

Block(100, 20, 20, 20):

peak memory: 261GB

constructionPhase: 35:58.744
reduceConstants: 06:37.769
reduceConstraints: 12:27:16.233
classifySignals: 02:18.745
generateWitnessNames: 01:24.083
generateR1cs: 10:17.195
generateC: 29:40.610
generateSyms: .001
Command being timed: "node --trace-gc --trace-gc-ignore-scavenger --max-old-space-size=1400000 --initial-old-space-size=65500 --no-global-gc-scheduling --no-incremental-marking --max-semi-space-size=1024 --initial-heap-size=65500 /home/ubuntu/repos/circuits/node_modules/.bin/circom /home/ubuntu/repos/circuits/tools/benchmark/data/massive/circuit.circom -r /home/ubuntu/repos/circuits/tools/benchmark/data/massive/circuit.r1cs -c /home/ubuntu/repos/circuits/tools/benchmark/data/massive/circuit.cpp -s /home/ubuntu/repos/circuits/tools/benchmark/data/massive/circuit.sym -v"
User time (seconds): 49801.72
System time (seconds): 1605.20
Percent of CPU this job got: 102%
Elapsed (wall clock) time (h:mm:ss or m:ss): 13:59:48
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 274080964
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 1010020882
Voluntary context switches: 4558624
Involuntary context switches: 150365
Swaps: 0
File system inputs: 0
File system outputs: 21368608
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0

lispc · 2021-10-09T08:59:32Z

zksyn zk run verify-keys gen circuit-size

transfer needs 2 chunks, swap needs 5 chunks. So transfer needs 200k gates, swap needs 500k gates.

   Compiling key_generator v1.0.0 (/home/ubuntu/repos/zksync/core/bin/key_generator)
    Finished release [optimized] target(s) in 1m 50s
     Running `target/release/key_generator circuit-size`
Oct 09 08:52:33.004  INFO key_generator::zksync_key: Counting max zkSync circuit size for setup
Oct 09 08:52:38.107  INFO key_generator::zksync_key: setup_size_log2: 21, chunks: 10, gate_count: 2148475
Oct 09 08:52:55.589  INFO key_generator::zksync_key: setup_size_log2: 22, chunks: 32, gate_count: 4348332
Oct 09 08:53:22.293  INFO key_generator::zksync_key: setup_size_log2: 23, chunks: 72, gate_count: 8431960
Oct 09 08:54:36.849  INFO key_generator::zksync_key: setup_size_log2: 24, chunks: 156, gate_count: 16917362
Oct 09 08:56:45.615  INFO key_generator::zksync_key: setup_size_log2: 25, chunks: 322, gate_count: 33701320
Oct 09 09:01:02.216  INFO key_generator::zksync_key: setup_size_log2: 26, chunks: 654, gate_count: 67255497

lispc assigned ChrisLinn Feb 14, 2021

0xmountaintop pinned this issue Feb 19, 2021

lispc mentioned this issue Mar 23, 2021

design: decide/test/benchmark rescue vs poseidon #132

Open

lispc closed this as completed Apr 7, 2021

lispc reopened this Apr 7, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

basic benchmark #41

basic benchmark #41

lispc commented Feb 14, 2021 •

edited

Loading

0xmountaintop commented Feb 19, 2021 •

edited

Loading

0xmountaintop commented Feb 19, 2021 •

edited

Loading

lispc commented Feb 19, 2021

0xmountaintop commented Feb 19, 2021

lispc commented Mar 23, 2021

lispc commented Mar 24, 2021 •

edited

Loading

lispc commented Mar 24, 2021

lispc commented Apr 7, 2021 •

edited

Loading

lispc commented Apr 7, 2021 •

edited

Loading

lispc commented May 15, 2021

lispc commented May 28, 2021

lispc commented Jul 1, 2021 •

edited

Loading

lispc commented Oct 9, 2021 •

edited

Loading

basic benchmark #41

basic benchmark #41

Comments

lispc commented Feb 14, 2021 • edited Loading

0xmountaintop commented Feb 19, 2021 • edited Loading

0xmountaintop commented Feb 19, 2021 • edited Loading

lispc commented Feb 19, 2021

0xmountaintop commented Feb 19, 2021

lispc commented Mar 23, 2021

lispc commented Mar 24, 2021 • edited Loading

lispc commented Mar 24, 2021

lispc commented Apr 7, 2021 • edited Loading

lispc commented Apr 7, 2021 • edited Loading

lispc commented May 15, 2021

lispc commented May 28, 2021

lispc commented Jul 1, 2021 • edited Loading

lispc commented Oct 9, 2021 • edited Loading

lispc commented Feb 14, 2021 •

edited

Loading

0xmountaintop commented Feb 19, 2021 •

edited

Loading

0xmountaintop commented Feb 19, 2021 •

edited

Loading

lispc commented Mar 24, 2021 •

edited

Loading

lispc commented Apr 7, 2021 •

edited

Loading

lispc commented Apr 7, 2021 •

edited

Loading

lispc commented Jul 1, 2021 •

edited

Loading

lispc commented Oct 9, 2021 •

edited

Loading