Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Decompress a PublicKey 500% performance regression in v1.4.0 #46

Closed
ChihChengLiang opened this issue Dec 19, 2020 · 1 comment
Closed

Decompress a PublicKey 500% performance regression in v1.4.0 #46

ChihChengLiang opened this issue Dec 19, 2020 · 1 comment

Comments

@ChihChengLiang
Copy link

What's wrong

The performance of decompressing a PublicKey is significant slow in v1.4.0. Cropped benchmark result:

compression/Decompress a PublicKey                                                                            
                        time:   [364.46 us 372.29 us 383.01 us]
                        change: [+497.89% +518.88% +547.26%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  7 (7.00%) high mild
  4 (4.00%) high severe

Steps to reproduce

git checkout v1.3.0 
git submodule update
rm -rf target/
cargo bench --features "bench"
git checkout v1.4.0
git submodule update 
cargo bench --features "bench"

Logs:

v1.4.0

Running target/release/deps/bls381_benches-21ec5f2d00d66323

signing/Create a Signature                                                                            
                        time:   [2.2649 ms 2.4643 ms 2.8161 ms]
                        change: [-4.1061% +2.3568% +11.914%] (p = 0.72 > 0.05)
                        No change in performance detected.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

signing/Verify a Signature                                                                           
                        time:   [7.6921 ms 8.3690 ms 9.3884 ms]
                        change: [-6.8269% +2.0599% +13.337%] (p = 0.70 > 0.05)
                        No change in performance detected.

Benchmarking multiple-signatures-verification-30/Verification of multiple aggregate signatures with optimizations: Collecting 10 samples in estimated 8.6845 s                                                                                                                                                                multiple-signatures-verification-30/Verification of multiple aggregate signatures with optimizations                        
                        time:   [81.784 ms 89.208 ms 96.381 ms]
                        change: [+3.7598% +9.7247% +18.196%] (p = 0.01 < 0.05)
                        Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

aggregation/Verifying aggregate of 128 signatures                                                                            
                        time:   [7.6322 ms 7.7766 ms 7.9435 ms]
                        change: [+2.2044% +4.3055% +6.3692%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) high mild
  4 (4.00%) high severe

aggregation/Aggregate a PublicKey                                                                             
                        time:   [2.1013 us 2.1592 us 2.2340 us]
                        change: [+54.977% +78.314% +105.88%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  4 (4.00%) high mild
  7 (7.00%) high severe

aggregation/Aggregate a Signature                                                                             
                        time:   [5.1918 us 5.3296 us 5.4961 us]
                        change: [+7.5565% +11.184% +14.766%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
  2 (2.00%) high mild
  10 (10.00%) high severe

compression/Decompress a Signature                                                                            
                        time:   [198.73 us 200.60 us 202.77 us]
                        change: [+6.7434% +9.5261% +13.081%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) high mild
  4 (4.00%) high severe

compression/Compress a Signature                                                                            
                        time:   [1.4246 us 1.4627 us 1.5011 us]
                        change: [-5.3093% +1.8072% +7.8935%] (p = 0.65 > 0.05)
                        No change in performance detected.

compression/Decompress a PublicKey                                                                            
                        time:   [364.46 us 372.29 us 383.01 us]
                        change: [+497.89% +518.88% +547.26%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  7 (7.00%) high mild
  4 (4.00%) high severe

compression/Compress a PublicKey                                                                            
                        time:   [1.4288 us 1.4490 us 1.4937 us]
                        change: [-10.385% -0.2387% +10.038%] (p = 0.97 > 0.05)
                        No change in performance detected.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

compression/Decompress a PublicKey from Bigs                                                                             
                        time:   [1.7494 us 1.7913 us 1.8458 us]
                        change: [+6.5483% +8.9457% +11.375%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe

compression/Compress a PublicKey to Bigs                                                                            
                        time:   [1.0970 us 1.1233 us 1.1606 us]
                        change: [+5.9204% +10.128% +16.469%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

key generation/Generate random keypair                                                                            
                        time:   [389.87 us 400.86 us 414.20 us]
                        change: [+8.9707% +11.088% +13.485%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
  2 (2.00%) high mild
  10 (10.00%) high severe

key generation/Generate keypair from known string                                                                            
                        time:   [419.51 us 431.32 us 445.95 us]
                        change: [+4.7468% +12.559% +19.381%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) high mild
  7 (7.00%) high severe

v1.3.0

Running target/release/deps/bls381_benches-fe64a3c500672cce
signing/Create a Signature                                                                            
                        time:   [2.2934 ms 2.3310 ms 2.3758 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

signing/Verify a Signature                                                                           
                        time:   [7.5973 ms 7.8653 ms 8.3068 ms]

Benchmarking multiple-signatures-verification-30/Verification of multiple aggregate signatures with optimizations: Collecting 10 samples in estimated 8.5392 s                                                                                                                                                                multiple-signatures-verification-30/Verification of multiple aggregate signatures with optimizations                        
                        time:   [75.792 ms 77.780 ms 80.390 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

aggregation/Verifying aggregate of 128 signatures                                                                            
                        time:   [7.4041 ms 7.4556 ms 7.5082 ms]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

aggregation/Aggregate a PublicKey                                                                             
                        time:   [1.6946 us 1.7270 us 1.7770 us]
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe

aggregation/Aggregate a Signature                                                                             
                        time:   [4.9031 us 5.0449 us 5.2287 us]
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe

compression/Decompress a Signature                                                                            
                        time:   [189.00 us 190.42 us 192.16 us]
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe

compression/Compress a Signature                                                                            
                        time:   [1.3475 us 1.4022 us 1.4885 us]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

compression/Decompress a PublicKey                                                                            
                        time:   [62.944 us 63.413 us 63.958 us]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

compression/Compress a PublicKey                                                                            
                        time:   [1.4373 us 1.5601 us 1.7437 us]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

compression/Decompress a PublicKey from Bigs                                                                             
                        time:   [1.6095 us 1.6231 us 1.6474 us]
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) high mild
  7 (7.00%) high severe

compression/Compress a PublicKey to Bigs                                                                            
                        time:   [1.0481 us 1.0510 us 1.0547 us]

key generation/Generate random keypair                                                                            
                        time:   [351.54 us 352.96 us 354.64 us]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

key generation/Generate keypair from known string                                                                            
                        time:   [389.64 us 410.85 us 440.89 us]
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe
This was referenced Dec 19, 2020
Merged
Closed
@kirk-baird
Copy link
Member

I believe this is due to key_validate() and subgroup checking
https://github.com/sigp/milagro_bls/blob/master/src/keys.rs#L137

@ncitron ncitron mentioned this issue Jan 31, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ChihChengLiang @kirk-baird