Skip to content

fix: stm tests run time regression #2307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 13, 2025
Merged

fix: stm tests run time regression #2307

merged 3 commits into from
Feb 13, 2025

Conversation

Alenar
Copy link
Collaborator

@Alenar Alenar commented Feb 13, 2025

Content

This PR fix a regression in run times of mithril-stm tests.

Merge of #2282 changed the feature set used by mithril-client-wasm, including the activation of the num-integer-backed feature from the mithril-client crate.
With feature unification this have the side effect of enabling this slower backend for everyone when you run tests at the workspace level.

Removing this feature activation in client wasm is no-op since mithril-stm enforce it when building to wasm or windows.

Pre-submit checklist

  • Branch
    • Crates versions are updated (if relevant)
    • Commit sequence broadly makes sense
    • Key commits have useful messages
  • PR
    • No clippy warnings in the CI
    • Self-reviewed the diff
    • Useful pull request description
    • Reviewer requested

Comments

Here are run times before and after this change:

Expand

  • Before:
cargo nextest run --features full,unstable --workspace -E "package(mithril-stm)"                                                                                                     
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.41s                                                                                                                
    Starting 38 tests across 44 binaries (run ID: ffeddb0c-5d7d-4695-b8ab-625ba7e52e6d, nextest profile: default)                                                                       
        PASS [   0.028s] mithril-stm merkle_tree::tests::test_bytes_tree_commitment_batch_compat                                                                                        
        PASS [   0.030s] mithril-stm merkle_tree::tests::test_bytes_tree_commitment                                                                                                     
        PASS [   0.035s] mithril-stm merkle_tree::tests::test_bytes_tree                                                                                                                
        PASS [   0.045s] mithril-stm eligibility_check::tests::test_precision_approximation                                                                                             
        PASS [   0.105s] mithril-stm merkle_tree::tests::test_create_batch_proof                                                                                                        
        PASS [   0.127s] mithril-stm merkle_tree::tests::test_bytes_batch_path                                                                                                          
        PASS [   0.130s] mithril-stm multi_sig::tests::serialize_deserialize_sk                                                                                                         
        PASS [   0.186s] mithril-stm merkle_tree::tests::test_create_proof                                                                                                              
        PASS [   0.372s] mithril-stm merkle_tree::tests::test_bytes_path                                                                                                                
        PASS [   0.449s] mithril-stm multi_sig::tests::test_eval_sanity_check                                                                                                           
        PASS [   0.039s] mithril-stm stm::tests::test_initializer_serialize_deserialize                                                                                                 
        PASS [   0.589s] mithril-stm multi_sig::tests::test_gen                                                                                                                         
        PASS [   0.733s] mithril-stm merkle_tree::tests::test_create_invalid_proof                                                                                                      
        PASS [   0.841s] mithril-stm merkle_tree::tests::test_create_invalid_batch_proof                                                                                                
        PASS [   0.704s] mithril-stm stm::tests::test_dedup                                                                                                                             
        PASS [   1.105s] mithril-stm eligibility_check::tests::early_break_taylor                                                                                                       
        PASS [   0.010s] mithril-stm stm::tests::test_parameters_serialize_deserialize                                                                                                  
        PASS [   0.952s] mithril-stm stm::tests::test_multisig_serialize_deserialize                                                                                                    
        PASS [   0.196s] mithril-stm stm::tests::test_sig_serialize_deserialize                                                                                                         
        PASS [   2.290s] mithril-stm multi_sig::tests::serialize_deserialize_vk                                                                                                         
        PASS [   1.678s] mithril-stm stm::tests::test_sig                                                                                                                               
        PASS [   2.758s] mithril-stm multi_sig::tests::test_sig                                                                                                                         
        PASS [   2.767s] mithril-stm multi_sig::tests::test_invalid_sig                                                                                                                 
        PASS [   0.899s] mithril-stm stm::tests::test_total_stake_core_verifier                                                                                                         
        PASS [  10.463s] mithril-stm key_reg::tests::test_keyreg                                                                                                                        
        PASS [  10.699s] mithril-stm stm::tests::test_core_verifier                                                                                                                     
        PASS [   8.168s] mithril-stm::stm_protocol test_full_protocol                                                                                                                   
        PASS [  12.589s] mithril-stm stm::tests::test_invalid_proof_index_unique
        PASS [  14.354s] mithril-stm stm::tests::test_invalid_proof_index_bound
        PASS [  14.343s] mithril-stm stm::tests::test_invalid_proof_quorum
        PASS [  14.636s] mithril-stm stm::tests::test_invalid_proof_path
        PASS [  15.913s] mithril-stm stm::tests::test_aggregate_sig
        PASS [  21.763s] mithril-stm::stm_protocol test_full_protocol_batch_verify
        PASS [  28.122s] mithril-stm multi_sig::tests::test_aggregate_sig
        PASS [  39.950s] mithril-stm::stm_core test_core_verifier
        PASS [  56.550s] mithril-stm multi_sig::tests::batch_verify
        SLOW [> 60.000s] mithril-stm stm::tests::batch_verify
        SLOW [> 60.000s] mithril-stm stm::tests::test_adversary_quorum
        PASS [  65.153s] mithril-stm stm::tests::batch_verify
        PASS [  76.503s] mithril-stm stm::tests::test_adversary_quorum
------------
     Summary [  76.631s] 38 tests run: 38 passed (2 slow), 0 skipped
  • After:
cargo nextest run --features full,unstable --workspace -E "package(mithril-stm)"                                                                                                     
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.26s                                                                                             11:15:16 [165/1821]
    Starting 37 tests across 44 binaries (run ID: addd65a7-3348-49cf-8722-ff121bb7fdb3, nextest profile: default)                                                                       
        PASS [   0.013s] mithril-stm eligibility_check::tests::test_precision_approximation                                                                                             
        PASS [   0.027s] mithril-stm merkle_tree::tests::test_bytes_tree_commitment_batch_compat                                                                                        
        PASS [   0.032s] mithril-stm merkle_tree::tests::test_bytes_tree_commitment                                                                                                     
        PASS [   0.035s] mithril-stm merkle_tree::tests::test_bytes_tree                                                                                                                
        PASS [   0.100s] mithril-stm merkle_tree::tests::test_create_batch_proof                                                                                                        
        PASS [   0.112s] mithril-stm multi_sig::tests::serialize_deserialize_sk                                                                                                         
        PASS [   0.122s] mithril-stm merkle_tree::tests::test_bytes_batch_path                                                                                                          
        PASS [   0.162s] mithril-stm merkle_tree::tests::test_create_proof                                                                                                              
        PASS [   0.355s] mithril-stm merkle_tree::tests::test_bytes_path                                                                                                                
        PASS [   0.040s] mithril-stm stm::tests::test_initializer_serialize_deserialize                                                                                                 
        PASS [   0.427s] mithril-stm multi_sig::tests::test_eval_sanity_check                                                                                                           
        PASS [   0.564s] mithril-stm multi_sig::tests::test_gen                                                                                                                         
        PASS [   0.741s] mithril-stm merkle_tree::tests::test_create_invalid_proof                                                                                                      
        PASS [   0.665s] mithril-stm stm::tests::test_dedup                                                                                                                             
        PASS [   0.842s] mithril-stm merkle_tree::tests::test_create_invalid_batch_proof                                                                                                
        PASS [   0.005s] mithril-stm stm::tests::test_parameters_serialize_deserialize                                                                                                  
        PASS [   0.728s] mithril-stm stm::tests::test_multisig_serialize_deserialize                                                                                                    
        PASS [   0.149s] mithril-stm stm::tests::test_sig_serialize_deserialize                                                                                                         
        PASS [   1.724s] mithril-stm stm::tests::test_core_verifier                                                                                                                     
        PASS [   2.202s] mithril-stm multi_sig::tests::serialize_deserialize_vk                                                                                                         
        PASS [   1.430s] mithril-stm stm::tests::test_sig                                                                                                                               
        PASS [   0.819s] mithril-stm stm::tests::test_total_stake_core_verifier                                                                                                         
        PASS [   2.799s] mithril-stm multi_sig::tests::test_invalid_sig                                                                                                                 
        PASS [   2.893s] mithril-stm multi_sig::tests::test_sig                                                                                                                         
        PASS [   0.749s] mithril-stm::stm_protocol test_full_protocol                                                                                                                   
        PASS [   1.448s] mithril-stm::stm_protocol test_full_protocol_batch_verify                                                                                                      
        PASS [   1.997s] mithril-stm::stm_core test_core_verifier                                                                                                                       
        PASS [   5.193s] mithril-stm stm::tests::test_aggregate_sig                                                                                                                     
        PASS [   5.708s] mithril-stm key_reg::tests::test_keyreg                                                                                                                        
        PASS [   8.295s] mithril-stm stm::tests::test_adversary_quorum                                                                                                                  
        PASS [   8.035s] mithril-stm stm::tests::test_invalid_proof_index_bound                                                                                                         
        PASS [   7.897s] mithril-stm stm::tests::test_invalid_proof_quorum                                                                                                              
        PASS [   8.219s] mithril-stm stm::tests::test_invalid_proof_path                                                                                                                
        PASS [   8.448s] mithril-stm stm::tests::test_invalid_proof_index_unique                                                                                                        
        PASS [  15.904s] mithril-stm multi_sig::tests::test_aggregate_sig                                                                                                               
        PASS [  22.904s] mithril-stm stm::tests::batch_verify                                                                                                                           
        PASS [  38.957s] mithril-stm multi_sig::tests::batch_verify                                                                                                                     
------------
     Summary [  38.959s] 37 tests run: 37 passed, 0 skipped

Merge of #2282 changed the feature set used by `mithril-client-wasm`,
including the activation of the `num-integer-backed` feature from the
`mithril-client` crate.
With feature unification this have the side effect of enabling this
slower backend for everyone when you run tests at the workspace level.

Removing this feature in client wasm is no-op since `mithril-stm`
enforce it when building to wasm or windows.
@Alenar Alenar self-assigned this Feb 13, 2025
Copy link

github-actions bot commented Feb 13, 2025

Test Results

    4 files  ±0     56 suites  ±0   10m 47s ⏱️ - 10m 14s
1 597 tests  - 1  1 597 ✅  - 1  0 💤 ±0  0 ❌ ±0 
1 901 runs   - 1  1 901 ✅  - 1  0 💤 ±0  0 ❌ ±0 

Results for commit c900596. ± Comparison against base commit 6c2ecb8.

This pull request removes 1 test.
mithril-stm ‑ eligibility_check::tests::early_break_taylor

♻️ This comment has been updated with latest results.

@Alenar Alenar temporarily deployed to testing-preview February 13, 2025 10:43 — with GitHub Actions Inactive
@Alenar Alenar temporarily deployed to testing-sanchonet February 13, 2025 10:43 — with GitHub Actions Inactive
* mithril-client-wasm from `0.8.1` to `0.8.2`
* [js] mithril-client-wasm from `0.8.1` to `0.8.2`
@Alenar Alenar temporarily deployed to testing-preview February 13, 2025 15:37 — with GitHub Actions Inactive
@Alenar Alenar temporarily deployed to testing-sanchonet February 13, 2025 15:37 — with GitHub Actions Inactive
@Alenar Alenar merged commit 9ef0fd4 into main Feb 13, 2025
37 of 41 checks passed
@Alenar Alenar deleted the djo/fix-tests-slowness branch February 13, 2025 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants