Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[consensus] fix batch availability counters #14855

Merged
merged 1 commit into from
Oct 3, 2024
Merged

Conversation

ibalajiarun
Copy link
Contributor

Description

Calculates missing batches properly.

Copy link

trunk-io bot commented Oct 3, 2024

⏱️ 35m total CI duration on this PR
Slowest 15 Jobs Cumulative Duration Recent Runs
test-target-determinator 8m 🟩🟩
rust-doc-tests 5m 🟩
execution-performance / test-target-determinator 4m 🟩
check 4m 🟩
rust-cargo-deny 4m 🟩🟩
check-dynamic-deps 2m 🟩🟩🟩
rust-move-tests 2m 🟩
rust-move-tests 2m 🟩
fetch-last-released-docker-image-tag 1m 🟩
semgrep/ci 1m 🟩🟩🟩
general-lints 54s 🟩🟩
file_change_determinator 28s 🟩🟩🟩
file_change_determinator 26s 🟩🟩
permission-check 17s 🟩🟩🟩
execution-performance / single-node-performance 11s 🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
execution-performance / single-node-performance 11s 17m -99%

settingsfeedbackdocs ⋅ learn more about trunk.io

@ibalajiarun ibalajiarun changed the title Balaji/optqs measure [consensus] fix batch availability counters Oct 3, 2024
@ibalajiarun ibalajiarun added the CICD:run-forge-e2e-perf Run the e2e perf forge only label Oct 3, 2024

This comment has been minimized.

@ibalajiarun ibalajiarun enabled auto-merge (squash) October 3, 2024 02:13

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

github-actions bot commented Oct 3, 2024

✅ Forge suite compat success on e8a6faf60a98afca8982e0582f929401294bbd33 ==> 077b252e50e628fe34b430759532a8870956fabe

Compatibility test results for e8a6faf60a98afca8982e0582f929401294bbd33 ==> 077b252e50e628fe34b430759532a8870956fabe (PR)
1. Check liveness of validators at old version: e8a6faf60a98afca8982e0582f929401294bbd33
compatibility::simple-validator-upgrade::liveness-check : committed: 11241.78 txn/s, latency: 2730.69 ms, (p50: 1900 ms, p70: 2200, p90: 3100 ms, p99: 27600 ms), latency samples: 448140
2. Upgrading first Validator to new version: 077b252e50e628fe34b430759532a8870956fabe
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 6671.91 txn/s, latency: 4292.86 ms, (p50: 4900 ms, p70: 5200, p90: 5300 ms, p99: 5400 ms), latency samples: 120040
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6594.68 txn/s, latency: 4837.25 ms, (p50: 5300 ms, p70: 5700, p90: 6000 ms, p99: 6300 ms), latency samples: 227060
3. Upgrading rest of first batch to new version: 077b252e50e628fe34b430759532a8870956fabe
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 6758.20 txn/s, latency: 4177.42 ms, (p50: 4700 ms, p70: 5000, p90: 5300 ms, p99: 5400 ms), latency samples: 125920
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6596.55 txn/s, latency: 4934.99 ms, (p50: 5300 ms, p70: 5500, p90: 6200 ms, p99: 6700 ms), latency samples: 227000
4. upgrading second batch to new version: 077b252e50e628fe34b430759532a8870956fabe
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 8459.38 txn/s, latency: 3107.08 ms, (p50: 2800 ms, p70: 3800, p90: 4800 ms, p99: 5300 ms), latency samples: 160580
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 6089.92 txn/s, latency: 4117.71 ms, (p50: 2900 ms, p70: 4500, p90: 5500 ms, p99: 15300 ms), latency samples: 289640
5. check swarm health
Compatibility test for e8a6faf60a98afca8982e0582f929401294bbd33 ==> 077b252e50e628fe34b430759532a8870956fabe passed
Test Ok

Copy link
Contributor

github-actions bot commented Oct 3, 2024

✅ Forge suite realistic_env_max_load success on 077b252e50e628fe34b430759532a8870956fabe

two traffics test: inner traffic : committed: 13534.99 txn/s, latency: 2938.35 ms, (p50: 2700 ms, p70: 3000, p90: 3100 ms, p99: 3600 ms), latency samples: 5146300
two traffics test : committed: 100.07 txn/s, latency: 2735.41 ms, (p50: 2500 ms, p70: 2600, p90: 2800 ms, p99: 11400 ms), latency samples: 1760
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.237, avg: 0.222", "QsPosToProposal: max: 0.278, avg: 0.234", "ConsensusProposalToOrdered: max: 0.322, avg: 0.299", "ConsensusOrderedToCommit: max: 0.490, avg: 0.465", "ConsensusProposalToCommit: max: 0.789, avg: 0.765"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.83s no progress at version 8893 (avg 0.21s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.35s no progress at version 1847819 (avg 7.70s) [limit 15].
Test Ok

Copy link
Contributor

github-actions bot commented Oct 3, 2024

✅ Forge suite framework_upgrade success on e8a6faf60a98afca8982e0582f929401294bbd33 ==> 077b252e50e628fe34b430759532a8870956fabe

Compatibility test results for e8a6faf60a98afca8982e0582f929401294bbd33 ==> 077b252e50e628fe34b430759532a8870956fabe (PR)
Upgrade the nodes to version: 077b252e50e628fe34b430759532a8870956fabe
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1177.93 txn/s, submitted: 1181.27 txn/s, failed submission: 3.34 txn/s, expired: 3.34 txn/s, latency: 2639.08 ms, (p50: 2100 ms, p70: 2600, p90: 5000 ms, p99: 6300 ms), latency samples: 105800
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1129.34 txn/s, submitted: 1131.80 txn/s, failed submission: 2.46 txn/s, expired: 2.46 txn/s, latency: 2652.43 ms, (p50: 2400 ms, p70: 2700, p90: 4400 ms, p99: 5400 ms), latency samples: 101080
5. check swarm health
Compatibility test for e8a6faf60a98afca8982e0582f929401294bbd33 ==> 077b252e50e628fe34b430759532a8870956fabe passed
Upgrade the remaining nodes to version: 077b252e50e628fe34b430759532a8870956fabe
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1176.22 txn/s, submitted: 1177.57 txn/s, failed submission: 1.35 txn/s, expired: 1.35 txn/s, latency: 2549.79 ms, (p50: 2400 ms, p70: 2700, p90: 3600 ms, p99: 5400 ms), latency samples: 104840
Test Ok

@ibalajiarun ibalajiarun merged commit b6c01b9 into main Oct 3, 2024
137 of 145 checks passed
@ibalajiarun ibalajiarun deleted the balaji/optqs-measure branch October 3, 2024 03:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CICD:run-forge-e2e-perf Run the e2e perf forge only
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants