Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(performance): add sleep after warm-up #9922

Merged
merged 1 commit into from
Jan 28, 2025

Conversation

juliayakovlev
Copy link
Contributor

@juliayakovlev juliayakovlev commented Jan 27, 2025

Add 2 minutes sleep after warm-up and before step start in predefined steps performnace test. This is for let to all background processes to finish. It may have the impact especially in tablet test - to complete tablets rebuild

Fixes: #9852

Testing

  • [ ]

PR pre-checks (self review)

  • I added the relevant backport labels
  • I didn't leave commented-out/debugging code

Reminders

  • Add New configuration option and document them (in sdcm/sct_config.py)
  • Add unit tests to cover my changes (under unit-test/ folder)
  • Update the Readme/doc folder relevant to this change (if needed)

@fruch
Copy link
Contributor

fruch commented Jan 28, 2025

Can you share a graph from a run using this code, to see if 2 min is enough time to be able to clearly see warmup doesn't have affect on the main load

@juliayakovlev
Copy link
Contributor Author

juliayakovlev commented Jan 28, 2025

https://argus.scylladb.com/tests/scylla-cluster-tests/9df777c1-3b2a-42ed-8846-dd7559d5efd0

This run (with tablets) with sleep 120 sec after warm-up and before 150K step. the latency is 1.75ms

P90 read [ms] P99 read [ms] Throughput read [op/s] duration [HH:MM:SS] start time Overview QA dashboard
Cycle #​1 $${\color{green}0.41}$$ $${\color{green}1.75}$$ 149,719 00:30:15 10:45:11 view view

@juliayakovlev
Copy link
Contributor Author

https://argus.scylladb.com/tests/scylla-cluster-tests/803ec3ab-91ec-441a-b324-f45b6cb6ba1c

This run (with tablets) with sleep 240 sec after warm-up and before 150K step. The latency is 0.48

P90 read [ms] P99 read [ms] Throughput read [op/s] duration [HH:MM:SS] start time Overview QA dashboard
Cycle #​1 $${\color{green}0.41}$$ $${\color{green}0.48}$$ 149,645 00:30:15 13:44:32 view view

Add 4 minutes sleep after warm-up and before step start in predefined steps performnace test.
This is for let to all background processes to finish. It may have the impact especially in
tablet test - to complete tablets rebuild

Fixes: scylladb#9852
@roydahan
Copy link
Contributor

Let's stop wasting more money on runs.

Copy link
Contributor

@fruch fruch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fruch fruch merged commit 998eff3 into scylladb:master Jan 28, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

P99 read error threshold for 150K step in read load predefined test with tablets
4 participants