Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run test suite with libeatmydata #5007

Closed
morgo opened this issue Jul 17, 2019 · 2 comments · Fixed by #5635
Closed

Run test suite with libeatmydata #5007

morgo opened this issue Jul 17, 2019 · 2 comments · Fixed by #5635
Milestone

Comments

@morgo
Copy link
Contributor

morgo commented Jul 17, 2019

Feature Description

The testsuite does a lot of mysqld --initialize-insecure steps, which take about 3-4 seconds on my local machine with the testsuite's default-fast.cnf configuration file.

I tried adjusting the config file -- but there don't appear to be any low hanging fruit. The innodb-log-file-size for example is already 4M.

But I didn't see a performance improvement by LD_PRELOAD'ing libeatmydata in a microbench:

Without LD_PRELOAD:

real	0m4.068s
user	0m0.543s
sys	0m0.276s

With LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libeatmydata.so:

real	0m1.971s
user	0m0.161s
sys	0m0.079s

I think tried on an isolated test (backup), and the performance went from 7m1.5s to 6m22s -- so not as big of an improvement to the total profile, but probably still worth doing.

This needs to be included in the Docker image, so the test command can change to:

export LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libeatmydata.so"
...
go run test.go $TEST_FLAGS $TEST_MATRIX

Use Case(s)

This helps improve the test case run speed, particularly in cases where there is high io latency, since libeatmydata turns fsync & friends into noops.

Based on the result of the backup test: we can assume it is a global 10-20% run time improvement. So maybe not worth doing immediately, but has some value.

@morgo morgo self-assigned this Nov 13, 2019
@morgo
Copy link
Contributor Author

morgo commented Nov 13, 2019

We can look at doing this in the GitHub actions tests. Currently, they are much faster than the Travis tests, so it's not a high priority.

@morgo morgo removed their assignment Nov 26, 2019
@morgo morgo added this to the v6.0 milestone Dec 7, 2019
@morgo
Copy link
Contributor Author

morgo commented Dec 7, 2019

As the python tests are getting converted, some GitHub tests are taking longer to run. I will likely take a look at this in Vitess 6.0 timeframe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant