cmake: allow running the tests in parallel like in the Makefile (#102) #103
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
isaac-io
force-pushed
the
102-cmake-allow-running-the-tests-in-parallel-like-in-the-makefile
branch
4 times, most recently
from
89ab443 to
0c20a9f
Compare
This was referenced Aug 9, 2022
mrambacher
reviewed
Aug 18, 2022
cmake/CTestRunner.cmake
Outdated
Comment on lines
29
to
96
| if(NOT DEFINED ENV{TEST_TMPDIR} AND "${CMAKE_HOST_SYSTEM_NAME}" STREQUAL "Linux") | ||
| # Use /dev/shm if the sticky bit is set | ||
| if(IS_DIRECTORY /dev/shm) | ||
| execute_process(COMMAND test -k /dev/shm RESULT_VARIABLE status OUTPUT_QUIET ERROR_QUIET) | ||
| if(status EQUAL 0) | ||
| set(test_dir /dev/shm) | ||
| endif() | ||
| endif() | ||
| # Fall back to /tmp if exists | ||
| if(NOT DEFINED test_dir AND IS_DIRECTORY /tmp) | ||
| set(test_dir /tmp) | ||
| endif() | ||
| # Create a temporary directory under the test location that we determined | ||
| if(DEFINED test_dir) |
There was a problem hiding this comment.
This is not the same as the Makefile.
- I think tests always create a directory under TEST_TMPDIR of the form rocksdb.XXX.
- The Makefile does this on all non-Windows machines (not just Linux).
- If TEST_TMPDIR is not set, it should default to TMPDIR (or /tmp)
There was a problem hiding this comment.
- Tests create their directories in the following format:
${TEST_TMPDIR:-/tmp/rocksdbtest-$(id -u)}/<test_name>_<pid>_<tid>. This is not a safe thing to do when running tests in parallel due to PID and TID reuse (not to mention two tests runs on the same machine). - Right, but the Makefile is wrong.
/dev/shmis a Linux thing, not a POSIX thing, and many POSIX compliant OSs simply don't have it, or don't allow it to be used as a general purpose fileystem. I'll allow the rest of the temporary directory creation logic to work for other POSIX systems as well though. - Will do.
There was a problem hiding this comment.
- Tests create their directories in the following format:
${TEST_TMPDIR:-/tmp/rocksdbtest-$(id -u)}/<test_name>_<pid>_<tid>. This is not a safe thing to do when running tests in parallel due to PID and TID reuse (not to mention two tests runs on the same machine).
I understand this is not a safe thing to do. I actually put a PR into RocksDB to attempt to fix this, but it has not been accepted yet.
- Right, but the Makefile is wrong.
/dev/shmis a Linux thing, not a POSIX thing, and many POSIX compliant OSs simply don't have it, or don't allow it to be used as a general purpose fileystem. I'll allow the rest of the temporary directory creation logic to work for other POSIX systems as well though.
Agreed that /dev/shm is a Linux thing. But ALL of the code for creating rocksdbtest-* is under Linux and that is a Posix thing.
isaac-io
force-pushed
the
102-cmake-allow-running-the-tests-in-parallel-like-in-the-makefile
branch
5 times, most recently
from
88f8efa to
4b6e6a1
Compare
|
I am guessing this parallel test is invoked via J=# make check, but when I do that, I get an error: J=20 make check CMake Warning (dev) at /Users/markrambacher/Documents/GitHub/speedb/cmake/CTestRunner.cmake:19 (set): when parsing string syntax error, unexpected {, expecting } (6) Policy CMP0010 is not set: Bad variable reference syntax is an error. Run |
isaac-io
force-pushed
the
102-cmake-allow-running-the-tests-in-parallel-like-in-the-makefile
branch
from
4b6e6a1 to
a45bc46
Compare
|
Oops, my bad (a leftover from earlier experimentation). I pushed a fix for this. |
mrambacher
approved these changes
Aug 26, 2022
cmake/CTestRunner.cmake
Outdated
Comment on lines
47
to
52
| if(NOT DEFINED test_dir AND IS_DIRECTORY "$ENV{TMPDIR}") | ||
| set(test_dir $ENV{TMPDIR}) | ||
| endif() |
There was a problem hiding this comment.
On Windows, should this also check ENV{TMP}?
There was a problem hiding this comment.
According to a Raymond Chen post on the subject, we should probably check both TMP and TEMP on Windows. I'll add that (though the code in WinFileSystem::GetTestDirectory() already checks TMP on its own in case TEST_TMPDIR isn't set).
isaac-io
force-pushed
the
102-cmake-allow-running-the-tests-in-parallel-like-in-the-makefile
branch
2 times, most recently
from
0fcc3c6 to
59bb462
Compare
isaac-io
force-pushed
the
102-cmake-allow-running-the-tests-in-parallel-like-in-the-makefile
branch
from
59bb462 to
bc75b1b
Compare
isaac-io
force-pushed
the
102-cmake-allow-running-the-tests-in-parallel-like-in-the-makefile
branch
from
bc75b1b to
8e288ca
Compare
mrambacher
approved these changes
Sep 23, 2022
Allow running tests in parallel by default by setting the CTEST_PARALLEL_LEVEL environment variable. The functionality is added using a CMake script in order to determine options when running the tests (rather than when configuring CMake) in compatibility with the Makefile (in particular -- the `J` and `TEST_TMPDIR` environment variables). While at it, add a test timeout parameter in order to kill stuck tests, and set the default timeout to 10 minutes (we don't currently have tests that are running longer than that). Test Plan: run the check target using different generators (ninja, make) to see the improvement in test runtime.
isaac-io
force-pushed
the
102-cmake-allow-running-the-tests-in-parallel-like-in-the-makefile
branch
from
8e288ca to
4010bf5
Compare
isaac-io
deleted the
102-cmake-allow-running-the-tests-in-parallel-like-in-the-makefile
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allow running tests in parallel by default by setting the CTEST_PARALLEL_LEVEL
environment variable. The functionality is added using a CMake script in
order to determine options when running the tests (rather than when configuring
CMake) in compatibility with the Makefile (in particular -- the
JandTEST_TMPDIRenvironment variables).Test Plan: run the check target using different generators (ninja, make)
to see the improvement in test runtime.