Parallelize some tests #8078
Parallelize some tests #8078
Conversation
This reduces the time taken to run all correctness tests from 8:15 to 3:15 on my machine.
|
This failure puzzles me: https://buildbot.halide-lang.org/master/#/builders/74/builds/59 This is with a thread-pool of size 1, so it's not due to race conditions or anything. Turning off our stack switching for lowering with by setting HL_COMPILER_STACK_SIZE=0 doesn't heal it either, so it's not about that. This crash only happens using the cmake preset. If I just use gcc, clang++-15, or the same clang++-19 as the preset with the makefile build, adding -fsanitize=address everywhere, no errors. valgrind runs the test with no complaints Cargo culting from online, Setting ASAN_OPTIONS=use_sigaltstack=0 does heal it, but I don't know why. @steven-johnson have you seen this before? Any idea what's going on? |
|
No, unfortunately. ASAN can be weird sometimes. I could try pulling this into google3 and testing there. |
We don't have a custom error handler in place, so they're always zero
| std::vector<std::pair<int, string>> vl_params; | ||
| Expr f_1, f_2, f_3, u_1, i_1; | ||
| }; | ||
| // clang-format off | ||
| TestParams test_params[2] = { | ||
| {32, in_f32, {{1, "s"}, {2, ".2s"}, {4, ".4s"}, { 8, ".4s"}}, f32_1, f32_2, f32_3, u32_1, i32_1}, | ||
| {32, in_f32, {{1, "s"}, {2, ".4s"}, {4, ".4s"}, { 8, ".4s"}}, f32_1, f32_2, f32_3, u32_1, i32_1}, |
There was a problem hiding this comment.
What's with this change?
There was a problem hiding this comment.
Accidental commit. Will revert.
* Parallelize some tests This reduces the time taken to run all correctness tests from 8:15 to 3:15 on my machine. * The FIXME is actually fine * Remove debug print * Fix when we're willing to run x86 code in simd_op_check * Use separate imageparams per task * Deep-copy the LoopLevels * Make float16_t neon op check test at least build * Revert accidental serialization * Throw return values from callable into the void We don't have a custom error handler in place, so they're always zero * Skip test under ASAN * Fix unintentional change to test
This reduces the time taken to run all correctness tests from 8:15 to 3:15 on my machine.
It composes reasonably well with test sharding, but it's a bit weird that both mechanisms exist.
simd_op_check needed some reworking to avoid shared state between the threads.