[TESTS] Refactor tests to run on either the GPU or CPU #6331
Conversation
tkonolige
force-pushed
the
test_markers
branch
3 times, most recently
from
e64a748
to
324066a
Compare
|
It might also be helpful to send a quick RFC given that the change of test infra will affect quite a lot of people |
|
I agree that an RFC could be useful, but maybe I could just add information to the docs instead? |
There was a problem hiding this comment.
Please keep in mind that eventually the mapping target_name -> device will no longer be sufficient. There can be several devices with different characteristics that may all fall under the same general target (e.g. gpu), and that any individual test should be able to specify required device properties.
The use of strings to pass targets around may be phased out at some point, please document somewhere the necessary updates to this scheme when that happens.
|
@kparzysz-quic I agree that using strings for targets here is pretty awkward. Is there a better way to check which device a target requires? I see that #6315 makes targets more structured, but I don't see a way to take a target string and get the associated device. |
|
Currently there is no good way to switch away from |
|
@tqchen I think I've addressed all issues. Do I need to do something to have Jenkins use the new Jenkinsfile? |
|
@tkonolige because uses the old Jenkinsfile (before it get merged). Please try to send another PR to add |
tkonolige
force-pushed
the
test_markers
branch
3 times, most recently
from
89f2623
to
64265ae
Compare
|
@tqchen I'm getting a couple errors with CUDA initialization failing. I'm really sure of the cause, but it seems like it might have to do with forking. |
tkonolige
force-pushed
the
test_markers
branch
10 times, most recently
from
a5abf4f
to
f18f71b
Compare
tkonolige
force-pushed
the
test_markers
branch
2 times, most recently
from
ab0b4bc
to
bd404c6
Compare
Much of the time spent in testing is duplicated work between CPU and GPU test nodes. The main reason is that there is no way to control which TVM devices are enabled at runtime, so tests that use LLVM will run on both GPU and CPU nodes. This patch adds an environment variable, TVM_TEST_DEVICES, which controls which TVM devices should be used by tests. Devices not in TVM_TEST_DEVICES can still be used, so tests must be careful to check that the desired device is enabled with `tvm.testing.device_enabled` or by enumerating all devices with `tvm.testing.enabled_devices`. All tests have been retrofitted with these checks. This patch also provides the decorator `@tvm.testing.gpu` to mark a test as possibly using the gpu. Tests that require the gpu can use `@tvm.testing.requires_gpu`. Tests without these flags will not be run on GPU nodes.
tkonolige
force-pushed
the
test_markers
branch
from
bd404c6
to
574a7dd
Compare
|
@tqchen All tests are passing now... can we merge? |
|
Thanks @tkonolige @zhiics @kparzysz-quic ! |
Much of the time spent in testing is duplicated work between CPU and GPU test nodes. The main reason is that there is no way to control which TVM devices are enabled at runtime, so tests that use LLVM will run on both GPU and CPU nodes. This patch adds an environment variable, TVM_TEST_DEVICES, which controls which TVM devices should be used by tests. Devices not in TVM_TEST_DEVICES can still be used, so tests must be careful to check that the desired device is enabled with `tvm.testing.device_enabled` or by enumerating all devices with `tvm.testing.enabled_devices`. All tests have been retrofitted with these checks. This patch also provides the decorator `@tvm.testing.gpu` to mark a test as possibly using the gpu. Tests that require the gpu can use `@tvm.testing.requires_gpu`. Tests without these flags will not be run on GPU nodes.
Much of the time spent in testing is duplicated work between CPU and GPU test nodes. The main reason is that there is no way to control which TVM devices are enabled at runtime, so tests that use LLVM will run on both GPU and CPU nodes. This patch adds an environment variable, TVM_TEST_DEVICES, which controls which TVM devices should be used by tests. Devices not in TVM_TEST_DEVICES can still be used, so tests must be careful to check that the desired device is enabled with `tvm.testing.device_enabled` or by enumerating all devices with `tvm.testing.enabled_devices`. All tests have been retrofitted with these checks. This patch also provides the decorator `@tvm.testing.gpu` to mark a test as possibly using the gpu. Tests that require the gpu can use `@tvm.testing.requires_gpu`. Tests without these flags will not be run on GPU nodes.
Much of the time spent in testing is duplicated work between CPU and GPU test nodes. The main reason is that there is no way to control which TVM devices are enabled at runtime, so tests that use LLVM will run on both GPU and CPU nodes. This patch adds an environment variable, TVM_TEST_DEVICES, which controls which TVM devices should be used by tests. Devices not in TVM_TEST_DEVICES can still be used, so tests must be careful to check that the desired device is enabled with `tvm.testing.device_enabled` or by enumerating all devices with `tvm.testing.enabled_devices`. All tests have been retrofitted with these checks. This patch also provides the decorator `@tvm.testing.gpu` to mark a test as possibly using the gpu. Tests that require the gpu can use `@tvm.testing.requires_gpu`. Tests without these flags will not be run on GPU nodes.
Much of the time spent in testing is duplicated work between CPU and GPU test nodes. The main reason is that there is no way to control which TVM devices are enabled at runtime, so tests that use LLVM will run on both GPU and CPU nodes.
This patch adds an environment variable,
TVM_TEST_DEVICES, which controls which TVM devices should be used by tests. Devices not inTVM_TEST_DEVICEScan still be used, so tests must be careful to check that the desired device is enabled withtvm.testing.device_enabledor by enumerating all devices withtvm.testing.enabled_devices. All tests have been retrofitted with these checks.This patch also provides the decorator
@tvm.testing.gputo mark a test as possibly using the gpu. Tests that require the gpu can use@tvm.testing.requires_gpu. Tests without these flags will not be run on GPU nodes.