[Vulkan][Topi] Parametrizing additional topi tests, marking vulkan failures #8904
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Lunderberg
requested review from
areusch,
comaniac,
jroesch,
junrushao,
kparzysz-quic,
masahi,
merrymercy,
tqchen,
vinx13,
yzhliu and
ZihengJiang
as code owners
Previously, would fail during build since support for Int64 primitives wasn't declared in the `"vulkan"` target. Now, uses `"vulkan -from_device=0"` target and marks the test as xfail if the current target doesn't support Int64.
Lunderberg
force-pushed
the
unittest_vulkan
branch
from
ffec6ba
to
c633209
Compare
tkonolige
approved these changes
Sep 1, 2021
Comment on lines
67
to
74
| # @tvm.testing.parametrize_targets | ||
| # def test_nll_loss(dev, target): | ||
| # verify_nll_loss(dev, target, (10, 5)) | ||
| # verify_nll_loss(dev, target, (10, 5, 2, 2)) | ||
| # verify_nll_loss(dev, target, (10, 5), reduction="sum") | ||
| # verify_nll_loss(dev, target, (10, 5), reduction="none") | ||
| # verify_nll_loss(dev, target, (10, 5), ignore_index=3) | ||
| # verify_nll_loss(dev, target, (10, 5), dtype="float64") |
There was a problem hiding this comment.
Maybe delete these instead? Or mark them as known failing?
There was a problem hiding this comment.
Whoops, those should have been removed entirely. Thank you for the catch.
…lures
- Parametrized topi modules
- test_topi_conv1d_transpose_ncw.py
- test_topi_conv2d_nhwc.py
- test_topi_correlation.py
- test_topi_loss.py
- test_topi_math.py
- test_topi_reduce.py
- test_topi_softmax.py
- test_topi_sort.py
- test_topi_unique.py
- test_topi_vision.py
- Unit Tests fixed
- `test_topi_loss::test_nll_loss`, failure due to `supports_float64`
not being passed from the target to the codegen.
- Known Vulkan failures (tracked in apache#8903)
- test_topi_math.py::test_ewise, ["tan", "erf", "isnan", "isfinite", "isinf"]
Unimplemented CallNode operations
- test_topi_reduce.py::test_reduce_map, ["sum", "any", "all"]
Fails during codegen, unexpected size of data type.
- test_topi_vision.py::test_proposal
Marked test_proposal as xfail on vulkan, currently has a type error
between bool/int8.
- test_topi_conv1d_transpose_ncw.py::test_conv1d_transpose_ncw
Incorrect numeric output, a few elements outside of allowed
tolerance, only occurs on vulkan backend.
- test_softmax.py::test_softmax
Marked float64 operations as xfail in vulkan, because GLSL.std.450
only supports 16/32-bit floats.
Lunderberg
force-pushed
the
unittest_vulkan
branch
from
c633209
to
d1860e1
Compare
masahi
approved these changes
Sep 2, 2021
|
Thanks @Lunderberg @tkonolige |
AndrewZhaoLuo
added a commit
to AndrewZhaoLuo/tvm
that referenced
this pull request
Sep 2, 2021
* main: [UnitTests][Contrib] Enable contrib tensorrt/coreml unit tests (apache#8902) [BUG] DataType Bug In SplitRel (apache#8899) Enable python debug runtime for exported network libraries (apache#8793) Set default value of p in LpPool as 2 (apache#8866) [Community] @Hzfengsy -> Committer (apache#8908) Trivial uTVM -> microTVM "spelling" fix to align with branding. (apache#8905) [Vulkan][Topi] Parametrizing additional topi tests, marking vulkan failures (apache#8904) Move to new style issue template system (apache#8898) [Onnx] Support Negative Log Loss (apache#8872) [ROCm][TVMC] Add ROCm to the TVMC driver (apache#8896) fix error report on Store (apache#8895) [Docker] Re-enabled automatic --tty flag when running bash. (apache#8861)
ylc
pushed a commit
to ylc/tvm
that referenced
this pull request
Sep 29, 2021
…ilures (apache#8904) * [Pytest] Fixed TestTargetAutoParametrization in cases where LLVM is disabled. * [UnitTests][Vulkan] Improved robustness of test_tir_intrin::test_clz Previously, would fail during build since support for Int64 primitives wasn't declared in the `"vulkan"` target. Now, uses `"vulkan -from_device=0"` target and marks the test as xfail if the current target doesn't support Int64. * [UnitTest][Topi] Parametrized several unit tests, identify vulkan failures - Parametrized topi modules - test_topi_conv1d_transpose_ncw.py - test_topi_conv2d_nhwc.py - test_topi_correlation.py - test_topi_loss.py - test_topi_math.py - test_topi_reduce.py - test_topi_softmax.py - test_topi_sort.py - test_topi_unique.py - test_topi_vision.py - Unit Tests fixed - `test_topi_loss::test_nll_loss`, failure due to `supports_float64` not being passed from the target to the codegen. - Known Vulkan failures (tracked in apache#8903) - test_topi_math.py::test_ewise, ["tan", "erf", "isnan", "isfinite", "isinf"] Unimplemented CallNode operations - test_topi_reduce.py::test_reduce_map, ["sum", "any", "all"] Fails during codegen, unexpected size of data type. - test_topi_vision.py::test_proposal Marked test_proposal as xfail on vulkan, currently has a type error between bool/int8. - test_topi_conv1d_transpose_ncw.py::test_conv1d_transpose_ncw Incorrect numeric output, a few elements outside of allowed tolerance, only occurs on vulkan backend. - test_softmax.py::test_softmax Marked float64 operations as xfail in vulkan, because GLSL.std.450 only supports 16/32-bit floats.
ylc
pushed a commit
to ylc/tvm
that referenced
this pull request
Jan 13, 2022
…ilures (apache#8904) * [Pytest] Fixed TestTargetAutoParametrization in cases where LLVM is disabled. * [UnitTests][Vulkan] Improved robustness of test_tir_intrin::test_clz Previously, would fail during build since support for Int64 primitives wasn't declared in the `"vulkan"` target. Now, uses `"vulkan -from_device=0"` target and marks the test as xfail if the current target doesn't support Int64. * [UnitTest][Topi] Parametrized several unit tests, identify vulkan failures - Parametrized topi modules - test_topi_conv1d_transpose_ncw.py - test_topi_conv2d_nhwc.py - test_topi_correlation.py - test_topi_loss.py - test_topi_math.py - test_topi_reduce.py - test_topi_softmax.py - test_topi_sort.py - test_topi_unique.py - test_topi_vision.py - Unit Tests fixed - `test_topi_loss::test_nll_loss`, failure due to `supports_float64` not being passed from the target to the codegen. - Known Vulkan failures (tracked in apache#8903) - test_topi_math.py::test_ewise, ["tan", "erf", "isnan", "isfinite", "isinf"] Unimplemented CallNode operations - test_topi_reduce.py::test_reduce_map, ["sum", "any", "all"] Fails during codegen, unexpected size of data type. - test_topi_vision.py::test_proposal Marked test_proposal as xfail on vulkan, currently has a type error between bool/int8. - test_topi_conv1d_transpose_ncw.py::test_conv1d_transpose_ncw Incorrect numeric output, a few elements outside of allowed tolerance, only occurs on vulkan backend. - test_softmax.py::test_softmax Marked float64 operations as xfail in vulkan, because GLSL.std.450 only supports 16/32-bit floats.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR aims to make the unit test suite run with
TVM_TEST_TARGETS="vulkan -from_device=0". This PR applies a few minor fixes, parametrizes several topi unit tests that were previously failing, and marks the specific failing cases with xfail. The failing cases will be tracked in #8903.[UnitTest][Topi] Parametrized several unit tests, identify vulkan failures
Parametrized topi modules
Unit Tests fixed
test_topi_loss::test_nll_loss, failure due tosupports_float64not being passed from the target to the codegen.Known Vulkan failures (tracked in [Tracking][Vulkan] Extending topi/relay tests to run on Vulkan #8903)
test_topi_math.py::test_ewise, ["tan", "erf", "isnan", "isfinite", "isinf"]
Unimplemented CallNode operations
test_topi_reduce.py::test_reduce_map, ["sum", "any", "all"]
Fails during codegen, unexpected size of data type.
test_topi_vision.py::test_proposal
Marked test_proposal as xfail on vulkan, currently has a type error between bool/int8.
test_topi_conv1d_transpose_ncw.py::test_conv1d_transpose_ncw
Incorrect numeric output, a few elements outside of allowed tolerance, only occurs on vulkan backend.
test_softmax.py::test_softmax
Marked float64 operations as xfail in vulkan, because GLSL.std.450 only supports 16/32-bit floats.
[UnitTests][Vulkan] Improved robustness of test_tir_intrin::test_clz
Previously, would fail during build since support for Int64 primitives wasn't declared in the
"vulkan"target. Now, uses"vulkan -from_device=0"target and marks the test as xfail if the current target doesn't support Int64.[Pytest] Fixed TestTargetAutoParametrization in cases where LLVM is disabled.