Skip to content

memadvise.cpp test fails on CUDA BE after #3365 #3553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
vladimirlaz opened this issue Apr 16, 2021 · 7 comments
Closed

memadvise.cpp test fails on CUDA BE after #3365 #3553

vladimirlaz opened this issue Apr 16, 2021 · 7 comments
Assignees
@vladimirlaz
Labels
bug Something isn't working

Comments

@vladimirlaz
Copy link
Contributor

vladimirlaz commented Apr 16, 2021 •
edited
Loading

Describe the bug
intel/llvm-test-suite/blob/intel/SYCL/USM/memadvise.cpp test start to fail with the following output:

[2021-04-13T23:31:43.662Z] FAIL: SYCL :: USM/memadvise.cpp (481 of 501)

[2021-04-13T23:31:43.662Z] ******************** TEST 'SYCL :: USM/memadvise.cpp' FAILED ********************

[2021-04-13T23:31:43.662Z] Script:

[2021-04-13T23:31:43.662Z] --

[2021-04-13T23:31:43.662Z] : 'RUN: at line 1';    /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/llvm.obj/bin/clang++     -fsycl -fsycl-targets=nvptx64-nvidia-cuda-sycldevice /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/llvm-test-suite/SYCL/USM/memadvise.cpp -o /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out

[2021-04-13T23:31:43.662Z] : 'RUN: at line 2';   true /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out

[2021-04-13T23:31:43.662Z] : 'RUN: at line 3';   true /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out

[2021-04-13T23:31:43.662Z] : 'RUN: at line 4';    env SYCL_DEVICE_FILTER=cuda:gpu,host  /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out

[2021-04-13T23:31:43.662Z] --

[2021-04-13T23:31:43.662Z] Exit Code: -6

[2021-04-13T23:31:43.662Z] 

[2021-04-13T23:31:43.662Z] Command Output (stdout):

[2021-04-13T23:31:43.662Z] --

[2021-04-13T23:31:43.662Z] $ ":" "RUN: at line 1"

[2021-04-13T23:31:43.662Z] note: command had no output on stdout or stderr

[2021-04-13T23:31:43.662Z] $ "/workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/llvm.obj/bin/clang++" "-fsycl" "-fsycl-targets=nvptx64-nvidia-cuda-sycldevice" "/workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/llvm-test-suite/SYCL/USM/memadvise.cpp" "-o" "/workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out"

[2021-04-13T23:31:43.662Z] note: command had no output on stdout or stderr

[2021-04-13T23:31:43.662Z] $ ":" "RUN: at line 2"

[2021-04-13T23:31:43.662Z] note: command had no output on stdout or stderr

[2021-04-13T23:31:43.662Z] $ "true" "/workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out"

[2021-04-13T23:31:43.662Z] note: command had no output on stdout or stderr

[2021-04-13T23:31:43.662Z] $ ":" "RUN: at line 3"

[2021-04-13T23:31:43.662Z] note: command had no output on stdout or stderr

[2021-04-13T23:31:43.662Z] $ "true" "/workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out"

[2021-04-13T23:31:43.662Z] note: command had no output on stdout or stderr

[2021-04-13T23:31:43.662Z] $ ":" "RUN: at line 4"

[2021-04-13T23:31:43.662Z] note: command had no output on stdout or stderr

[2021-04-13T23:31:43.662Z] $ "env" "SYCL_DEVICE_FILTER=cuda:gpu,host" "/workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out"

[2021-04-13T23:31:43.662Z] # command stderr:

[2021-04-13T23:31:43.662Z] memadvise.cpp.tmp1.out: /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/llvm.src/sycl/plugins/cuda/pi_cuda.cpp:329: _pi_event::_pi_event(pi_command_type, pi_context, pi_queue): Assertion `type != PI_COMMAND_TYPE_USER' failed.

[2021-04-13T23:31:43.662Z] 

[2021-04-13T23:31:43.662Z] error: command failed with exit status: -6

[2021-04-13T23:31:43.662Z] 

[2021-04-13T23:31:43.662Z] --

[2021-04-13T23:31:43.662Z]

To Reproduce
The problem can be reproduced following instructions under https://github.com/intel/llvm-test-suite/blob/intel/SYCL/README.md

git clone https://github.com/intel/llvm-test-suite
cd llvm-test-suite
mkdir build
cd build
cmake -G "Ninja" -DCMAKE_CXX_COMPILER=<path_to_intel_sycl_compiler_with cuda_support>/clang++  -DTEST_SUITE_SUBDIRS=SYCL -DTEST_SUITE_LIT=<path_to_lit_fw>/lit.py -DCHECK_SYCL_ALL="cuda:gpu" -DTEST_SUITE_COLLECT_CODE_SIZE=OFF ..
# Build/run all tests
ninja check-sycl-all
# Run single test
llvm-lit --param target_devices=gpu --param sycl_be=cuda SYCL/USM/memadvise.cpp

Environment (please complete the following information):
@vladimirlaz vladimirlaz added the bug Something isn't working label Apr 16, 2021
@vladimirlaz vladimirlaz mentioned this issue Apr 16, 2021
Merged
@bader
Copy link
Contributor

bader commented Apr 16, 2021

@vladimirlaz, I expect 3269c47 to fix this bug. Could you check, please?

@vladimirlaz
Copy link
Contributor Author

according to postcommit testing for the commit. The test is still failing but fail symptoms changed to:

[2021-04-14T20:43:00.833Z] ******************** TEST 'SYCL :: USM/memadvise.cpp' FAILED ********************

[2021-04-14T20:43:00.833Z] Script:

[2021-04-14T20:43:00.833Z] --

[2021-04-14T20:43:00.833Z] : 'RUN: at line 1';    /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/llvm.obj/bin/clang++     -fsycl -fsycl-targets=nvptx64-nvidia-cuda-sycldevice /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/llvm-test-suite/SYCL/USM/memadvise.cpp -o /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out

[2021-04-14T20:43:00.833Z] : 'RUN: at line 2';   true /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out

[2021-04-14T20:43:00.834Z] : 'RUN: at line 3';   true /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out

[2021-04-14T20:43:00.834Z] : 'RUN: at line 4';    env SYCL_DEVICE_FILTER=cuda:gpu,host  /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out

[2021-04-14T20:43:00.834Z] --

[2021-04-14T20:43:00.834Z] Exit Code: -6

[2021-04-14T20:43:00.834Z] 

[2021-04-14T20:43:00.834Z] Command Output (stdout):

[2021-04-14T20:43:00.834Z] --

[2021-04-14T20:43:00.834Z] $ ":" "RUN: at line 1"

[2021-04-14T20:43:00.834Z] note: command had no output on stdout or stderr

[2021-04-14T20:43:00.834Z] $ "/workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/llvm.obj/bin/clang++" "-fsycl" "-fsycl-targets=nvptx64-nvidia-cuda-sycldevice" "/workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/llvm-test-suite/SYCL/USM/memadvise.cpp" "-o" "/workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out"

[2021-04-14T20:43:00.834Z] note: command had no output on stdout or stderr

[2021-04-14T20:43:00.834Z] $ ":" "RUN: at line 2"

[2021-04-14T20:43:00.834Z] note: command had no output on stdout or stderr

[2021-04-14T20:43:00.834Z] $ "true" "/workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out"

[2021-04-14T20:43:00.834Z] note: command had no output on stdout or stderr

[2021-04-14T20:43:00.834Z] $ ":" "RUN: at line 3"

[2021-04-14T20:43:00.834Z] note: command had no output on stdout or stderr

[2021-04-14T20:43:00.834Z] $ "true" "/workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out"

[2021-04-14T20:43:00.834Z] note: command had no output on stdout or stderr

[2021-04-14T20:43:00.834Z] $ ":" "RUN: at line 4"

[2021-04-14T20:43:00.834Z] note: command had no output on stdout or stderr

[2021-04-14T20:43:00.834Z] $ "env" "SYCL_DEVICE_FILTER=cuda:gpu,host" "/workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/build/SYCL/USM/Output/memadvise.cpp.tmp1.out"

[2021-04-14T20:43:00.834Z] # command stderr:

[2021-04-14T20:43:00.834Z] 

[2021-04-14T20:43:00.834Z] PI CUDA ERROR:

[2021-04-14T20:43:00.834Z] 	Value:           1

[2021-04-14T20:43:00.834Z] 	Name:            CUDA_ERROR_INVALID_VALUE

[2021-04-14T20:43:00.834Z] 	Description:     invalid argument

[2021-04-14T20:43:00.834Z] 	Function:        cuda_piextUSMEnqueueMemAdvise

[2021-04-14T20:43:00.834Z] 	Source Location: /workspace/SYCL_CI/intel/Lin/LLVM_Test_Suite_CUDA/llvm.src/sycl/plugins/cuda/pi_cuda.cpp:4465

[2021-04-14T20:43:00.834Z] 

[2021-04-14T20:43:00.834Z] terminate called after throwing an instance of 'cl::sycl::runtime_error'

[2021-04-14T20:43:00.834Z]   what():  Native API failed. Native API returns: -30 (CL_INVALID_VALUE) -30 (CL_INVALID_VALUE)

[2021-04-14T20:43:00.834Z] 

[2021-04-14T20:43:00.834Z] error: command failed with exit status: -6

[2021-04-14T20:43:00.834Z] 

[2021-04-14T20:43:00.834Z] --

[2021-04-14T20:43:00.834Z]

@bader
Copy link
Contributor

bader commented Apr 16, 2021

@nyalloc, does this test work for you?

@nyalloc
Copy link
Contributor

nyalloc commented Apr 19, 2021 •
edited
Loading

Memadvise test is not correct. The value passed into memadvise is backend-specific, this test assumes 0 is a valid value. For CUDA this is not the case. The valid values for CUDA are 1 through 6.

I think it would be a good idea to modify these tests so that the enumerations passed to memadvise are driven through the preprocessor, so if the tests are targetting CUDA, pass one of the valid enumerations for CUDA, if if the tests are targetting OpenCL, pass one of the valid enumerations for OpenCL, etc.

@bader
Copy link
Contributor

bader commented Apr 20, 2021

@nyalloc, thanks for sharing this insight.

@vladimirlaz, could you update the test, please?

@nyalloc
Copy link
Contributor

nyalloc commented Apr 20, 2021

No problem

@vladimirlaz
Copy link
Contributor Author

Fixed in intel/llvm-test-suite#240

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vladimirlaz vladimirlaz

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@bader @nyalloc @vladimirlaz