Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test CUDA compilation on Windows #115

Closed
wants to merge 18 commits into from

Conversation

traversaro
Copy link
Contributor

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

@traversaro
Copy link
Contributor Author

@conda-forge-admin, please rerender

@conda-forge-webservices
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe:

conda-forge-webservices[bot] and others added 2 commits May 8, 2024 09:25
@traversaro
Copy link
Contributor Author

@conda-forge-admin, please rerender

@traversaro
Copy link
Contributor Author

traversaro commented May 16, 2024

CUDA 12 error:

(%BUILD_PREFIX%) %SRC_DIR%>python tools/ci_build/build.py     --compile_no_warning_as_error     --build_dir build-ci     --cmake_extra_defines EIGEN_MPL2_ONLY=ON "onnxruntime_USE_COREML=OFF" "onnxruntime_BUILD_SHARED_LIB=ON" "onnxruntime_BUILD_UNIT_TESTS=ON" CMAKE_PREFIX_PATH=%PREFIX%\Library CMAKE_INSTALL_PREFIX=%PREFIX%\Library CMAKE_DISABLE_FIND_PACKAGE_Protobuf=ON CMAKE_CUDA_ARCHITECTURES=all     --cmake_generator Ninja     --build_wheel     --config Release     --update     --build     --skip_submodule_sync     --use_cuda --cudnn_home %PREFIX%\Library 
2024-05-08 09:44:58,414 build [DEBUG] - Command line arguments:
  --compile_no_warning_as_error --build_dir build-ci --cmake_extra_defines EIGEN_MPL2_ONLY=ON onnxruntime_USE_COREML=OFF onnxruntime_BUILD_SHARED_LIB=ON onnxruntime_BUILD_UNIT_TESTS=ON 'CMAKE_PREFIX_PATH=D:\bld\onnxruntime_1715161041292\_h_env\Library' 'CMAKE_INSTALL_PREFIX=D:\bld\onnxruntime_1715161041292\_h_env\Library' CMAKE_DISABLE_FIND_PACKAGE_Protobuf=ON CMAKE_CUDA_ARCHITECTURES=all --cmake_generator Ninja --build_wheel --config Release --update --build --skip_submodule_sync --use_cuda --cudnn_home 'D:\bld\onnxruntime_1715161041292\_h_env\Library'
Namespace(build_dir='build-ci', config=['Release'], update=True, build=True, clean=False, parallel=1, nvcc_threads=-1, test=False, skip_tests=False, compile_no_warning_as_error=True, enable_nvtx_profile=False, enable_memory_profile=False, enable_training=False, enable_training_apis=False, enable_training_ops=False, enable_nccl=False, mpi_home=None, nccl_home=None, use_mpi=False, enable_onnx_tests=False, path_to_protoc_exe=None, fuzz_testing=False, enable_symbolic_shape_infer_tests=False, gen_doc=None, gen_api_doc=False, use_cuda=True, cuda_version=None, cuda_home=None, cudnn_home='D:\\bld\\onnxruntime_1715161041292\\_h_env\\Library', enable_cuda_line_info=False, enable_cuda_nhwc_ops=False, enable_pybind=False, build_wheel=True, wheel_name_suffix=None, numpy_version=None, skip_keras_test=False, build_csharp=False, build_nuget=False, msbuild_extra_options=None, build_java=False, build_nodejs=False, build_objc=False, build_shared_lib=False, build_apple_framework=False, cmake_extra_defines=[['EIGEN_MPL2_ONLY=ON', 'onnxruntime_USE_COREML=OFF', 'onnxruntime_BUILD_SHARED_LIB=ON', 'onnxruntime_BUILD_UNIT_TESTS=ON', 'CMAKE_PREFIX_PATH=D:\\bld\\onnxruntime_1715161041292\\_h_env\\Library', 'CMAKE_INSTALL_PREFIX=D:\\bld\\onnxruntime_1715161041292\\_h_env\\Library', 'CMAKE_DISABLE_FIND_PACKAGE_Protobuf=ON', 'CMAKE_CUDA_ARCHITECTURES=all']], target=None, x86=False, arm=False, arm64=False, arm64ec=False, buildasx=False, msvc_toolset=None, windows_sdk_version=None, android=False, android_abi='arm64-v8a', android_api=27, android_sdk_path='C:\\Android\\android-sdk', android_ndk_path='C:\\Android\\android-sdk\\ndk\\25.2.9519653', android_cpp_shared=False, android_run_emulator=False, use_gdk=False, gdk_edition='.', gdk_platform='Scarlett', ios=False, apple_sysroot='', ios_toolchain_file='', xcode_code_signing_team_id='', xcode_code_signing_identity='', cmake_generator='Ninja', osx_arch='x86_64', apple_deploy_target=None, enable_address_sanitizer=False, enable_qspectre=False, disable_memleak_checker=False, build_wasm=False, build_wasm_static_lib=False, emsdk_version='3.1.51', enable_wasm_simd=False, enable_wasm_threads=False, disable_wasm_exception_catching=False, enable_wasm_api_exception_catching=False, enable_wasm_exception_throwing_override=True, wasm_run_tests_in_browser=False, enable_wasm_profiling=False, enable_wasm_debug_info=False, wasm_malloc=None, emscripten_settings=None, use_extensions=False, extensions_overridden_path=None, cmake_path='cmake', ctest_path='ctest', skip_submodule_sync=True, use_mimalloc=False, use_dnnl=False, dnnl_gpu_runtime='', dnnl_opencl_root='', use_openvino=None, dnnl_aarch64_runtime='', dnnl_acl_root='', use_coreml=False, use_webnn=False, use_snpe=False, snpe_root=None, use_nnapi=False, nnapi_min_api=None, use_jsep=False, use_qnn=False, qnn_home=None, use_rknpu=False, use_preinstalled_eigen=False, eigen_path=None, enable_msinternal=False, llvm_path=None, use_vitisai=False, use_tvm=False, tvm_cuda_runtime=False, use_tvm_hash=False, use_tensorrt=False, use_tensorrt_builtin_parser=True, use_tensorrt_oss_parser=False, tensorrt_home=None, test_all_timeout='10800', use_migraphx=False, migraphx_home=None, use_full_protobuf=False, llvm_config='', skip_onnx_tests=False, skip_winml_tests=False, skip_nodejs_tests=False, enable_msvc_static_runtime=False, enable_language_interop_ops=False, use_dml=False, dml_path='', use_winml=False, winml_root_namespace_override=None, dml_external_project=False, use_telemetry=False, enable_wcos=False, enable_lto=False, enable_transformers_tool_test=False, use_acl=None, acl_home=None, acl_libs=None, use_armnn=False, armnn_relu=False, armnn_bn=False, armnn_home=None, armnn_libs=None, build_micro_benchmarks=False, minimal_build=None, include_ops_by_config=None, enable_reduced_operator_type_support=False, disable_contrib_ops=False, disable_ml_ops=False, disable_rtti=False, disable_types=[], disable_exceptions=False, rocm_version=None, use_rocm=False, rocm_home=None, code_coverage=False, enable_lazy_tensor=False, ms_experimental=False, enable_external_custom_op_schemas=False, external_graph_transformer_path=None, enable_cuda_profiling=False, use_cann=False, cann_home=None, enable_rocm_profiling=False, use_xnnpack=False, use_azure=False, use_cache=False, use_triton_kernel=False, use_lock_free_queue=False)
2024-05-08 09:44:58,508 build [ERROR] - cuda_home and cudnn_home paths must be specified and valid.
cuda_home='None' valid=False. cudnn_home='D:\bld\onnxruntime_1715161041292\_h_env\Library' valid=True

Usual CUDA 12 migration problem.

CUDA 11.8 error:

-- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/include (found version "11.8.89")
-- The CUDA compiler identification is NVIDIA 11.8.89
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - failed
-- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/bin/nvcc.exe
CMake Error at D:/bld/onnxruntime_1715162047283/_build_env/Library/share/cmake-3.29/Modules/CMakeTestCUDACompiler.cmake:59 (message):
-- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/bin/nvcc.exe - broken
  The CUDA compiler

    "C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/bin/nvcc.exe"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: 'D:/bld/onnxruntime_1715162047283/work/build-ci/Release/CMakeFiles/CMakeScratch/TryCompile-z6ph5l'
    
    Run Build Command(s): D:/bld/onnxruntime_1715162047283/_build_env/Library/bin/ninja.exe -v cmTC_da649
    [1/2] C:\PROGRA~1\NVIDIA~2\CUDA\v11.8\bin\nvcc.exe -forward-unknown-to-host-compiler   /DWIN32 /D_WINDOWS /DWINAPI_FAMILY=100 /DWINVER=0x0A00 /D_WIN32_WINNT=0x0A00 /DNTDDI_VERSION=0x0A000000 /DONNXRUNTIME_ENABLE_INTEL_METEOR_LAKE_MOBILE_PLATFORM_PERF_PATCH /DNDEBUG -Xcompiler=" /guard:cf /MP /Qspectre /O2 /Ob2"  -D_WINDOWS -Xcompiler=" /EHsc"  -Xcompiler=" -Ob0 -Od /RTC1" -arch=all -Xcompiler=-MDd -Xcompiler=-Zi -MD -MT CMakeFiles\cmTC_da649.dir\main.cu.obj -MF CMakeFiles\cmTC_da649.dir\main.cu.obj.d -x cu -c D:\bld\onnxruntime_1715162047283\work\build-ci\Release\CMakeFiles\CMakeScratch\TryCompile-z6ph5l\main.cu -o CMakeFiles\cmTC_da649.dir\main.cu.obj -Xcompiler=-FdCMakeFiles\cmTC_da649.dir\,-FS
    FAILED: CMakeFiles/cmTC_da649.dir/main.cu.obj 
    C:\PROGRA~1\NVIDIA~2\CUDA\v11.8\bin\nvcc.exe -forward-unknown-to-host-compiler   /DWIN32 /D_WINDOWS /DWINAPI_FAMILY=100 /DWINVER=0x0A00 /D_WIN32_WINNT=0x0A00 /DNTDDI_VERSION=0x0A000000 /DONNXRUNTIME_ENABLE_INTEL_METEOR_LAKE_MOBILE_PLATFORM_PERF_PATCH /DNDEBUG -Xcompiler=" /guard:cf /MP /Qspectre /O2 /Ob2"  -D_WINDOWS -Xcompiler=" /EHsc"  -Xcompiler=" -Ob0 -Od /RTC1" -arch=all -Xcompiler=-MDd -Xcompiler=-Zi -MD -MT CMakeFiles\cmTC_da649.dir\main.cu.obj -MF CMakeFiles\cmTC_da649.dir\main.cu.obj.d -x cu -c D:\bld\onnxruntime_1715162047283\work\build-ci\Release\CMakeFiles\CMakeScratch\TryCompile-z6ph5l\main.cu -o CMakeFiles\cmTC_da649.dir\main.cu.obj -Xcompiler=-FdCMakeFiles\cmTC_da649.dir\,-FS
    nvcc warning : The 'compute_35', 'compute_37', 'sm_35', and 'sm_37' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
    nvcc fatal   : A single input file is required for a non-link phase when an outputfile is specified

    return _func_defaulting_env_to_os_environ("call", *popenargs, **kwargs)
  File "C:\Miniforge\lib\site-packages\conda_build\utils.py", line 384, in _func_defaulting_env_to_os_environ
    raise subprocess.CalledProcessError(proc.returncode, _args)
subprocess.CalledProcessError: Command '['cmd.exe', '/d', '/c', 'conda_build.bat']' returned non-zero exit status 1.

@traversaro
Copy link
Contributor Author

CUDA 11.8 error:

Same error of #63 (comment), however now CUDA 11.8 should still be supported. Anyhow, it make sense to first concentrate on CUDA 12.0 .

@traversaro
Copy link
Contributor Author

@traversaro
Copy link
Contributor Author

@conda-forge-admin, please rerender

@traversaro
Copy link
Contributor Author

Now the builds (also the non-CUDA one) fails with:

CMake Error at D:/bld/onnxruntime_1715858000512/_build_env/Library/share/cmake-3.29/Modules/CMakeDetermineCCompiler.cmake:49 (message):
  Could not find compiler set in environment variable CC:
-- Configuring incomplete, errors occurred!

  cl.exe.
Call Stack (most recent call first):
  CMakeLists.txt:20 (project)

this smells like an environment regression, for example in CI images or similar.

@traversaro traversaro mentioned this pull request May 16, 2024
5 tasks
@traversaro traversaro closed this May 20, 2024
@traversaro traversaro reopened this May 20, 2024
@conda-forge-webservices
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe:

  • This recipe is using a compiler, which now requires adding a build dependence on {{ stdlib("c") }} as well. For further details, please see META: {{ stdlib("c") }} migration conda-forge.github.io#2102.
  • In your conda_build_config.yaml, please change the name of MACOSX_DEPLOYMENT_TARGET, to c_stdlib_version!

@traversaro
Copy link
Contributor Author

Also CUDA 12 fails with:

  It fails with the following output:

    Change Dir: 'D:/bld/onnxruntime_1716234839539/work/build-ci/Release/CMakeFiles/CMakeScratch/TryCompile-fjxebz'
    
    Run Build Command(s): D:/bld/onnxruntime_1716234839539/_build_env/Library/bin/ninja.exe -v cmTC_dd5bb
    [1/2] D:\bld\onnxruntime_1716234839539\_build_env\Library\bin\nvcc.exe -forward-unknown-to-host-compiler   /DWIN32 /D_WINDOWS /DWINAPI_FAMILY=100 /DWINVER=0x0A00 /D_WIN32_WINNT=0x0A00 /DNTDDI_VERSION=0x0A000000 /DONNXRUNTIME_ENABLE_INTEL_METEOR_LAKE_MOBILE_PLATFORM_PERF_PATCH /DNDEBUG -Xcompiler=" /guard:cf /MP /Qspectre /O2 /Ob2"  -D_WINDOWS -Xcompiler=" /EHsc"  -Xcompiler=" -Ob0 -Od /RTC1" -arch=all -Xcompiler=-MDd -Xcompiler=-Zi -MD -MT CMakeFiles\cmTC_dd5bb.dir\main.cu.obj -MF CMakeFiles\cmTC_dd5bb.dir\main.cu.obj.d -x cu -c D:\bld\onnxruntime_1716234839539\work\build-ci\Release\CMakeFiles\CMakeScratch\TryCompile-fjxebz\main.cu -o CMakeFiles\cmTC_dd5bb.dir\main.cu.obj -Xcompiler=-FdCMakeFiles\cmTC_dd5bb.dir\,-FS
    FAILED: CMakeFiles/cmTC_dd5bb.dir/main.cu.obj 
    D:\bld\onnxruntime_1716234839539\_build_env\Library\bin\nvcc.exe -forward-unknown-to-host-compiler   /DWIN32 /D_WINDOWS /DWINAPI_FAMILY=100 /DWINVER=0x0A00 /D_WIN32_WINNT=0x0A00 /DNTDDI_VERSION=0x0A000000 /DONNXRUNTIME_ENABLE_INTEL_METEOR_LAKE_MOBILE_PLATFORM_PERF_PATCH /DNDEBUG -Xcompiler=" /guard:cf /MP /Qspectre /O2 /Ob2"  -D_WINDOWS -Xcompiler=" /EHsc"  -Xcompiler=" -Ob0 -Od /RTC1" -arch=all -Xcompiler=-MDd -Xcompiler=-Zi -MD -MT CMakeFiles\cmTC_dd5bb.dir\main.cu.obj -MF CMakeFiles\cmTC_dd5bb.dir\main.cu.obj.d -x cu -c D:\bld\onnxruntime_1716234839539\work\build-ci\Release\CMakeFiles\CMakeScratch\TryCompile-fjxebz\main.cu -o CMakeFiles\cmTC_dd5bb.dir\main.cu.obj -Xcompiler=-FdCMakeFiles\cmTC_dd5bb.dir\,-FS
    nvcc fatal   : A single input file is required for a non-link phase when an outputfile is specified

    ninja: build stopped: subcommand failed.
    
    

  

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:705 (enable_language)

@traversaro
Copy link
Contributor Author

Probably there is some argument that should not be there, that is then interpreter by the compiler as an input file, generating the error (see https://forums.developer.nvidia.com/t/solved-nvcc-fatal-a-single-input-file-is-required-for-a-non-link-phase-when-an-outputfile-is-s/37807 for a similar error). I can easily check this locally to understand which command is wrong.

@traversaro
Copy link
Contributor Author

Now the builds (also the non-CUDA one) fails with:

CMake Error at D:/bld/onnxruntime_1715858000512/_build_env/Library/share/cmake-3.29/Modules/CMakeDetermineCCompiler.cmake:49 (message):
  Could not find compiler set in environment variable CC:
-- Configuring incomplete, errors occurred!

  cl.exe.
Call Stack (most recent call first):
  CMakeLists.txt:20 (project)

this smells like an environment regression, for example in CI images or similar.

By the way, this was fixed by conda-forge/vc-feedstock#76, thanks @mgovers!

@traversaro
Copy link
Contributor Author

The problem in:

    [1/2] D:\bld\onnxruntime_1716234839539\_build_env\Library\bin\nvcc.exe -forward-unknown-to-host-compiler   /DWIN32 /D_WINDOWS /DWINAPI_FAMILY=100 /DWINVER=0x0A00 /D_WIN32_WINNT=0x0A00 /DNTDDI_VERSION=0x0A000000 /DONNXRUNTIME_ENABLE_INTEL_METEOR_LAKE_MOBILE_PLATFORM_PERF_PATCH /DNDEBUG -Xcompiler=" /guard:cf /MP /Qspectre /O2 /Ob2"  -D_WINDOWS -Xcompiler=" /EHsc"  -Xcompiler=" -Ob0 -Od /RTC1" -arch=all -Xcompiler=-MDd -Xcompiler=-Zi -MD -MT CMakeFiles\cmTC_dd5bb.dir\main.cu.obj -MF CMakeFiles\cmTC_dd5bb.dir\main.cu.obj.d -x cu -c D:\bld\onnxruntime_1716234839539\work\build-ci\Release\CMakeFiles\CMakeScratch\TryCompile-fjxebz\main.cu -o CMakeFiles\cmTC_dd5bb.dir\main.cu.obj -Xcompiler=-FdCMakeFiles\cmTC_dd5bb.dir\,-FS

Are the /DWIN32 /D_WINDOWS /DWINAPI_FAMILY=100 /DWINVER=0x0A00 /D_WIN32_WINNT=0x0A00 /DNTDDI_VERSION=0x0A000000 /DONNXRUNTIME_ENABLE_INTEL_METEOR_LAKE_MOBILE_PLATFORM_PERF_PATCH /DNDEBUG options, manually converting them to -DWIN32 -D_WINDOWS -DWINAPI_FAMILY=100 -DWINVER=0x0A00 -D_WIN32_WINNT=0x0A00 -DNTDDI_VERSION=0x0A000000 -DONNXRUNTIME_ENABLE_INTEL_METEOR_LAKE_MOBILE_PLATFORM_PERF_PATCH -DNDEBUG everything compiles fine (actually, at least the error is different).

@traversaro
Copy link
Contributor Author

The problem in:

    [1/2] D:\bld\onnxruntime_1716234839539\_build_env\Library\bin\nvcc.exe -forward-unknown-to-host-compiler   /DWIN32 /D_WINDOWS /DWINAPI_FAMILY=100 /DWINVER=0x0A00 /D_WIN32_WINNT=0x0A00 /DNTDDI_VERSION=0x0A000000 /DONNXRUNTIME_ENABLE_INTEL_METEOR_LAKE_MOBILE_PLATFORM_PERF_PATCH /DNDEBUG -Xcompiler=" /guard:cf /MP /Qspectre /O2 /Ob2"  -D_WINDOWS -Xcompiler=" /EHsc"  -Xcompiler=" -Ob0 -Od /RTC1" -arch=all -Xcompiler=-MDd -Xcompiler=-Zi -MD -MT CMakeFiles\cmTC_dd5bb.dir\main.cu.obj -MF CMakeFiles\cmTC_dd5bb.dir\main.cu.obj.d -x cu -c D:\bld\onnxruntime_1716234839539\work\build-ci\Release\CMakeFiles\CMakeScratch\TryCompile-fjxebz\main.cu -o CMakeFiles\cmTC_dd5bb.dir\main.cu.obj -Xcompiler=-FdCMakeFiles\cmTC_dd5bb.dir\,-FS

Are the /DWIN32 /D_WINDOWS /DWINAPI_FAMILY=100 /DWINVER=0x0A00 /D_WIN32_WINNT=0x0A00 /DNTDDI_VERSION=0x0A000000 /DONNXRUNTIME_ENABLE_INTEL_METEOR_LAKE_MOBILE_PLATFORM_PERF_PATCH /DNDEBUG options, manually converting them to -DWIN32 -D_WINDOWS -DWINAPI_FAMILY=100 -DWINVER=0x0A00 -D_WIN32_WINNT=0x0A00 -DNTDDI_VERSION=0x0A000000 -DONNXRUNTIME_ENABLE_INTEL_METEOR_LAKE_MOBILE_PLATFORM_PERF_PATCH -DNDEBUG everything compiles fine (actually, at least the error is different).

As it does not seem to me that nvcc supports MSVC-style definitions (i.e. /DSYMBOL_DEFINED instead of -DSYMBOL_DEFINED ) I added in f8f1f98 a simple patch to normalize definitions options in the python build script. I am not sure why this problem does not appear upstream.

@traversaro traversaro closed this May 21, 2024
@traversaro traversaro reopened this May 21, 2024
@traversaro
Copy link
Contributor Author

As it does not seem to me that nvcc supports MSVC-style definitions (i.e. /DSYMBOL_DEFINED instead of -DSYMBOL_DEFINED ) I added in f8f1f98 a simple patch to normalize definitions options in the python build script. I am not sure why this problem does not appear upstream.

This fix worked fine. Now the CUDA 12 builds fail (at test level) as it run out of disk space, while CUDA 11 runs out of time. We can switch to CMAKE_CUDA_ARCHITECTURES=all-major for CUDA 11, while for CUDA 12 we can check how to get more space in the azure image, or in the worst case we can remove tests.

@traversaro
Copy link
Contributor Author

while for CUDA 12 we can check how to get more space in the azure image, or in the worst case we can remove tests.

Apparently free_disk_space is only available on Linux. Let's just switch to all-major for now and see how it goes.

@traversaro
Copy link
Contributor Author

while for CUDA 12 we can check how to get more space in the azure image, or in the worst case we can remove tests.

Apparently free_disk_space is only available on Linux. Let's just switch to all-major for now and see how it goes.

I remember that there was another trick on Windows: f3662ef . Let's try to use that an still use all on cuda 12.0 .

@traversaro
Copy link
Contributor Author

@conda-forge-admin please rerender

@traversaro
Copy link
Contributor Author

Not sure why, but now all the jobs (even the non-cuda one) are failing with MemoryError while linking onnxruntime_test_all.exe :

[1912/1927] Linking CXX executable onnxruntime_mlas_q4dq.exe
[1913/1927] Building CUDA object CMakeFiles\onnxruntime_shared_lib_test.dir\C_\bld\onnxruntime_1716326872295\work\onnxruntime\test\shared_lib\cuda_ops.cu.obj
Traceback (most recent call last):
  File "C:\Miniforge\Scripts\conda-build-script.py", line 10, in <module>
    return build_tree(
  File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 3638, in build_tree
    packages_from_this = build(
  File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 2462, in build
    windows.build(
  File "C:\Miniforge\lib\site-packages\conda_build\windows.py", line 339, in build
    check_call_env(
  File "C:\Miniforge\lib\site-packages\conda_build\utils.py", line 405, in check_call_env
    return _func_defaulting_env_to_os_environ("call", *popenargs, **kwargs)
  File "C:\Miniforge\lib\site-packages\conda_build\utils.py", line 376, in _func_defaulting_env_to_os_environ
    proc = PopenWrapper(_args, **kwargs)
  File "C:\Miniforge\lib\site-packages\conda_build\utils.py", line 246, in __init__
    self.out, self.err = self._execute(*args, **kwargs)
  File "C:\Miniforge\lib\site-packages\conda_build\utils.py", line 320, in _execute
    self.disk = max(directory_size(disk_usage_dir), self.disk)
  File "C:\Miniforge\lib\site-packages\conda_build\utils.py", line 146, in directory_size
    out = subprocess.check_output(command.format(path), shell=True)
  File "C:\Miniforge\lib\subprocess.py", line 421, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "C:\Miniforge\lib\subprocess.py", line 505, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "C:\Miniforge\lib\subprocess.py", line 1141, in communicate
    stdout = self.stdout.read()
MemoryError
cuda_ops.cu

tmpxft_000014f0_00000000-10_cuda_ops.compute_90.cudafe1.cpp

[1914/1927] Linking CXX executable onnxruntime_test_all.exe
FAILED: onnxruntime_test_all.exe 

@hmaarrfk
Copy link
Contributor

Maybe we can skip making the test executable….

@traversaro
Copy link
Contributor Author

Maybe we can skip making the test executable….

Yes, this is exactly what is happening now. I would still like to run tests on non-cuda builds, but that is not a blocker.

@hmaarrfk
Copy link
Contributor

Yes, this is exactly what is happening now.

ah very cool!

@traversaro
Copy link
Contributor Author

traversaro commented May 22, 2024

Ok, we have a solution working, let me check if I can improve this, but if I fail I will go back to this and cleanup the PR (eventually rebasing on the 1.18.0 one).

@traversaro traversaro closed this May 22, 2024
@traversaro traversaro reopened this May 22, 2024
@hmaarrfk hmaarrfk mentioned this pull request May 22, 2024
3 tasks
@conda-forge-webservices
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I was trying to look for recipes to lint for you, but it appears we have a merge conflict.
Please try to merge or rebase with the base branch to resolve this conflict.

Please ping the 'conda-forge/core' team (using the @ notation in a comment) if you believe this is a bug.

@traversaro
Copy link
Contributor Author

Follow up and full matrix test in #121 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants