Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issues building libshortfin on Windows with -DSHORTFIN_BUILD_DYNAMIC=ON #211

Open
ScottTodd opened this issue Sep 24, 2024 · 1 comment
Open

Issues building libshortfin on Windows with -DSHORTFIN_BUILD_DYNAMIC=ON #211

ScottTodd opened this issue Sep 24, 2024 · 1 comment

Comments

@ScottTodd
Copy link
Member

Related to #130.

I see errors trying to run pip install -v -e . and letting the setup.py file run CMake internally.

With the build fixes in #210, I'm able to build on Windows with MSVC, but only with -DSHORTFIN_BUILD_STATIC=ON (which setup.py does not set):

cmake \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE \
  -DSHORTFIN_BUILD_DYNAMIC=OFF \
  -DSHORTFIN_BUILD_STATIC=ON \
  -DSHORTFIN_IREE_SOURCE_DIR=D:/dev/projects/iree \
  -SD:/dev/projects/SHARK-Platform/libshortfin \
  -Bd:/dev/projects/SHARK-Platform/build 
  -G Ninja

cmake \
  --build d:/dev/projects/SHARK-Platform/build \
  --target all

...
[build] [26/26  96% :: 4.064] Linking CXX executable src\shortfin\support\shortfin_support_test.exe
[build] [26/26 100% :: 4.143] Linking CXX executable src\shortfin\array\shortfin_array_test.exe
[driver] Build completed: 00:00:04.193
[build] Build finished with exit code 0

(This works with and without -DSHORTFIN_IREE_SOURCE_DIR)

If I flip the options to -DSHORTFIN_BUILD_DYNAMIC=ON -DSHORTFIN_BUILD_STATIC=OFF, I get an error linking the tests:

[build] [29/29 100% :: 4.549] Linking CXX executable src\shortfin\support\shortfin_support_test.exe
[build] FAILED: src/shortfin/support/shortfin_support_test.exe src/shortfin/support/shortfin_support_test[1]_tests.cmake D:/dev/projects/SHARK-Platform/build/src/shortfin/support/shortfin_support_test[1]_tests.cmake 
[build] C:\WINDOWS\system32\cmd.exe /C "cd . && "C:\Program Files\CMake\bin\cmake.exe" -E vs_link_exe --intdir=src\shortfin\support\CMakeFiles\shortfin_support_test.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x64\mt.exe --manifests  -- C:\PROGRA~2\MICROS~1\2022\Preview\VC\Tools\MSVC\1431~1.311\bin\Hostx64\x64\link.exe /nologo @CMakeFiles\shortfin_support_test.rsp  /out:src\shortfin\support\shortfin_support_test.exe /implib:src\shortfin\support\shortfin_support_test.lib /pdb:src\shortfin\support\shortfin_support_test.pdb /version:0.0 /machine:x64 /debug /INCREMENTAL /subsystem:console  -natvis:D:/dev/projects/iree/runtime/iree.natvis && C:\WINDOWS\system32\cmd.exe /C "cd /D D:\dev\projects\SHARK-Platform\build\src\shortfin\support && "C:\Program Files\CMake\bin\cmake.exe" -D TEST_TARGET=shortfin_support_test -D TEST_EXECUTABLE=D:/dev/projects/SHARK-Platform/build/src/shortfin/support/shortfin_support_test.exe -D TEST_EXECUTOR= -D TEST_WORKING_DIR=D:/dev/projects/SHARK-Platform/build/src/shortfin/support -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D TEST_FILTER= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=shortfin_support_test_TESTS -D CTEST_FILE=D:/dev/projects/SHARK-Platform/build/src/shortfin/support/shortfin_support_test[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=5 -D TEST_XML_OUTPUT_DIR= -P "C:/Program Files/CMake/share/cmake-3.30/Modules/GoogleTestAddTests.cmake"""
[build] CMake Error at C:/Program Files/CMake/share/cmake-3.30/Modules/GoogleTestAddTests.cmake:112 (message):
[build]   Error running test executable.
[build] 
[build]     Path: 'D:/dev/projects/SHARK-Platform/build/src/shortfin/support/shortfin_support_test.exe'
[build]     Working directory: 'D:/dev/projects/SHARK-Platform/build/src/shortfin/support'
[build]     Result: Exit code 0xc0000135
[build] 
[build]   
[build] 
[build]     Output:
[build]       
[build] 
[build] Call Stack (most recent call first):
[build]   C:/Program Files/CMake/share/cmake-3.30/Modules/GoogleTestAddTests.cmake:226 (gtest_discover_tests_impl)

(more complete configure + build logs here: https://gist.github.com/ScottTodd/87fb74a06140c97418bbebf562820e71)

What I've tried so far

  • Some searching around suggests that 0xc0000135 exit code is STATUS_DLL_NOT_FOUND, so I tried changing https://github.com/nod-ai/SHARK-Platform/blob/61eacac59317f2b2db1e54778ca8a850db6f2734/libshortfin/build_tools/cmake/shortfin_library.cmake#L133-L138 to a hardcoded path like "D:/dev/projects/SHARK-Platform/build/src/shortfin.lib" instead of ${SHORTFIN_LINK_LIBRARY_NAME}. (Not sure if that should be the .dll or the .lib)
  • That changed the error message to libshortfin\src\shortfin\array\dims_test.cc(7): fatal error C1083: Cannot open include file: 'shortfin/array/dims.h': No such file or directory, so I also added a dep on _shortfin_defs to the same target_link_libraries call
  • That changed the error message to libshortfin\src\shortfin/support/iree_concurrency.h(12): fatal error C1083: Cannot open include file: 'iree/base/internal/synchronization.h': No such file or directory
  • I tried adding hardcoded includes for IREE and other dep headers next: 
      target_include_directories(
    ${_RULE_NAME}
    PRIVATE
      "D:/dev/projects/SHARK-Platform/build/_deps/fmt-src/include"
      "D:/dev/projects/SHARK-Platform/build/_deps/shortfin_iree-src/runtime/src"
      "D:/dev/projects/SHARK-Platform/build/_deps/spdlog-src/include"
      "D:/dev/projects/SHARK-Platform/build/_deps/xtensor-src/include"
      "D:/dev/projects/SHARK-Platform/build/_deps/xtl-src/include"
  )
  • That hit new errors, indicating C files being compiled as C++ (note the /TP in the command), but I didn't have those issues with static linking 🤔 
    [build] [7/9  11% :: 2.049] Building CXX object src\shortfin\array\CMakeFiles\shortfin_array_test.dir\dtype_test.cc.obj
[build] FAILED: src/shortfin/array/CMakeFiles/shortfin_array_test.dir/dtype_test.cc.obj 
[build] C:\PROGRA~2\MICROS~1\2022\Preview\VC\Tools\MSVC\1431~1.311\bin\Hostx64\x64\cl.exe  /nologo /TP -DSHORTFIN_HAVE_AMDGPU -ID:\dev\projects\SHARK-Platform\build\_deps\fmt-src\include -ID:\dev\projects\SHARK-Platform\build\_deps\shortfin_iree-src\runtime\src -ID:\dev\projects\SHARK-Platform\build\_deps\spdlog-src\include -ID:\dev\projects\SHARK-Platform\build\_deps\xtensor-src\include -ID:\dev\projects\SHARK-Platform\build\_deps\xtl-src\include -ID:\dev\projects\SHARK-Platform\libshortfin\src -ID:\dev\projects\SHARK-Platform\build\src -external:ID:\dev\projects\iree\third_party\googletest\googlemock\include -external:ID:\dev\projects\iree\third_party\googletest\googlemock -external:ID:\dev\projects\iree\third_party\googletest\googletest\include -external:ID:\dev\projects\iree\third_party\googletest\googletest -external:W0 /DWIN32 /D_WINDOWS /EHsc /O2 /Ob1 /DNDEBUG -std:c++20 -MD -Zi /showIncludes /Fosrc\shortfin\array\CMakeFiles\shortfin_array_test.dir\dtype_test.cc.obj /Fdsrc\shortfin\array\CMakeFiles\shortfin_array_test.dir\ /FS -c D:\dev\projects\SHARK-Platform\libshortfin\src\shortfin\array\dtype_test.cc
[build] D:\dev\projects\SHARK-Platform\build\_deps\shortfin_iree-src\runtime\src\iree/base/alignment.h(244): error C4576: a parenthesized type followed by an initializer list is a non-standard explicit type conversion syntax
[build] D:\dev\projects\SHARK-Platform\build\_deps\shortfin_iree-src\runtime\src\iree/base/alignment.h(253): error C4576: a parenthesized type followed by an initializer list is a non-standard explicit type conversion syntax
@ScottTodd
Copy link
Member Author

Discussed this a bit on IREE's discord server here: https://discord.com/channels/689900678990135345/1245423631626932264/1288274538886332448

@ScottTodd ScottTodd mentioned this issue Sep 25, 2024
Merged
ScottTodd added a commit that referenced this issue Sep 25, 2024
@ScottTodd
Set linking mode to static by default on Windows. (#214)
5c7fc17 
This is a workaround for
#211 while I/we find a
way to enable dynamic linking builds on Windows.

With this and #210, I can
configure and build libshortfin on Windows using default CMake options
and the CMake that `pip install -v -e .` runs internally. The local
package build passes an initial smoketest too (can run `import shortfin`
and poke around in the python interpreter).
This was referenced Dec 2, 2024
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ScottTodd