Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: run on win #24

Merged
merged 40 commits into from
Feb 24, 2025
Merged

feat: run on win #24

merged 40 commits into from
Feb 24, 2025

Conversation

chraac
Copy link
Owner

@chraac chraac commented Feb 20, 2025

Closes #22

Here we will add windows platform support to qnn backend

Build with msvc 2022

  1. Download Qualcomm AI Engine Direct SDK, from here, and then extract it into a folder

  2. Install the latest Visual Studio, make sure clang toolchain and 'cmake' are installed
    Image
    Image

  3. Launch vs2022, tap Continue without code, and then in File menu, select Open -> CMake, in file open dialog, navigate to llama.cpp root directory, select CMakeLists.txt

  4. Edit llama.cpp\CMakePresets.json, add following line to config arm64-windows-llvm

    {
        "name": "arm64-windows-llvm", "hidden": true,
        "architecture": { "value": "arm64",    "strategy": "external" },
        "toolset":      { "value": "host=x64", "strategy": "external" },
        "cacheVariables": {
    -        "CMAKE_TOOLCHAIN_FILE": "${sourceDir}/cmake/arm64-windows-llvm.cmake"
    +        "CMAKE_TOOLCHAIN_FILE": "${sourceDir}/cmake/arm64-windows-llvm.cmake",
    +        "GGML_QNN": "ON",
    +        "GGML_QNN_SDK_PATH": "<path to your Qualcomm AI Engine Direct SDK, like x:/ml/qnn_sdk/qairt/2.31.0.250130/>",
    +        "BUILD_SHARED_LIBS": "OFF"
        }
    },
  5. Select config arm64-windows-llvm-debug
    Image

  6. In Build menu, tap Build All, output file are located at build-arm64-windows-llvm-debug\bin\

  7. Please copy those files from sdk lib directory (at qairt\2.31.0.250130\lib\aarch64-windows-msvc\) to the output directory before run:

QnnSystem.dll
QnnCpu.dll
QnnGpu.dll
QnnHtp.dll
QnnHtp*.dll

Tests

image

#22 (comment)

For detailed build instructions, please refer to #22 (comment)

@chraac chraac added enhancement New feature or request windows labels Feb 20, 2025
@chraac chraac self-assigned this Feb 20, 2025
@chraac chraac linked an issue Feb 20, 2025 that may be closed by this pull request
4 tasks
@chraac chraac marked this pull request as draft February 20, 2025 03:38
@chraac chraac marked this pull request as ready for review February 23, 2025 10:44
@chraac chraac merged commit a822d00 into dev-refactoring Feb 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request windows
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[feat] Port to windows
1 participant