Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use env vars instead of CMake vars for Dawn/Node bindings #7422

Merged
merged 21 commits into from
May 2, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
ac41747
Split WebGPU runtime into two variants (#7248 workaround)
steven-johnson Mar 14, 2023
fa0ae85
A few more fixes
steven-johnson Mar 14, 2023
2b34e19
Use env vars instead of CMake vars for Dawn/Node bindings
steven-johnson Mar 15, 2023
2f55173
Merge branch 'main' into srj/webgpu-two-variants
steven-johnson Mar 15, 2023
9743ddd
Merge branch 'srj/webgpu-two-variants' into srj/webgpu-env-vars
steven-johnson Mar 15, 2023
944555e
Merge branch 'main' into srj/webgpu-env-vars
steven-johnson Mar 16, 2023
0103abc
Merge branch 'main' into srj/webgpu-env-vars
steven-johnson Mar 17, 2023
aa49544
Merge branch 'main' into srj/webgpu-env-vars
steven-johnson Mar 28, 2023
91c3fc5
Improve "Could not find a native WebGPU library" error msg
steven-johnson Mar 28, 2023
f3bcf0c
Update README_webgpu.md
steven-johnson Mar 28, 2023
76e024e
Use a launcher script for wasm (with or without webgpu)
steven-johnson Mar 28, 2023
6735fc3
Merge branch 'main' into srj/webgpu-env-vars
steven-johnson Apr 18, 2023
8f09166
Use a find-module to locate Dawn/WebGPU (#7525)
alexreinking Apr 20, 2023
c00440a
Merge branch 'main' into srj/webgpu-env-vars
steven-johnson Apr 20, 2023
e441425
Merge branch 'srj/webgpu-env-vars' of https://github.com/halide/Halid…
steven-johnson Apr 20, 2023
9a3312b
trigger buildbots
steven-johnson Apr 20, 2023
98b98bf
Merge branch 'main' into srj/webgpu-env-vars
steven-johnson Apr 24, 2023
78a0226
Update FindHalide_WebGPU.cmake
steven-johnson Apr 27, 2023
8379c53
trigger buildbots
steven-johnson Apr 28, 2023
bb01f3c
Update FindHalide_WebGPU.cmake
steven-johnson May 1, 2023
24eb78c
Update FindHalide_WebGPU.cmake
steven-johnson May 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 15 additions & 7 deletions README_webgpu.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ The following is a non-comprehensive list of known limitations:
* 64-bit integers and floats will likely remain unsupported until WGSL gains
extensions to support them.
- Wrapping native device buffer handles is not yet implemented.
- You must use CMake/CTest to build/test Halide for WebGPU; using the Makefile
is not supported for WebGPU testing (and probably never will be).

In addition to these functional limitations, the performance of the WebGPU
backend has not yet been evaluated, and so optimizations in the runtime or
Expand All @@ -35,8 +37,9 @@ When invoking `emcc` to link Halide-generated objects, include these flags:

Tests that use AOT compilation can be run using a native WebGPU implementation
that has Node.js bindings, such as [Dawn](dawn.googlesource.com/dawn/).
When configuring Halide, use `-DWEBGPU_NODE_BINDINGS=/path/to/dawn.node` to
enable these tests.
You must set an environment variable named `HL_WEBGPU_NODE_BINDINGS` that
has an absolute path to the bindings to run these tests, e.g. `HL_WEBGPU_NODE_BINDINGS=/path/to/dawn.node`.

See [below](#setting-up-dawn) for instructions on building the Dawn Node.js
bindings.

Expand All @@ -51,9 +54,14 @@ For testing purposes, Halide can also target native WebGPU libraries, such as
This is currently the only path that can run the JIT correctness tests.
See [below](#setting-up-dawn) for instructions on building Dawn.

Pass `-DWEBGPU_NATIVE_LIB=/path/to/native/library.{so,dylib.dll}` to CMake when
configuring Halide to enable this path, which will automatically use this
library for the AOT and JIT tests.
When targeting WebGPU with a native target, Halide defaults to looking for a
build of Dawn (with several common names and suffixes); you can override this
by setting the `HL_WEBGPU_NATIVE_LIB` environment variable to the absolute path
to the library you want.

Note that it is explicitly legal to define both `HL_WEBGPU_NATIVE_LIB` and
`HL_WEBGPU_NODE_BINDINGS` at the same time; the correct executable environment
will be selected based on the Halide target specified.

Note that it is explicitly legal to specify both WEBGPU_NATIVE_LIB and
WEBGPU_NODE_BINDINGS for the same build; the correct executable environment
Expand Down Expand Up @@ -98,5 +106,5 @@ This will produce the following artifacts:
- Node.js bindings: `<build_dir>/dawn.node`
- Native library: `<build_dir>/src/dawn/native/libwebgpu_dawn.{so,dylib,dll}`

These paths can then be used for the `-DWEBGPU_NODE_BINDINGS` and
`-DWEBGPU_NATIVE_LIB` CMake options when configuring Halide.
These paths can then be used for the `HL_WEBGPU_NODE_BINDINGS` and
`HL_WEBGPU_NATIVE_LIB` environment variables when using Halide.
27 changes: 27 additions & 0 deletions cmake/FindHalide_WebGPU.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
cmake_minimum_required(VERSION 3.22)

# tip: uncomment this line to get better debugging information if find_library() fails
# set(CMAKE_FIND_DEBUG_MODE TRUE)

if (EXISTS "$ENV{HL_WEBGPU_NATIVE_LIB}")
set(Halide_WebGPU_NATIVE_LIB "$ENV{HL_WEBGPU_NATIVE_LIB}"
CACHE FILEPATH "")
endif ()

find_library(Halide_WebGPU_NATIVE_LIB NAMES webgpu_dawn wgpu)

include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(
Halide_WebGPU
REQUIRED_VARS Halide_WebGPU_NATIVE_LIB
HANDLE_COMPONENTS
)

if (Halide_WebGPU_NATIVE_LIB AND NOT TARGET Halide::WebGPU)
add_library(Halide::WebGPU UNKNOWN IMPORTED)
set_target_properties(
Halide::WebGPU
PROPERTIES
IMPORTED_LOCATION "${Halide_WebGPU_NATIVE_LIB}"
)
endif ()
5 changes: 2 additions & 3 deletions cmake/HalideGeneratorHelpers.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -718,9 +718,8 @@ function(_Halide_target_link_gpu_libs TARGET VISIBILITY)
endif ()

if ("${ARGN}" MATCHES "webgpu")
if (WEBGPU_NATIVE_LIB)
target_link_libraries(${TARGET} INTERFACE ${WEBGPU_NATIVE_LIB})
endif ()
find_package(Halide_WebGPU REQUIRED)
target_link_libraries(${TARGET} ${VISIBILITY} Halide::WebGPU)
endif ()
endfunction()

Expand Down
11 changes: 1 addition & 10 deletions dependencies/wasm/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -153,18 +153,9 @@ function(add_wasm_halide_test TARGET)
return()
endif ()

set(script "require('./${TARGET}.js')")
if (WEBGPU_NODE_BINDINGS)
set(script "\
const provider = require('${WEBGPU_NODE_BINDINGS}')\n\
const gpu = provider.create([])\n\
const navigator = { gpu: gpu }\n
${script}")
endif ()

add_halide_test("${TARGET}"
GROUPS ${args_GROUPS}
COMMAND ${NODE_JS_EXECUTABLE} "-e" "${script}")
COMMAND ${NODE_JS_EXECUTABLE} "${Halide_SOURCE_DIR}/tools/launch_wasm_test.js" "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}.js" "${Halide_TARGET}")
endfunction()

function(find_node_js)
Expand Down
1 change: 1 addition & 0 deletions packaging/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,7 @@ install(FILES
${CMAKE_CURRENT_BINARY_DIR}/HalideHelpersConfig.cmake
${CMAKE_CURRENT_BINARY_DIR}/HalideHelpersConfigVersion.cmake
${Halide_SOURCE_DIR}/cmake/HalideGeneratorHelpers.cmake
${Halide_SOURCE_DIR}/cmake/FindHalide_WebGPU.cmake
${Halide_SOURCE_DIR}/cmake/HalideTargetHelpers.cmake
${Halide_SOURCE_DIR}/cmake/TargetExportScript.cmake
DESTINATION ${Halide_INSTALL_HELPERSDIR}
Expand Down
7 changes: 0 additions & 7 deletions src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -573,13 +573,6 @@ endif ()
option(TARGET_WEBGPU "Include WebGPU target" ON)
if (TARGET_WEBGPU)
target_compile_definitions(Halide PRIVATE WITH_WEBGPU)
set(WEBGPU_NATIVE_LIB "" CACHE STRING
"WebGPU native library to link against")
if (WEBGPU_NATIVE_LIB)
set_property(SOURCE JITModule.cpp PROPERTY COMPILE_DEFINITIONS
WEBGPU_NATIVE_LIB=\"${WEBGPU_NATIVE_LIB}\"
APPEND_STRING)
endif ()
endif()

if (TARGET_SPIRV)
Expand Down
2 changes: 1 addition & 1 deletion src/CodeGen_WebGPU_Dev.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -345,7 +345,7 @@ void CodeGen_WebGPU_Dev::CodeGen_WGSL::add_kernel(
user_warning
<< "buffers of small integer types are currently emulated "
<< "using atomics in the WebGPU backend, and accesses to "
<< "them will be slow";
<< "them will be slow.";
buffers_with_emulated_accesses.insert(arg.name);
type_decl = "atomic<u32>";
} else {
Expand Down
58 changes: 33 additions & 25 deletions src/JITModule.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -117,35 +117,43 @@ void load_metal() {

void load_webgpu() {
debug(1) << "Looking for a native WebGPU implementation...\n";
const char *libnames[] = {
#ifdef WEBGPU_NATIVE_LIB
// Specified via CMake.
WEBGPU_NATIVE_LIB,
#endif
// Dawn (Chromium).
"libwebgpu_dawn.so",
"libwebgpu_dawn.dylib",
"webgpu_dawn.dll",

// wgpu (Firefox).
"libwgpu.so",
"libwgpu.dylib",
"wgpu.dll",
};
string error;
for (const char *libname : libnames) {

const auto try_load = [](const char *libname) -> string {
debug(1) << "Trying " << libname << "... ";
error.clear();
string error;
llvm::sys::DynamicLibrary::LoadLibraryPermanently(libname, &error);
if (error.empty()) {
debug(1) << "found!\n";
break;
} else {
debug(1) << "not found.\n";
debug(1) << (error.empty() ? "found!\n" : "not found.\n");
return error;
};

string error;

auto env_libname = get_env_variable("HL_WEBGPU_NATIVE_LIB");
if (!env_libname.empty()) {
error = try_load(env_libname.c_str());
}
if (!error.empty()) {
const char *libnames[] = {
// Dawn (Chromium).
"libwebgpu_dawn.so",
"libwebgpu_dawn.dylib",
"webgpu_dawn.dll",

// wgpu (Firefox).
"libwgpu.so",
"libwgpu.dylib",
"wgpu.dll",
};

for (const char *libname : libnames) {
error = try_load(libname);
if (error.empty()) {
break;
}
}
}
user_assert(error.empty()) << "Could not find a native WebGPU library: "
<< error << "\n";
user_assert(error.empty()) << "Could not find a native WebGPU library: " << error << "\n"
<< "(Try setting the env var HL_WEBGPU_NATIVE_LIB to an explicit path to fix this.)\n";
}

} // namespace
Expand Down
Loading