Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support WebGPU #110

Merged
merged 16 commits into from
May 10, 2024
Merged

Support WebGPU #110

merged 16 commits into from
May 10, 2024

Conversation

DanielHauschildt
Copy link
Contributor

Added support for

  • WebGPU
  • Patched ISNET for WebGPU execution
  • Onnx runtime with WebGPU support
  • Smaller ModNet Model

Added "device" parameter with options "cpu | gpu". For now "cpu" is default.

Copy link
Contributor

@mirko314 mirko314 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works great and super quick in Chrome!!
However, I could not get the example to work in Safari and Firefox. I'd expect a gracefull fallback to lower-performance runtimes until one is found that works in the browser

Comment on lines 43 to 40
ort.env.wasm.wasmPaths = {
'ort-wasm-simd-threaded.wasm':
capabilities.simd && capabilities.threads
useThreads && useSimd
? await loadAsUrl(
'/onnxruntime-web/ort-wasm-simd-threaded.wasm',
useWebGPU
? '/onnxruntime-web/ort-wasm-simd-threaded.jsep.wasm'
: '/onnxruntime-web/ort-wasm-simd-threaded.wasm',
config
)
: undefined,
'ort-wasm-simd.wasm':
capabilities.simd && !capabilities.threads
? await loadAsUrl('/onnxruntime-web/ort-wasm-simd.wasm', config)
!useThreads && useSimd
? await loadAsUrl(
useWebGPU
? '/onnxruntime-web/ort-wasm-simd.jsep.wasm'
: '/onnxruntime-web/ort-wasm-simd.wasm',
config
)
: undefined,
'ort-wasm-threaded.wasm':
!capabilities.simd && capabilities.threads
!useWebGPU && useThreads && !useSimd
? await loadAsUrl('/onnxruntime-web/ort-wasm-threaded.wasm', config)
: undefined,
'ort-wasm.wasm':
!capabilities.simd && !capabilities.threads
!useWebGPU && !useThreads && !useSimd
? await loadAsUrl('/onnxruntime-web/ort-wasm.wasm', config)
: undefined
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this code path seems a bit hard to scan, I'd personally like to simplify this logic

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feeded your comment to CoPilot and my sidekick wants to change it to this:

const wasmPaths = {
'ort-wasm-simd-threaded.wasm': useThreads && useSimd,
'ort-wasm-simd.wasm': !useThreads && useSimd,
'ort-wasm-threaded.wasm': !useWebGPU && useThreads && !useSimd,
'ort-wasm.wasm': !useWebGPU && !useThreads && !useSimd
};

for (const [key, value] of Object.entries(wasmPaths)) {
if (value) {
const wasmPath = useWebGPU && key.includes('simd') ? /onnxruntime-web/${key.replace('.wasm', '.jsep.wasm')} : /onnxruntime-web/${key};
ort.env.wasm.wasmPaths[key] = await loadAsUrl(wasmPath, config);
}
}

@DanielHauschildt DanielHauschildt changed the title Danielhauschildt/webgpu+modnet Danielhauschildt/webgpu Apr 29, 2024
@DanielHauschildt DanielHauschildt changed the title Danielhauschildt/webgpu Support WebGPU May 10, 2024
@DanielHauschildt DanielHauschildt merged commit e1c2588 into main May 10, 2024
1 check passed
@DanielHauschildt DanielHauschildt deleted the danielhauschildt/webgpu+modnet branch May 10, 2024 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants