Skip to content

Commit 14768e6

Browse files
jeffbolznvqnixsynapse
authored andcommitted
vulkan: Rewrite synchronization to allow some overlap between nodes (ggml-org#15489)
Track a list of nodes that need synchronization, and only sync if the new node depends on them (or overwrites them). This allows some overlap which can improve performance, and centralizes a big chunk of the synchronization logic. The remaining synchronization logic involves writes to memory other than the nodes, e.g. for dequantization or split_k. Each of these allocations has a bool indicating whether they were in use and need to be synced. This should be checked before they are written to, and set to true after they are done being consumed.
1 parent d2c8ec6 commit 14768e6

File tree

1 file changed

+193
-40
lines changed

1 file changed

+193
-40
lines changed

0 commit comments

Comments
 (0)