We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent ac4fd87 commit 65f2de5Copy full SHA for 65f2de5
launch/dynamo-run/src/subprocess/trtllm_config/sample.yaml
@@ -20,5 +20,6 @@
20
# You might have to tweak this config based on your model size and GPU memory.
21
22
backend: pytorch
23
+disable_overlap_scheduler: true
24
kv_cache_config:
25
free_gpu_memory_fraction: 0.40
0 commit comments