Skip to content

Commit f6e2c49

Browse files
committed
Removed some comments
1 parent 2190b45 commit f6e2c49

File tree

1 file changed

+1
-15
lines changed

1 file changed

+1
-15
lines changed

examples/pytorch/continuous_batching.py

Lines changed: 1 addition & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -252,19 +252,5 @@ def batch_generate(
252252
expected_outputs=expected_outputs,
253253
)
254254

255-
256-
# python examples/pytorch/continuous_batching.py --attn sdpa_paged --matmul-precision none --samples 50 --displayed 0
257-
# Using calculated self.num_blocks = 4096, self.block_size = 32, self.max_batch_tokens = 2048
258-
# CB generation took: 18.80 seconds for 13775 tokens. 732.74tok/s
259-
260-
261-
# python examples/pytorch/continuous_batching.py --attn sdpa_paged --matmul-precision none --samples 100 --displayed 1
262-
# Setting up static tensors with T = 4096, max_token_budget = 524288, 139538202624 bytes available
263-
# CB generation took: 29.53 seconds for 26384 tokens. 893.41tok/s
264-
265-
# Without changes to continuous_batching.py
266-
# Using calculated num_blocks=369, block_size=32, max concurrent requests 23
267-
# CB generation took: 79.58 seconds for 25813 tokens. 324.38tok/s
268-
269-
255+
# Example usage:
270256
# python examples/pytorch/continuous_batching.py --num-blocks 369 --max-batch-tokens 23 --attn sdpa_paged -mp none --samples 1 --displayed 0 --output-file sliced.json

0 commit comments

Comments
 (0)