[Bug Fix] Illegal memory access, FP8 Llama 3.1 405b #6852

Merged

comaniac merged 1 commit intovllm-project:mainfrom neuralmagic:fix-illegal-memory-access-fp8

Jul 27, 2024

+37-9

Commits on Jul 26, 2024

add predicate to ensure we don't read of the end of the scales
LucasWilkinson
committed