Skip to content
Benchmark on Comment

OOM error on 7B model despite 4-bit quantization with 24GB VRAM #325

Sign in to view logs

OOM error on 7B model despite 4-bit quantization with 24GB VRAM

OOM error on 7B model despite 4-bit quantization with 24GB VRAM #325

Triggered via issue October 10, 2023 18:18
@younesbelkadayounesbelkada
commented on #846 95aea7c
Status Skipped
Total duration 2s
Artifacts –

benchmark.yml

on: issue_comment
Matrix: Benchmark
Fit to window
Zoom out
Zoom in