Search results

How to transform the orignal Mixtral 8*7B into the mixed HQQ quantized model ?

Status: Open.

#40 In dvmazur/mixtral-offloading;

· eljrte opened on Feb 6, 2025

Hard to benchmark the operation in the repo

Status: Open.

#39 In dvmazur/mixtral-offloading;

· mynotwo opened on Aug 29, 2024

Mixtral Instruct tokenizer from Colab notebook doesn't work.

Status: Open.

#38 In dvmazur/mixtral-offloading;

· jmuntaner-smd opened on Jul 8, 2024

Support DeepSeek V2 model

Status: Open.

#36 In dvmazur/mixtral-offloading;

· Minami-su opened on May 15, 2024

Having issue loading my HQQ quantized model

Status: Open.

#35 In dvmazur/mixtral-offloading;

· BeichenHuang opened on Apr 24, 2024

How to split the model parameter safetensors file into multiple small files

Status: Open.

#34 In dvmazur/mixtral-offloading;

· YLSnowy opened on Apr 18, 2024

Implementation of benchmarks (C4 perplexity, Wikitext perplexity)

Status: Open.

#33 In dvmazur/mixtral-offloading;

· ChengSashankh opened on Apr 14, 2024

runtimeerror when nbit = 4 and group_size =64

Status: Open.

#32 In dvmazur/mixtral-offloading;

· Eutenacity opened on Apr 13, 2024

Can this be used for Jambo inference

Status: Open.

#30 In dvmazur/mixtral-offloading;

· freQuensy23-coder opened on Apr 8, 2024

a strange issue with default parameters " RuntimeError about memory"

Status: Open.

#26 In dvmazur/mixtral-offloading;

· a1564740774 opened on Mar 24, 2024

Update Requirements.txt

Status: Open.

#25 In dvmazur/mixtral-offloading;

· Soumadip-Saha opened on Feb 18, 2024

Run on second GPU (torch.device("cuda:1"))

Status: Open.

#24 In dvmazur/mixtral-offloading;

· imabot2 opened on Feb 9, 2024