[FEATURE] Support Mixtral 7Bx8E on H2O LLM Studio

[binga]

🚀 Feature

Support Mistral's new release - Mixtral 7Bx8E

Motivation

Mixtral matches or outperforms Llama 2 70B, as well as GPT3.5, on most benchmarks.

[psinger]

https://huggingface.co/DiscoResearch/mixtral-7b-8expert

That one is already supported, you will just need to update transformers from dev.

Official HF release is not out yet.

[binga]

Thank you @psinger. I'll try this out.

[psinger]

huggingface/transformers#27942

We can leave this issue open to track, also we need to update transformers.

[maxjeblick]

Release already out https://github.com/huggingface/transformers/releases/tag/v4.36.0

[psinger]

Mixtral is supported now after package upgrades.