feat(intel): add diffusers/transformers support #1746
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for localai canceled.
|
mudler
force-pushed
the
intel-gpu-support
branch
from
0be681e
to
6941afc
Compare
mudler
force-pushed
the
intel-gpu-support
branch
2 times, most recently
from
8e059cf
to
3ee6d12
Compare
mudler
changed the title
mudler
force-pushed
the
intel-gpu-support
branch
from
a6a039c
to
46f38a7
Compare
mudler
force-pushed
the
intel-gpu-support
branch
from
6f7154c
to
72d1cd2
Compare
|
|
mudler
force-pushed
the
intel-gpu-support
branch
from
6703787
to
95fb025
Compare
|
Merging, waiting for fixes upstream but will likely follow-up |
dave-gray101
added a commit
to dave-gray101/LocalAI
that referenced
this pull request
Mar 7, 2024
dave-gray101
added a commit
to dave-gray101/LocalAI
that referenced
this pull request
Mar 7, 2024
truecharts-admin
referenced
this pull request
in truecharts/public
Mar 17, 2024
…0.0@5cd0285 by renovate (#19391) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) | minor | `v2.9.0` -> `v2.10.0` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>mudler/LocalAI (docker.io/localai/localai)</summary> ### [`v2.10.0`](https://togithub.com/mudler/LocalAI/releases/tag/v2.10.0) [Compare Source](https://togithub.com/mudler/LocalAI/compare/v2.9.0...v2.10.0) ##### LocalAI v2.10.0 Release Notes Excited to announce the release of LocalAI v2.10.0! This version introduces significant changes, including breaking changes, numerous bug fixes, exciting new features, dependency updates, and more. Here's a summary of what's new: ##### Breaking Changes 🛠 - The `trust_remote_code` setting in the YAML config file of the model are now consumed for enhanced security measures also for the AutoGPTQ and transformers backend, thanks to [@​dave-gray101](https://togithub.com/dave-gray101)'s contribution ([#​1799](https://togithub.com/mudler/LocalAI/pull/1799)). If your model relied on the old behavior and you are sure of what you are doing, set `trust_remote_code: true` in the YAML config file. ##### Bug Fixes 🐛 - Various fixes have been implemented to enhance the stability and performance of LocalAI: - SSE no longer omits empty `finish_reason` fields for better compatibility with the OpenAI API, fixed by [@​mudler](https://togithub.com/mudler) ([#​1745](https://togithub.com/mudler/LocalAI/pull/1745)). - Functions now correctly handle scenarios with no results, also addressed by [@​mudler](https://togithub.com/mudler) ([#​1758](https://togithub.com/mudler/LocalAI/pull/1758)). - A Command Injection Vulnerability has been fixed by [@​ouxs-19](https://togithub.com/ouxs-19) ([#​1778](https://togithub.com/mudler/LocalAI/pull/1778)). - OpenCL-based builds for llama.cpp have been restored, thanks to [@​cryptk](https://togithub.com/cryptk)'s efforts ([#​1828](https://togithub.com/mudler/LocalAI/pull/1828), [#​1830](https://togithub.com/mudler/LocalAI/pull/1830)). - An issue with OSX build `default.metallib` has been resolved, which should now allow running the llama-cpp backend on Apple arm64, fixed by [@​dave-gray101](https://togithub.com/dave-gray101) ([#​1837](https://togithub.com/mudler/LocalAI/pull/1837)). ##### Exciting New Features 🎉 - LocalAI continues to evolve with several new features: - Ongoing implementation of the assistants API, making great progress thanks to community contributions, including an initial implementation by [@​christ66](https://togithub.com/christ66) ([#​1761](https://togithub.com/mudler/LocalAI/pull/1761)). - Addition of diffusers/transformers support for Intel GPU - now you can generate images and use the `transformer` backend also on Intel GPUs, implemented by [@​mudler](https://togithub.com/mudler) ([#​1746](https://togithub.com/mudler/LocalAI/pull/1746)). - Introduction of Bitsandbytes quantization for transformer backend enhancement and a fix for transformer backend error on CUDA by [@​fakezeta](https://togithub.com/fakezeta) ([#​1823](https://togithub.com/mudler/LocalAI/pull/1823)). - Compatibility layers for Elevenlabs and OpenAI TTS, enhancing text-to-speech capabilities: Now LocalAI is compatible with Elevenlabs and OpenAI TTS, thanks to [@​mudler](https://togithub.com/mudler) ([#​1834](https://togithub.com/mudler/LocalAI/pull/1834)). - vLLM now supports `stream: true`! This feature was introduced by [@​golgeek](https://togithub.com/golgeek) ([#​1749](https://togithub.com/mudler/LocalAI/pull/1749)). ##### Dependency Updates 👒 - Our continuous effort to keep dependencies up-to-date includes multiple updates to `ggerganov/llama.cpp`, `donomii/go-rwkv.cpp`, `mudler/go-stable-diffusion`, and others, ensuring that LocalAI is built on the latest and most secure libraries. ##### Other Changes - Several internal changes have been made to improve the development process and documentation, including updates to integration guides, stress reduction on self-hosted runners, and more. #### Details of What's Changed ##### Breaking Changes 🛠 - feat(autogpt/transformers): consume `trust_remote_code` by [@​dave-gray101](https://togithub.com/dave-gray101) in [https://github.com/mudler/LocalAI/pull/1799](https://togithub.com/mudler/LocalAI/pull/1799) ##### Bug fixes 🐛 - fix(sse): do not omit empty finish_reason by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1745](https://togithub.com/mudler/LocalAI/pull/1745) - fix(functions): handle correctly when there are no results by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1758](https://togithub.com/mudler/LocalAI/pull/1758) - fix(tests): re-enable tests after code move by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1764](https://togithub.com/mudler/LocalAI/pull/1764) - Fix Command Injection Vulnerability by [@​ouxs-19](https://togithub.com/ouxs-19) in [https://github.com/mudler/LocalAI/pull/1778](https://togithub.com/mudler/LocalAI/pull/1778) - fix: the correct BUILD_TYPE for OpenCL is clblas (with no t) by [@​cryptk](https://togithub.com/cryptk) in [https://github.com/mudler/LocalAI/pull/1828](https://togithub.com/mudler/LocalAI/pull/1828) - fix: missing OpenCL libraries from docker containers during clblas docker build by [@​cryptk](https://togithub.com/cryptk) in [https://github.com/mudler/LocalAI/pull/1830](https://togithub.com/mudler/LocalAI/pull/1830) - fix: osx build default.metallib by [@​dave-gray101](https://togithub.com/dave-gray101) in [https://github.com/mudler/LocalAI/pull/1837](https://togithub.com/mudler/LocalAI/pull/1837) ##### Exciting New Features 🎉 - fix: vllm - use AsyncLLMEngine to allow true streaming mode by [@​golgeek](https://togithub.com/golgeek) in [https://github.com/mudler/LocalAI/pull/1749](https://togithub.com/mudler/LocalAI/pull/1749) - refactor: move remaining api packages to core by [@​dave-gray101](https://togithub.com/dave-gray101) in [https://github.com/mudler/LocalAI/pull/1731](https://togithub.com/mudler/LocalAI/pull/1731) - Bump vLLM version + more options when loading models in vLLM by [@​golgeek](https://togithub.com/golgeek) in [https://github.com/mudler/LocalAI/pull/1782](https://togithub.com/mudler/LocalAI/pull/1782) - feat(assistant): Initial implementation of assistants api by [@​christ66](https://togithub.com/christ66) in [https://github.com/mudler/LocalAI/pull/1761](https://togithub.com/mudler/LocalAI/pull/1761) - feat(intel): add diffusers/transformers support by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1746](https://togithub.com/mudler/LocalAI/pull/1746) - fix(config): set better defaults for inferencing by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1822](https://togithub.com/mudler/LocalAI/pull/1822) - fix(docker-compose): update docker compose file by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1824](https://togithub.com/mudler/LocalAI/pull/1824) - feat(model-help): display help text in markdown by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1825](https://togithub.com/mudler/LocalAI/pull/1825) - feat: Add Bitsandbytes quantization for transformer backend enhancement [#​1775](https://togithub.com/mudler/LocalAI/issues/1775) and fix: Transformer backend error on CUDA [#​1774](https://togithub.com/mudler/LocalAI/issues/1774) by [@​fakezeta](https://togithub.com/fakezeta) in [https://github.com/mudler/LocalAI/pull/1823](https://togithub.com/mudler/LocalAI/pull/1823) - feat(tts): add Elevenlabs and OpenAI TTS compatibility layer by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1834](https://togithub.com/mudler/LocalAI/pull/1834) - feat(embeddings): do not require to be configured by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1842](https://togithub.com/mudler/LocalAI/pull/1842) ##### 👒 Dependencies - ⬆️ Update docs version mudler/LocalAI by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1752](https://togithub.com/mudler/LocalAI/pull/1752) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1753](https://togithub.com/mudler/LocalAI/pull/1753) - deps(llama.cpp): update by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1759](https://togithub.com/mudler/LocalAI/pull/1759) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1756](https://togithub.com/mudler/LocalAI/pull/1756) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1767](https://togithub.com/mudler/LocalAI/pull/1767) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1772](https://togithub.com/mudler/LocalAI/pull/1772) - ⬆️ Update donomii/go-rwkv.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1771](https://togithub.com/mudler/LocalAI/pull/1771) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1779](https://togithub.com/mudler/LocalAI/pull/1779) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1789](https://togithub.com/mudler/LocalAI/pull/1789) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1791](https://togithub.com/mudler/LocalAI/pull/1791) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1794](https://togithub.com/mudler/LocalAI/pull/1794) - depedencies(sentencentranformers): update dependencies by [@​TwinFinz](https://togithub.com/TwinFinz) in [https://github.com/mudler/LocalAI/pull/1797](https://togithub.com/mudler/LocalAI/pull/1797) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1801](https://togithub.com/mudler/LocalAI/pull/1801) - ⬆️ Update mudler/go-stable-diffusion by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1802](https://togithub.com/mudler/LocalAI/pull/1802) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1805](https://togithub.com/mudler/LocalAI/pull/1805) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1811](https://togithub.com/mudler/LocalAI/pull/1811) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1827](https://togithub.com/mudler/LocalAI/pull/1827) ##### Other Changes - ci: add stablediffusion to release by [@​sozercan](https://togithub.com/sozercan) in [https://github.com/mudler/LocalAI/pull/1757](https://togithub.com/mudler/LocalAI/pull/1757) - Update integrations.md by [@​Joshhua5](https://togithub.com/Joshhua5) in [https://github.com/mudler/LocalAI/pull/1765](https://togithub.com/mudler/LocalAI/pull/1765) - ci: reduce stress on self-hosted runners by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1776](https://togithub.com/mudler/LocalAI/pull/1776) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1785](https://togithub.com/mudler/LocalAI/pull/1785) - Revert "feat(assistant): Initial implementation of assistants api" by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1790](https://togithub.com/mudler/LocalAI/pull/1790) - Edit links in readme and integrations page by [@​lunamidori5](https://togithub.com/lunamidori5) in [https://github.com/mudler/LocalAI/pull/1796](https://togithub.com/mudler/LocalAI/pull/1796) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1813](https://togithub.com/mudler/LocalAI/pull/1813) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1816](https://togithub.com/mudler/LocalAI/pull/1816) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1818](https://togithub.com/mudler/LocalAI/pull/1818) - fix(doc/examples): set defaults to mirostat by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1820](https://togithub.com/mudler/LocalAI/pull/1820) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1821](https://togithub.com/mudler/LocalAI/pull/1821) - fix: OSX Build Files for llama.cpp by [@​dave-gray101](https://togithub.com/dave-gray101) in [https://github.com/mudler/LocalAI/pull/1836](https://togithub.com/mudler/LocalAI/pull/1836) - ⬆️ Update go-skynet/go-llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1835](https://togithub.com/mudler/LocalAI/pull/1835) - docs(transformers): add docs section about transformers by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1841](https://togithub.com/mudler/LocalAI/pull/1841) - ⬆️ Update mudler/go-piper by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1844](https://togithub.com/mudler/LocalAI/pull/1844) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1840](https://togithub.com/mudler/LocalAI/pull/1840) #### New Contributors - [@​golgeek](https://togithub.com/golgeek) made their first contribution in [https://github.com/mudler/LocalAI/pull/1749](https://togithub.com/mudler/LocalAI/pull/1749) - [@​Joshhua5](https://togithub.com/Joshhua5) made their first contribution in [https://github.com/mudler/LocalAI/pull/1765](https://togithub.com/mudler/LocalAI/pull/1765) - [@​ouxs-19](https://togithub.com/ouxs-19) made their first contribution in [https://github.com/mudler/LocalAI/pull/1778](https://togithub.com/mudler/LocalAI/pull/1778) - [@​TwinFinz](https://togithub.com/TwinFinz) made their first contribution in [https://github.com/mudler/LocalAI/pull/1797](https://togithub.com/mudler/LocalAI/pull/1797) - [@​cryptk](https://togithub.com/cryptk) made their first contribution in [https://github.com/mudler/LocalAI/pull/1828](https://togithub.com/mudler/LocalAI/pull/1828) - [@​fakezeta](https://togithub.com/fakezeta) made their first contribution in [https://github.com/mudler/LocalAI/pull/1823](https://togithub.com/mudler/LocalAI/pull/1823) Thank you to all contributors and users for your continued support and feedback, making LocalAI better with each release! **Full Changelog**: mudler/LocalAI@v2.9.0...v2.10.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNTAuMSIsInVwZGF0ZWRJblZlciI6IjM3LjI1MC4xIiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIn0=-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR closes #1653 and fixes #982.
Relates to #1497 as it now sets the HF env vars to the model path if not specified. This allows the models downloaded by diffusers and the transformers backend to output the downloaded models in the model folder specified when starting LocalAI
Reference: