Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC: Update installation part in readme #253

Merged
merged 5 commits into from
Jul 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 24 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,6 @@ and serve your or state-of-the-art built-in models using just a single command.
researcher, developer, or data scientist, Xorbits Inference empowers you to unleash the full
potential of cutting-edge AI models.

![demo](assets/demo.gif)

<div align="center">
<i><a href="https://join.slack.com/t/xorbitsio/shared_invite/zt-1z3zsm9ep-87yI9YZ_B79HLB2ccTq4WA">👉 Join our Slack community!</a></i>
</div>
Expand Down Expand Up @@ -52,14 +50,35 @@ with popular third-party libraries like LangChain and LlamaIndex. (Coming soon)
## Getting Started
Xinference can be installed via pip from PyPI. It is highly recommended to create a new virtual
environment to avoid conflicts.

### Installation
```bash
$ pip install "xinference[all]"
$ pip install "xinference"
```
`xinference[all]` installs all the necessary packages for serving models. If you want to achieve acceleration on
`xinference` installs basic packages for serving models.

#### Installation with GGML
To serve ggml models, you need to install the following extra dependencies:
```bash
$ pip install "xinference[ggml]"
```
If you want to achieve acceleration on
different hardware, refer to the installation documentation of the corresponding package.
- [llama-cpp-python](https://github.com/abetlen/llama-cpp-python#installation-from-pypi-recommended) is required to run `baichuan`, `wizardlm-v1.0`, `vicuna-v1.3` and `orca`.
- [chatglm-cpp-python](https://github.com/li-plus/chatglm.cpp#getting-started) is required to run `chatglm` and `chatglm2`.

#### Installation with PyTorch
To serve PyTorch models, you need to install the following extra dependencies:
```bash
$ pip install "xinference[pytorch]"
```

#### Installation with all dependencies
If you want to serve all the supported models, install all the dependencies:
```bash
$ pip install "xinference[all]"
```


### Deployment
You can deploy Xinference locally with a single command or deploy it in a distributed cluster.
Expand Down Expand Up @@ -97,7 +116,7 @@ You can also view a web UI using the Xinference endpoint to chat with all the
builtin models. You can even **chat with two cutting-edge AI models side-by-side to compare
their performance**!

![web UI](assets/xinference-downloading.png)
![web UI](assets/demo.gif)

### Xinference CLI
Xinference provides a command line interface (CLI) for model management. Here are some useful
Expand Down
4 changes: 1 addition & 3 deletions README_ja_JP.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,6 @@ Xorbits Inference(Xinference) は、言語、音声認識、マルチモーダ
あなたや最先端のビルトインモデルを簡単にデプロイし、提供することができます。 Xorbits Inference は、
研究者、開発者、データサイエンティストを問わず、最先端の AI モデルの可能性を最大限に引き出すことができます。

![demo](assets/demo.gif)

<div align="center">
<i><a href="https://join.slack.com/t/xorbitsio/shared_invite/zt-1z3zsm9ep-87yI9YZ_B79HLB2ccTq4WA">👉 Slack コミュニティにご参加ください!</a></i>
</div>
Expand Down Expand Up @@ -89,7 +87,7 @@ Xinference が起動すると、CLI または Xinference クライアントか
また、Xinference エンドポイントを使用してウェブ UI を表示し、すべての内蔵モデルとチャットすることもできます。
**2 つの最先端 AI モデルを並べてチャットし、パフォーマンスを比較することもできます**!

![web UI](assets/xinference-downloading.png)
![web UI](assets/demo.gif)

### Xinference CLI
Xinference には、モデル管理のためのコマンドラインインターフェース(CLI)が用意されています。便利なコマンドをいくつか紹介します:
Expand Down
29 changes: 24 additions & 5 deletions README_zh_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,6 @@ Xorbits Inference(Xinference)是一个性能强大且功能全面的分布
无论你是研究者,开发者,或是数据科学家,都可以通过 Xorbits Inference 与最前沿的 AI 模型,发掘更多可能。


![demo](assets/demo.gif)

<div align="center">
<i><a href="https://join.slack.com/t/xorbitsio/shared_invite/zt-1z3zsm9ep-87yI9YZ_B79HLB2ccTq4WA">👉 立刻加入我们的 Slack 社区!</a></i>
</div>
Expand Down Expand Up @@ -48,13 +46,34 @@ Xorbits Inference(Xinference)是一个性能强大且功能全面的分布

## 快速入门
Xinference 可以通过 pip 从 PyPI 安装。我们非常推荐在安装前创建一个新的虚拟环境以避免依赖冲突。

### 安装
```bash
$ pip install "xinference[all]"
$ pip install "xinference"
```
`xinference[all]` 将会安装所有用于推理的必要依赖。如果你想要获得更高效的加速,请查看下列依赖的安装文档:
`xinference` 将会安装所有用于推理的基础依赖。

#### 支持 ggml 推理
想要利用 ggml 推理,可以用以下命令:
```bash
$ pip install "xinference[ggml]"
```
如果你想要获得更高效的加速,请查看下列依赖的安装文档:
- [llama-cpp-python](https://github.com/abetlen/llama-cpp-python#installation-from-pypi-recommended) 用于 `baichuan`, `wizardlm-v1.0`, `vicuna-v1.3` 及 `orca`.
- [chatglm-cpp-python](https://github.com/li-plus/chatglm.cpp#getting-started) 用于 `chatglm` 及 `chatglm2`.

#### 支持 PyTorch 推理
想要利用 PyTorch 推理,可以使用以下命令:
```bash
$ pip install "xinference[pytorch]"
```

#### 支持所有类型
如果想要支持推理所有支持的模型,可以安装所有的依赖:
```bash
$ pip install "xinference[all]"
```


### 部署
你可以一键进行本地部署,或按照下面的步骤将 Xinference 部署在计算集群。
Expand Down Expand Up @@ -89,7 +108,7 @@ supervisor 所在服务器的主机名或 IP 地址。
你还可以通过 web UI 与任意内置模型聊天。Xinference 甚至**支持同时与两个最前沿的 AI 模型聊天并比较它们的回复质
量**!

![web UI](assets/xinference-downloading.png)
![web UI](assets/demo.gif)

### Xinference 命令行
Xinference 提供了命令行工具用于模型管理。支持的命令包括:
Expand Down
11 changes: 11 additions & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,17 @@ all =
transformers_stream_generator
bitsandbytes
protobuf
ggml =
chatglm-cpp
llama-cpp-python>=0.1.77
pytorch =
transformers>=4.31.0
torch
accelerate>=0.20.3
sentencepiece
transformers_stream_generator
bitsandbytes
protobuf
doc =
ipython>=6.5.0
sphinx>=3.0.0,<5.0.0
Expand Down