DOC: Add image model doc (#947)

xorbitsai · Jan 30, 2024 · 5dad18a · 5dad18a
1 parent 7c899cc
commit 5dad18a
Show file tree

Hide file tree

Showing 15 changed files with 173 additions and 14 deletions.
diff --git a/doc/source/gen_docs.py b/doc/source/gen_docs.py
@@ -138,5 +138,23 @@ def get_unique_id(spec):
             rendered_index = env.get_template('rerank_index.rst.jinja').render(models=sorted_models)
             file.write(rendered_index)
 
+    with open('../../xinference/model/image/model_spec.json', 'r') as file:
+        models = json.load(file)
+
+        sorted_models = sorted(models, key=lambda x: x['model_name'].lower())
+        output_dir = './models/builtin/image'
+        os.makedirs(output_dir, exist_ok=True)
+
+        for model in sorted_models:
+            rendered = env.get_template('image.rst.jinja').render(model)
+            output_file_path = os.path.join(output_dir, f"{model['model_name'].lower()}.rst")
+            with open(output_file_path, 'w') as output_file:
+                output_file.write(rendered)
+
+        index_file_path = os.path.join(output_dir, "index.rst")
+        with open(index_file_path, "w") as file:
+            rendered_index = env.get_template('image_index.rst.jinja').render(models=sorted_models)
+            file.write(rendered_index)
+
 if __name__ == "__main__":
     main()
diff --git a/doc/source/models/builtin/image/index.rst b/doc/source/models/builtin/image/index.rst
@@ -0,0 +1,21 @@
+.. _models_image_index:
+
+================
+Image Models
+================
+
+The following is a list of built-in image models in Xinference:
+
+
+.. toctree::
+   :maxdepth: 1
+
+
+   sd-turbo
+
+   sdxl-turbo
+
+   stable-diffusion-v1.5
+
+   stable-diffusion-xl-base-1.0
+
diff --git a/doc/source/models/builtin/image/sd-turbo.rst b/doc/source/models/builtin/image/sd-turbo.rst
@@ -0,0 +1,17 @@
+.. _models_builtin_sd-turbo:
+
+========
+sd-turbo
+========
+
+- **Model Name:** sd-turbo
+- **Abilities:** text-to-image
+
+Specifications
+^^^^^^^^^^^^^^
+
+- **Model ID:** stabilityai/sd-turbo
+
+Execute the following command to launch the model::
+
+   xinference launch --model-name sd-turbo --model-type image
diff --git a/doc/source/models/builtin/image/sdxl-turbo.rst b/doc/source/models/builtin/image/sdxl-turbo.rst
@@ -0,0 +1,17 @@
+.. _models_builtin_sdxl-turbo:
+
+==========
+sdxl-turbo
+==========
+
+- **Model Name:** sdxl-turbo
+- **Abilities:** text-to-image
+
+Specifications
+^^^^^^^^^^^^^^
+
+- **Model ID:** stabilityai/sdxl-turbo
+
+Execute the following command to launch the model::
+
+   xinference launch --model-name sdxl-turbo --model-type image
diff --git a/doc/source/models/builtin/image/stable-diffusion-v1.5.rst b/doc/source/models/builtin/image/stable-diffusion-v1.5.rst
@@ -0,0 +1,17 @@
+.. _models_builtin_stable-diffusion-v1.5:
+
+=====================
+stable-diffusion-v1.5
+=====================
+
+- **Model Name:** stable-diffusion-v1.5
+- **Abilities:** text-to-image
+
+Specifications
+^^^^^^^^^^^^^^
+
+- **Model ID:** runwayml/stable-diffusion-v1-5
+
+Execute the following command to launch the model::
+
+   xinference launch --model-name stable-diffusion-v1.5 --model-type image
diff --git a/doc/source/models/builtin/image/stable-diffusion-xl-base-1.0.rst b/doc/source/models/builtin/image/stable-diffusion-xl-base-1.0.rst
@@ -0,0 +1,17 @@
+.. _models_builtin_stable-diffusion-xl-base-1.0:
+
+============================
+stable-diffusion-xl-base-1.0
+============================
+
+- **Model Name:** stable-diffusion-xl-base-1.0
+- **Abilities:** text-to-image
+
+Specifications
+^^^^^^^^^^^^^^
+
+- **Model ID:** stabilityai/stable-diffusion-xl-base-1.0
+
+Execute the following command to launch the model::
+
+   xinference launch --model-name stable-diffusion-xl-base-1.0 --model-type image
diff --git a/doc/source/models/builtin/index.rst b/doc/source/models/builtin/index.rst
@@ -35,7 +35,6 @@ You can list all models of a certain type that are available to launch in Xinfer
 The following ``MODEL_TYPE`` is supported by Xinference:
 
 * ``LLM``   
-* ``multimodal``
 * ``embedding``
 * ``image`` 
 * ``rerank``
@@ -73,4 +72,5 @@ and the model format.  Please refer to the list of LLM :ref:`model families <mod
 
    llm/index
    embedding/index
-   rerank/index
+   image/index
+   rerank/index
diff --git a/doc/source/models/builtin/llm/index.rst b/doc/source/models/builtin/llm/index.rst
@@ -4,7 +4,7 @@
 Large language Models
 =====================
 
-The following is a list of built-in LLM in Xinference. Each entry represents a model family.
+The following is a list of built-in LLM in Xinference:
 
 .. list-table::
    :widths: 25 25 25 50
@@ -52,7 +52,7 @@ The following is a list of built-in LLM in Xinference. Each entry represents a m
      - ChatGLM2-32k is a special version of ChatGLM2, with a context window of 32k tokens instead of 8k.
 
    * - :ref:`chatglm3 <models_llm_chatglm3>`
-     - chat
+     - chat, tools
      - 8192
      - ChatGLM3 is the third generation of ChatGLM, still open-source and trained on Chinese and English data.
 
@@ -131,6 +131,11 @@ The following is a list of built-in LLM in Xinference. Each entry represents a m
      - 4096
      - Internlm-chat is a fine-tuned version of the Internlm LLM, specializing in chatting.
 
+   * - :ref:`internlm2-chat <models_llm_internlm2-chat>`
+     - chat
+     - 204800
+     - The second generation of the InternLM model, InternLM2.
+
    * - :ref:`llama-2 <models_llm_llama-2>`
      - generate
      - 4096
@@ -186,16 +191,31 @@ The following is a list of built-in LLM in Xinference. Each entry represents a m
      - 2048
      - Orca is an LLM trained by fine-tuning LLaMA on explanation traces obtained from GPT-4.
 
+   * - :ref:`orion-chat <models_llm_orion-chat>`
+     - chat
+     - 4096
+     - Orion-14B series models are open-source multilingual large language models trained from scratch by OrionStarAI.
+
+   * - :ref:`orion-chat-rag <models_llm_orion-chat-rag>`
+     - chat
+     - 4096
+     - Orion-14B series models are open-source multilingual large language models trained from scratch by OrionStarAI.
+
    * - :ref:`phi-2 <models_llm_phi-2>`
      - generate
      - 2048
      - Phi-2 is a 2.7B Transformer based LLM used for research on model safety, trained with data similar to Phi-1.5 but augmented with synthetic texts and curated websites.
 
    * - :ref:`qwen-chat <models_llm_qwen-chat>`
-     - chat
-     - 2048
+     - chat, tools
+     - 32768
      - Qwen-chat is a fine-tuned version of the Qwen LLM trained with alignment techniques, specializing in chatting.
 
+   * - :ref:`qwen-vl-chat <models_llm_qwen-vl-chat>`
+     - chat, vision
+     - 4096
+     - Qwen-VL-Chat supports more flexible interaction, such as multiple image inputs, multi-round question answering, and creative capabilities.
+
    * - :ref:`skywork <models_llm_skywork>`
      - generate
      - 4096

diff --git a/doc/source/models/builtin/llm/internlm2-chat.rst b/doc/source/models/builtin/llm/internlm2-chat.rst
@@ -21,7 +21,7 @@ Model Spec 1 (pytorch, 7 Billion)
 - **Model Size (in billions):** 7
 - **Quantizations:** none
 - **Model ID:** internlm/internlm2-chat-7b
-- **Model Hubs**:  `Hugging Face <https://huggingface.co/internlm/internlm2-chat-7b>`_, `ModelScope <https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-7b>`_
+- **Model Hubs**:  `Hugging Face <https://huggingface.co/internlm/internlm2-chat-7b>`__, `ModelScope <https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-7b>`__
 
 Execute the following command to launch the model, remember to replace ``${quantization}`` with your
 chosen quantization method from the options listed above::
@@ -36,7 +36,7 @@ Model Spec 2 (pytorch, 20 Billion)
 - **Model Size (in billions):** 20
 - **Quantizations:** none
 - **Model ID:** internlm/internlm2-chat-20b
-- **Model Hubs**:  `Hugging Face <https://huggingface.co/internlm/internlm2-chat-20b>`_, `ModelScope <https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-20b>`_
+- **Model Hubs**:  `Hugging Face <https://huggingface.co/internlm/internlm2-chat-20b>`__, `ModelScope <https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-20b>`__
 
 Execute the following command to launch the model, remember to replace ``${quantization}`` with your
 chosen quantization method from the options listed above::

diff --git a/doc/source/models/builtin/llm/mixtral-instruct-v0.1.rst b/doc/source/models/builtin/llm/mixtral-instruct-v0.1.rst
@@ -36,7 +36,7 @@ Model Spec 2 (awq, 46_7 Billion)
 - **Model Size (in billions):** 46_7
 - **Quantizations:** 4-bit
 - **Model ID:** TheBloke/Mixtral-8x7B-Instruct-v0.1-AWQ
-- **Model Hubs**:  `Hugging Face <https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-AWQ>`_
+- **Model Hubs**:  `Hugging Face <https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-AWQ>`__
 
 Execute the following command to launch the model, remember to replace ``${quantization}`` with your
 chosen quantization method from the options listed above::

diff --git a/doc/source/models/builtin/llm/orion-chat-rag.rst b/doc/source/models/builtin/llm/orion-chat-rag.rst
@@ -21,7 +21,7 @@ Model Spec 1 (pytorch, 14 Billion)
 - **Model Size (in billions):** 14
 - **Quantizations:** none, 4-bit, 8-bit
 - **Model ID:** OrionStarAI/Orion-14B-Chat-RAG
-- **Model Hubs**:  `Hugging Face <https://huggingface.co/OrionStarAI/Orion-14B-Chat-RAG>`_, `ModelScope <https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat-RAG>`_
+- **Model Hubs**:  `Hugging Face <https://huggingface.co/OrionStarAI/Orion-14B-Chat-RAG>`__, `ModelScope <https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat-RAG>`__
 
 Execute the following command to launch the model, remember to replace ``${quantization}`` with your
 chosen quantization method from the options listed above::

diff --git a/doc/source/models/builtin/llm/orion-chat.rst b/doc/source/models/builtin/llm/orion-chat.rst
@@ -21,7 +21,7 @@ Model Spec 1 (pytorch, 14 Billion)
 - **Model Size (in billions):** 14
 - **Quantizations:** none, 4-bit, 8-bit
 - **Model ID:** OrionStarAI/Orion-14B-Chat
-- **Model Hubs**:  `Hugging Face <https://huggingface.co/OrionStarAI/Orion-14B-Chat>`_, `ModelScope <https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat>`_
+- **Model Hubs**:  `Hugging Face <https://huggingface.co/OrionStarAI/Orion-14B-Chat>`__, `ModelScope <https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat>`__
 
 Execute the following command to launch the model, remember to replace ``${quantization}`` with your
 chosen quantization method from the options listed above::
@@ -36,7 +36,7 @@ Model Spec 2 (awq, 14 Billion)
 - **Model Size (in billions):** 14
 - **Quantizations:** Int4
 - **Model ID:** OrionStarAI/Orion-14B-Chat-{quantization}
-- **Model Hubs**:  `Hugging Face <https://huggingface.co/OrionStarAI/Orion-14B-Chat-{quantization}>`_, `ModelScope <https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat-{quantization}>`_
+- **Model Hubs**:  `Hugging Face <https://huggingface.co/OrionStarAI/Orion-14B-Chat-{quantization}>`__, `ModelScope <https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat-{quantization}>`__
 
 Execute the following command to launch the model, remember to replace ``${quantization}`` with your
 chosen quantization method from the options listed above::

diff --git a/doc/source/models/builtin/llm/qwen-vl-chat.rst b/doc/source/models/builtin/llm/qwen-vl-chat.rst
@@ -21,7 +21,7 @@ Model Spec 1 (pytorch, 7 Billion)
 - **Model Size (in billions):** 7
 - **Quantizations:** none
 - **Model ID:** Qwen/Qwen-VL-Chat
-- **Model Hubs**:  `Hugging Face <https://huggingface.co/Qwen/Qwen-VL-Chat>`_, `ModelScope <https://modelscope.cn/models/Qwen/Qwen-VL-Chat>`_
+- **Model Hubs**:  `Hugging Face <https://huggingface.co/Qwen/Qwen-VL-Chat>`__, `ModelScope <https://modelscope.cn/models/Qwen/Qwen-VL-Chat>`__
 
 Execute the following command to launch the model, remember to replace ``${quantization}`` with your
 chosen quantization method from the options listed above::
@@ -36,7 +36,7 @@ Model Spec 2 (gptq, 7 Billion)
 - **Model Size (in billions):** 7
 - **Quantizations:** Int4
 - **Model ID:** Qwen/Qwen-VL-Chat-{quantization}
-- **Model Hubs**:  `Hugging Face <https://huggingface.co/Qwen/Qwen-VL-Chat-{quantization}>`_, `ModelScope <https://modelscope.cn/models/Qwen/Qwen-VL-Chat-{quantization}>`_
+- **Model Hubs**:  `Hugging Face <https://huggingface.co/Qwen/Qwen-VL-Chat-{quantization}>`__, `ModelScope <https://modelscope.cn/models/Qwen/Qwen-VL-Chat-{quantization}>`__
 
 Execute the following command to launch the model, remember to replace ``${quantization}`` with your
 chosen quantization method from the options listed above::

diff --git a/doc/templates/image.rst.jinja b/doc/templates/image.rst.jinja
@@ -0,0 +1,17 @@
+.. _models_builtin_{{ model_name|lower }}:
+
+{{ "=" * model_name|length }}
+{{ model_name }}
+{{ "=" * model_name|length }}
+
+- **Model Name:** {{ model_name }}
+- **Abilities:** text-to-image
+
+Specifications
+^^^^^^^^^^^^^^
+
+- **Model ID:** {{ model_id }}
+
+Execute the following command to launch the model::
+
+   xinference launch --model-name {{ model_name }} --model-type image
diff --git a/doc/templates/image_index.rst.jinja b/doc/templates/image_index.rst.jinja
@@ -0,0 +1,15 @@
+.. _models_image_index:
+
+================
+Image Models
+================
+
+The following is a list of built-in image models in Xinference:
+
+
+.. toctree::
+   :maxdepth: 1
+
+  {% for model in models %}
+   {{ model.model_name|lower }}
+  {% endfor %}