diff --git a/doc/source/models/builtin/index.rst b/doc/source/models/builtin/index.rst index 5d621ce45d..6e1d55b906 100644 --- a/doc/source/models/builtin/index.rst +++ b/doc/source/models/builtin/index.rst @@ -4,70 +4,6 @@ Builtin Models ============== - -Xinference offers an extensive array of AI models, encompassing everything from text generation and multimodal models, -to text embedding and rerank models. - - -List the Built-in Models -============================ - -You can list all models of a certain type that are available to launch in Xinference: - -.. tabs:: - - .. code-tab:: bash shell - - xinference registrations --model-type \ - --endpoint "http://:" \ - - .. code-tab:: bash cURL - - curl http://:/v1/model_registrations/ - - .. code-tab:: python - - from xinference.client import Client - client = Client("http://:") - print(client.list_model_registrations(model_type='')) - - -The following ``MODEL_TYPE`` is supported by Xinference: - -* ``LLM`` -* ``embedding`` -* ``image`` -* ``audio`` -* ``rerank`` - - -Launch a Built-in Model -============================ - -You can launch a model in Xinference either via command line or Xinference's Python client: - -.. tabs:: - - .. code-tab:: bash shell - - xinference launch --model-name \ - --model-type \ - --endpoint "http://:" \ - - - .. code-tab:: python - - from xinference.client import Client - - client = Client("http://:") - model_uid = client.launch_model(model_name="", model_type="") - print(model_uid) - - -For model type ``LLM``, launching the model requires not only specifying the model name, but also the size of the parameters -and the model format. Please refer to the list of LLM :ref:`model families `. - - .. toctree:: :maxdepth: 1 diff --git a/doc/source/models/builtin/rerank/index.rst b/doc/source/models/builtin/rerank/index.rst index 8e6fde00a6..a8b6e9e479 100644 --- a/doc/source/models/builtin/rerank/index.rst +++ b/doc/source/models/builtin/rerank/index.rst @@ -1,4 +1,4 @@ -.. _models_rarank_index: +.. _models_rerank_index: ================ Rerank Models diff --git a/doc/source/models/index.rst b/doc/source/models/index.rst index 61268772ac..c3e3de6350 100644 --- a/doc/source/models/index.rst +++ b/doc/source/models/index.rst @@ -4,10 +4,151 @@ Models ====== +List Models +============================ + +You can list all models of a certain type that are available to launch in Xinference: + +.. tabs:: + + .. code-tab:: bash shell + + xinference registrations --model-type \ + --endpoint "http://:" \ + + .. code-tab:: bash cURL + + curl http://:/v1/model_registrations/ + + .. code-tab:: python + + from xinference.client import Client + client = Client("http://:") + print(client.list_model_registrations(model_type='')) + +The following ``MODEL_TYPE`` is supported by Xinference: + +.. list-table:: + :widths: 25 50 50 + :header-rows: 1 + + * - Type + - Description + - Index + + * - LLM + - Text generation models or large language models + - :ref:`Index ` + + * - embedding + - Text embeddings models + - :ref:`Index ` + + * - image + - Image generation or manipulation models + - :ref:`Index ` + + * - audio + - Audio models + - :ref:`Index ` + + * - rerank + - Rerank models + - :ref:`Index ` + + +You can see all the built-in models supported by xinference :ref:`here `. If the model +you need is not available, xinference also allows you to register your own :ref:`custom models `. + +Launch Model +============================ + +You can launch a model in Xinference either via command line or Xinference's Python client: + +.. tabs:: + + .. code-tab:: bash shell + + xinference launch --model-name \ + --model-type \ + --endpoint "http://:" \ + + + .. code-tab:: python + + from xinference.client import Client + + client = Client("http://:") + model_uid = client.launch_model(model_name="", model_type="") + print(model_uid) + + +For model type ``LLM``, launching the model requires not only specifying the model name, but also the size of the parameters +and the model format. Please refer to the list of LLM :ref:`model families `. + + +Model Usage +============================ + + +.. grid:: 2 + + .. grid-item-card:: Chat & Generate + :link: chat + :link-type: ref + + Learn how to chat with LLMs in Xinference. + + .. grid-item-card:: Tools + :link: tools + :link-type: ref + + Learn how to connect LLM with external tools. + + +.. grid:: 2 + + .. grid-item-card:: Embeddings + :link: embed + :link-type: ref + + Learn how to create text embeddings in Xinference. + + .. grid-item-card:: Rerank + :link: rerank + :link-type: ref + + Learn how to use rerank models in Xinference. + + +.. grid:: 2 + + .. grid-item-card:: Images + :link: image + :link-type: ref + + Learn how to generate images with Xinference. + + .. grid-item-card:: Vision + :link: vision + :link-type: ref + + Learn how to process image with LLMs. + + +.. grid:: 2 + + .. grid-item-card:: Audio + :link: audio + :link-type: ref + + Learn how to turn audio into text or text into audio with Xinference. + .. toctree:: :maxdepth: 2 + model_abilities/index builtin/index custom sources/sources diff --git a/doc/source/user_guide/model_abilities/audio.rst b/doc/source/models/model_abilities/audio.rst similarity index 100% rename from doc/source/user_guide/model_abilities/audio.rst rename to doc/source/models/model_abilities/audio.rst diff --git a/doc/source/user_guide/model_abilities/chat.rst b/doc/source/models/model_abilities/chat.rst similarity index 100% rename from doc/source/user_guide/model_abilities/chat.rst rename to doc/source/models/model_abilities/chat.rst diff --git a/doc/source/user_guide/model_abilities/embed.rst b/doc/source/models/model_abilities/embed.rst similarity index 100% rename from doc/source/user_guide/model_abilities/embed.rst rename to doc/source/models/model_abilities/embed.rst diff --git a/doc/source/user_guide/model_abilities/image.rst b/doc/source/models/model_abilities/image.rst similarity index 100% rename from doc/source/user_guide/model_abilities/image.rst rename to doc/source/models/model_abilities/image.rst diff --git a/doc/source/user_guide/model_abilities/index.rst b/doc/source/models/model_abilities/index.rst similarity index 100% rename from doc/source/user_guide/model_abilities/index.rst rename to doc/source/models/model_abilities/index.rst diff --git a/doc/source/user_guide/model_abilities/rerank.rst b/doc/source/models/model_abilities/rerank.rst similarity index 100% rename from doc/source/user_guide/model_abilities/rerank.rst rename to doc/source/models/model_abilities/rerank.rst diff --git a/doc/source/user_guide/model_abilities/tools.rst b/doc/source/models/model_abilities/tools.rst similarity index 100% rename from doc/source/user_guide/model_abilities/tools.rst rename to doc/source/models/model_abilities/tools.rst diff --git a/doc/source/user_guide/model_abilities/vision.rst b/doc/source/models/model_abilities/vision.rst similarity index 100% rename from doc/source/user_guide/model_abilities/vision.rst rename to doc/source/models/model_abilities/vision.rst