[Docs] Add mlc.ai/package to DEPENDENCY INSTALLATION group (mlc-a…

…i#1055) Co-authored-by: Junru Shao <junrushao1994@gmail.com>
masahi · Oct 12, 2023 · 98ebd28 · 98ebd28
1 parent b9179cf
commit 98ebd28
Show file tree

Hide file tree

Showing 7 changed files with 142 additions and 22 deletions.
diff --git a/docs/community/faq.rst b/docs/community/faq.rst
@@ -13,5 +13,4 @@ This is a list of Frequently Asked Questions (FAQ) about the MLC-LLM. Feel free
 
 ... Why do I encounter an error ``free(): invalid pointer, Aborted (core dumped)`` at the end of model compilation?
    This happens if you compiled TVM-Unity from source and didn't hide LLVM symbols in cmake configurations.
-   Please follow our instructions in :ref:`Building TVM Unity from Source  <tvm-unity-build-from-source>` tutorial to compile TVM-Unity which hides LLVM symbols,
-   or use our pre-builc MLC-AI pip wheels from `MLC Packages <https://mlc.ai/package/>`__.
+   Please follow our instructions in :ref:`Building TVM Unity from Source  <tvm-unity-build-from-source>` tutorial to compile TVM-Unity which hides LLVM symbols, or use our pre-built MLC-LLM :doc:`pip wheels <../install/mlc_llm>`.
diff --git a/docs/deploy/python.rst b/docs/deploy/python.rst
@@ -11,8 +11,7 @@ We also provide a web demo based on `gradio <https://gradio.app/>`_ as an exampl
 Python API
 ----------
 
-The Python API is a part of the MLC-Chat package, which we have prepared pre-built pip wheels and you can install it by
-following the instructions in `<https://mlc.ai/package/>`_.
+The Python API is a part of the MLC-Chat package, which we have prepared pre-built pip wheels via the :doc:`installation page <../install/mlc_llm>`.
 
 Verify Installation
 ^^^^^^^^^^^^^^^^^^^

diff --git a/docs/deploy/rest.rst b/docs/deploy/rest.rst
@@ -11,8 +11,7 @@ for user to interact with MLC-Chat in their own programs.
 Install MLC-Chat Package
 ------------------------
 
-The REST API is a part of the MLC-Chat package, which we have prepared pre-built pip wheels and you can install it by
-following the instructions in `<https://mlc.ai/package/>`_.
+The REST API is a part of the MLC-Chat package, which we have prepared pre-built :doc:`pip wheels <../install/mlc_llm>`.
 
 Verify Installation
 ^^^^^^^^^^^^^^^^^^^

diff --git a/docs/index.rst b/docs/index.rst
@@ -11,13 +11,13 @@ Getting Started
 ---------------
 
 To begin with, try out MLC LLM support for int4-quantized Llama2 7B.
-It is recommended to have at least 4.5GB of free VRAM to run it.
+It is recommended to have at least 6GB free VRAM to run it.
 
 .. tabs::
 
   .. tab:: Python
 
-    **Install MLC Chat**. `MLC Chat <https://mlc.ai/package/>`_ is available via pip.
+    **Install MLC Chat Python**. :doc:`MLC LLM <install/mlc_llm>` is available via pip.
     It is always recommended to install it in an isolated conda virtual environment.
 
     **Download pre-quantized weights**. The comamnds below download the int4-quantized Llama2-7B from HuggingFace:
@@ -209,6 +209,7 @@ It is recommended to have at least 4.5GB of free VRAM to run it.
    :hidden:
 
    install/tvm.rst
+   install/mlc_llm.rst
    install/conda.rst
    install/gpu.rst
    install/emcc.rst

diff --git a/docs/install/gpu.rst b/docs/install/gpu.rst
@@ -105,7 +105,7 @@ After installation, you can run ``vulkaninfo`` in command line and see if you ca
 Vulkan SDK
 ----------
 
-Vulkan SDK is required for compiling models to Vulkan backend. To build TVM Unity compiler from source, you will need to install Vulkan SDK as a dependency, but our `pre-built wheels <https://mlc.ai/package>`__ already ships with Vulkan SDK.
+Vulkan SDK is required for compiling models to Vulkan backend. To build TVM Unity compiler from source, you will need to install Vulkan SDK as a dependency, but our :doc:`pre-built wheels <../install/mlc_llm>` already ships with Vulkan SDK.
 
 Check Vulkan SDK installation guide according to your platform:
 

diff --git a/docs/install/mlc_llm.rst b/docs/install/mlc_llm.rst
@@ -0,0 +1,133 @@
+.. _install-mlc-packages:
+
+Install MLC LLM Python Package
+==============================
+
+.. contents:: Table of Contents
+    :local:
+    :depth: 2
+
+MLC LLM Python Package can be installed directly from a prebuilt developer package, or built from source.
+
+Option 1. Prebuilt Package
+--------------------------
+
+We provide nightly built pip wheels for MLC-LLM via pip.
+Select your operating system/compute platform and run the command in your terminal:
+
+.. note::
+    ❗ Whenever using Python, it is highly recommended to use **conda** to manage an isolated Python environment to avoid missing dependencies, incompatible versions, and package conflicts.
+
+.. tabs::
+
+    .. tab:: Linux
+
+        .. tabs::
+
+            .. tab:: CPU
+
+                .. code-block:: bash
+
+                    conda activate your-environment
+                    python3 -m pip install --pre --force-reinstall -f https://mlc.ai/wheels mlc-chat-nightly mlc-ai-nightly
+
+            .. tab:: CUDA 11.7
+
+                .. code-block:: bash
+
+                    conda activate your-environment
+                    python3 -m pip install --pre --force-reinstall -f https://mlc.ai/wheels mlc-chat-nightly-cu117 mlc-ai-nightly-cu117
+
+            .. tab:: CUDA 11.8
+
+                .. code-block:: bash
+
+                    conda activate your-environment
+                    python3 -m pip install --pre --force-reinstall -f https://mlc.ai/wheels mlc-chat-nightly-cu118 mlc-ai-nightly-cu118
+
+            .. tab:: CUDA 12.1
+
+                .. code-block:: bash
+
+                    conda activate your-environment
+                    python3 -m pip install --pre --force-reinstall -f https://mlc.ai/wheels mlc-chat-nightly-cu121 mlc-ai-nightly-cu121
+
+            .. tab:: CUDA 12.2
+
+                .. code-block:: bash
+
+                    conda activate your-environment
+                    python3 -m pip install --pre --force-reinstall -f https://mlc.ai/wheels mlc-chat-nightly-cu122 mlc-ai-nightly-cu122
+
+            .. tab:: ROCm 5.6
+
+                .. code-block:: bash
+
+                    conda activate your-environment
+                    python3 -m pip install --pre --force-reinstall -f https://mlc.ai/wheels mlc-chat-nightly-rocm56 mlc-ai-nightly-rocm56
+    
+            .. tab:: ROCm 5.7
+
+                .. code-block:: bash
+
+                    conda activate your-environment
+                    python3 -m pip install --pre --force-reinstall -f https://mlc.ai/wheels mlc-chat-nightly-rocm57 mlc-ai-nightly-rocm57
+
+            .. tab:: Vulkan
+
+                Supported in all Linux packages.
+
+        .. note::
+
+            If encountering issues with GLIBC not found, please install the latest glibc in conda:
+
+            .. code-block:: bash
+
+                conda install -c conda-forge libgcc-ng
+
+    .. tab:: macOS
+
+        .. tabs::
+
+            .. tab:: CPU + Metal
+
+                .. code-block:: bash
+
+                    conda activate your-environment
+                    python3 -m pip install --pre --force-reinstall -f https://mlc.ai/wheels mlc-chat-nightly mlc-ai-nightly
+
+        .. note::
+
+            Always check if conda is installed properly in macOS using the command below:
+
+            .. code-block:: bash
+
+                conda info | grep platform
+
+            It should return "osx-64" for Mac with Intel chip, and "osx-arm64" for Mac with Apple chip.
+
+    .. tab:: Windows
+
+        .. tabs::
+
+            .. tab:: CPU + Vulkan
+
+                .. code-block:: bash
+
+                    conda activate your-environment
+                    python3 -m pip install --pre --force-reinstall -f https://mlc.ai/wheels mlc-chat-nightly mlc-ai-nightly
+
+        .. note::
+            If encountering the error below:
+
+            .. code-block:: bash
+
+                FileNotFoundError: Could not find module 'path\to\site-packages\tvm\tvm.dll' (or one of its dependencies). Try using the full path with constructor syntax.
+
+            It is likely `zstd`, a dependency to LLVM, was missing. Please `download <https://github.com/facebook/zstd/releases/tag/v1.5.5>`__ the precompiled binary, rename it to `zstd.dll` and copy to the same folder as `tvm.dll`.
+
+
+Option 2. Build from Source
+---------------------------
+
+Upcoming.
diff --git a/docs/install/tvm.rst b/docs/install/tvm.rst
@@ -97,14 +97,7 @@ A nightly prebuilt Python package of Apache TVM Unity is provided.
 
       .. tabs::
 
-         .. tab:: CPU
-
-            .. code-block:: bash
-
-              conda activate your-environment
-              python3 -m pip install --pre --force-reinstall -f https://mlc.ai/wheels mlc-ai-nightly
-
-         .. tab:: Metal
+         .. tab:: CPU + Metal
 
             .. code-block:: bash
 
@@ -125,17 +118,13 @@ A nightly prebuilt Python package of Apache TVM Unity is provided.
 
       .. tabs::
 
-         .. tab:: CPU
+         .. tab:: CPU + Vulkan
 
             .. code-block:: bash
 
               conda activate your-environment
               python3 -m pip install --pre --force-reinstall -f https://mlc.ai/wheels mlc-ai-nightly
 
-         .. tab:: Vulkan
-
-            Supported in all Windows packages.
-
       .. note::
         If encountering the error below: