[Docs] Add docs for run many jobs (skypilot-org#3847)

* Add docs for running N jobs * Fix language * Fix job queue * Add link to managed jobs * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Add script for generating config files * Fix comments * fix title * fix title * fix * reduce image size * restructure * rename * adopt comments * Add benefits * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * update * rename * fix * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update docs/source/running-jobs/many-jobs.rst Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Minor fix for comments --------- Co-authored-by: Zongheng Yang <zongheng.y@gmail.com>
asaiacai · Aug 29, 2024 · 95b52c0 · 95b52c0
1 parent bd40b93
commit 95b52c0
Show file tree

Hide file tree

Showing 8 changed files with 363 additions and 15 deletions.
diff --git a/docs/source/_static/custom.js b/docs/source/_static/custom.js
@@ -27,8 +27,8 @@ document.addEventListener('DOMContentLoaded', () => {
     const newItems = [
         { selector: '.caption-text', text: 'SkyServe: Model Serving' },
         { selector: '.toctree-l1 > a', text: 'Managed Jobs' },
-        { selector: '.toctree-l1 > a', text: 'Running on Kubernetes' },
         { selector: '.toctree-l1 > a', text: 'Llama-3.1 (Meta)' },
+        { selector: '.toctree-l1 > a', text: 'Many Parallel Jobs' },
     ];
     newItems.forEach(({ selector, text }) => {
         document.querySelectorAll(selector).forEach((el) => {

diff --git a/docs/source/developers/index.rst b/docs/source/developers/index.rst
@@ -0,0 +1,8 @@
+Developer Guides
+=================
+
+.. toctree::
+   :maxdepth: 1
+
+   ../developers/CONTRIBUTING
+   Guide: Adding a New Cloud <https://docs.google.com/document/d/1oWox3qb3Kz3wXXSGg9ZJWwijoa99a3PIQUHBR8UgEGs/edit?usp=sharing>
diff --git a/docs/source/docs/index.rst b/docs/source/docs/index.rst
@@ -129,8 +129,8 @@ Read the research:
 
    ../getting-started/installation
    ../getting-started/quickstart
-   ../getting-started/tutorial
    ../examples/interactive-development
+   ../getting-started/tutorial
 
 
 .. toctree::
@@ -143,6 +143,7 @@ Read the research:
    ../examples/auto-failover
    ../reference/kubernetes/index
    ../running-jobs/distributed-jobs
+   ../running-jobs/many-jobs
 
 .. toctree::
    :hidden:
@@ -184,14 +185,6 @@ Read the research:
    SkyPilot vs. Other Systems <../reference/comparison>
 
 
-.. toctree::
-   :hidden:
-   :maxdepth: 1
-   :caption: Developer Guides
-
-   ../developers/CONTRIBUTING
-   Guide: Adding a New Cloud <https://docs.google.com/document/d/1oWox3qb3Kz3wXXSGg9ZJWwijoa99a3PIQUHBR8UgEGs/edit?usp=sharing>
-
 .. toctree::
    :hidden:
    :maxdepth: 1
@@ -210,4 +203,5 @@ Read the research:
    ../reference/cli
    ../reference/api
    ../reference/config
+   ../developers/index
 
diff --git a/docs/source/getting-started/quickstart.rst b/docs/source/getting-started/quickstart.rst
@@ -219,7 +219,7 @@ Congratulations!  In this quickstart, you have launched a cluster, run a task, a
 
 Next steps:
 
-- Adapt :ref:`Tutorial: DNN Training <dnn-training>` to start running your own project on SkyPilot!
+- Adapt :ref:`Tutorial: AI Training <ai-training>` to start running your own project on SkyPilot!
 - See the :ref:`Task YAML reference <yaml-spec>`, :ref:`CLI reference <cli>`, and `more examples <https://github.com/skypilot-org/skypilot/tree/master/examples>`_
 - To learn more, try out `SkyPilot Tutorials <https://github.com/skypilot-org/skypilot-tutorial>`_ in Jupyter notebooks
 

diff --git a/docs/source/getting-started/tutorial.rst b/docs/source/getting-started/tutorial.rst
@@ -1,6 +1,6 @@
-.. _dnn-training:
+.. _ai-training:
 
-Tutorial: DNN Training
+Tutorial: AI Training
 ======================
 This example uses SkyPilot to train a Transformer-based language model from HuggingFace.
 

diff --git a/docs/source/reference/job-queue.rst b/docs/source/reference/job-queue.rst
@@ -160,7 +160,7 @@ SkyPilot's scheduler serves two goals:
 2. **Minimizing resource idleness**: If a resource is idle, SkyPilot will schedule a
    queued job that can utilize that resource.
 
-We illustrate the scheduling behavior by revisiting :ref:`Tutorial: DNN Training <dnn-training>`.
+We illustrate the scheduling behavior by revisiting :ref:`Tutorial: AI Training <ai-training>`.
 In that tutorial, we have a task YAML that specifies these resource requirements:
 
 .. code-block:: yaml

diff --git a/docs/source/running-jobs/distributed-jobs.rst b/docs/source/running-jobs/distributed-jobs.rst
@@ -1,6 +1,6 @@
 .. _dist-jobs:
 
-Distributed Jobs on Many Nodes
+Distributed Multi-Node Jobs
 ================================================
 
 SkyPilot supports multi-node cluster