[Doc] Restructure core API docs (#32236)

Similar to #31204, refactor the core api reference for better layout and view.

Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>

doc/source/_toc.yml

@@ -33,7 +33,7 @@ parts:
               - file: ray-core/examples/batch_training
               - file: ray-core/examples/automl_for_time_series
               - file: ray-core/examples/web-crawler
-          - file: ray-core/api
+          - file: ray-core/api/index
 
       - file: cluster/getting-started
         title: "Ray Clusters"

doc/source/cluster/running-applications/job-submission/jobs-package-ref.rst

@@ -8,7 +8,7 @@ Python SDK API Reference
 For an overview with examples see :ref:`Ray Jobs <jobs-overview>`.
 
 For the CLI reference see :ref:`Ray Job Submission CLI Reference <ray-job-submission-cli-ref>`.
- 
+
 .. _job-submission-client-ref:
 
 JobSubmissionClient

doc/source/ray-core/actors.rst

@@ -337,7 +337,7 @@ By default, Ray actors won't be :ref:`restarted <fault-tolerance-actors>` and
 actor tasks won't be retried when actors crash unexpectedly.
 You can change this behavior by setting
 ``max_restarts`` and ``max_task_retries`` options
-in :ref:`ray.remote() <ray-remote-ref>` and :ref:`.options() <ray-options-ref>`.
+in :func:`ray.remote() <ray.remote>` and :meth:`.options() <ray.actor.ActorClass.options>`.
 See :ref:`Ray fault tolerance <fault-tolerance>` for more details.
 
 FAQ: Actors, Workers and Resources

doc/source/ray-core/actors/actor-utils.rst

@@ -11,7 +11,7 @@ Actor Pool
 
     .. literalinclude:: ../doc_code/actor-pool.py
 
-    See the :ref:`package reference <ray-actor-pool-ref>` for more information.
+    See the :class:`package reference <ray.util.ActorPool>` for more information.
 
 .. tabbed:: Java
 
@@ -25,7 +25,7 @@ Message passing using Ray Queue
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Sometimes just using one signal to synchronize is not enough. If you need to send data among many tasks or
-actors, you can use :ref:`ray.util.queue.Queue <ray-queue-ref>`.
+actors, you can use :class:`ray.util.queue.Queue <ray.util.queue.Queue>`.
 
 .. literalinclude:: ../doc_code/actor-queue.py
 

doc/source/ray-core/api/core.rst

@@ -0,0 +1,60 @@
+Core API
+========
+
+.. autosummary::
+    :toctree: doc/
+
+    ray.init
+    ray.shutdown
+    ray.is_initialized
+
+Tasks
+-----
+
+.. autosummary::
+    :toctree: doc/
+
+    ray.remote
+    ray.remote_function.RemoteFunction.options
+    ray.cancel
+
+Actors
+------
+
+.. autosummary::
+    :toctree: doc/
+
+    ray.remote
+    ray.actor.ActorClass.options
+    ray.method
+    ray.get_actor
+    ray.kill
+
+Objects
+-------
+
+.. autosummary::
+    :toctree: doc/
+
+    ray.get
+    ray.wait
+    ray.put
+
+.. _runtime-context-apis:
+
+Runtime Context
+---------------
+.. autosummary::
+    :toctree: doc/
+
+    ray.runtime_context.get_runtime_context
+    ray.runtime_context.RuntimeContext
+    ray.get_gpu_ids
+
+Cross Language
+--------------
+.. autosummary::
+    :toctree: doc/
+
+    ray.cross_language.java_function
+    ray.cross_language.java_actor_class

doc/source/ray-core/api/exceptions.rst

@@ -0,0 +1,32 @@
+.. _ray-core-exceptions:
+
+Exceptions
+==========
+
+.. autosummary::
+    :toctree: doc/
+
+    ray.exceptions.RayError
+    ray.exceptions.RayTaskError
+    ray.exceptions.RayActorError
+    ray.exceptions.TaskCancelledError
+    ray.exceptions.TaskUnschedulableError
+    ray.exceptions.ActorUnschedulableError
+    ray.exceptions.AsyncioActorExit
+    ray.exceptions.LocalRayletDiedError
+    ray.exceptions.WorkerCrashedError
+    ray.exceptions.TaskPlacementGroupRemoved
+    ray.exceptions.ActorPlacementGroupRemoved
+    ray.exceptions.ObjectStoreFullError
+    ray.exceptions.OutOfDiskError
+    ray.exceptions.ObjectLostError
+    ray.exceptions.ObjectFetchTimedOutError
+    ray.exceptions.GetTimeoutError
+    ray.exceptions.OwnerDiedError
+    ray.exceptions.PlasmaObjectNotAvailable
+    ray.exceptions.ObjectReconstructionFailedError
+    ray.exceptions.ObjectReconstructionFailedMaxAttemptsExceededError
+    ray.exceptions.ObjectReconstructionFailedLineageEvictedError
+    ray.exceptions.RuntimeEnvSetupError
+    ray.exceptions.CrossLanguageError
+    ray.exceptions.RaySystemError

doc/source/ray-core/api/index.rst

@@ -0,0 +1,14 @@
+Ray Core API
+============
+
+.. toctree::
+    :maxdepth: 2
+
+    core.rst
+    scheduling.rst
+    runtime-env.rst
+    utility.rst
+    exceptions.rst
+    cli.rst
+    ../../ray-observability/api/state/cli.rst
+    ../../ray-observability/api/state/api.rst

doc/source/ray-core/api/runtime-env.rst

@@ -0,0 +1,8 @@
+Runtime Env API
+===============
+
+.. autosummary::
+    :toctree: doc/
+
+    ray.runtime_env.RuntimeEnvConfig
+    ray.runtime_env.RuntimeEnv

doc/source/ray-core/api/scheduling.rst

@@ -0,0 +1,25 @@
+Scheduling API
+==============
+
+Scheduling Strategy
+-------------------
+
+.. autosummary::
+    :toctree: doc/
+
+    ray.util.scheduling_strategies.PlacementGroupSchedulingStrategy
+    ray.util.scheduling_strategies.NodeAffinitySchedulingStrategy
+
+.. _ray-placement-group-ref:
+
+Placement Group
+---------------
+
+.. autosummary::
+    :toctree: doc/
+
+    ray.util.placement_group.placement_group
+    ray.util.placement_group.PlacementGroup
+    ray.util.placement_group.placement_group_table
+    ray.util.placement_group.remove_placement_group
+    ray.util.placement_group.get_current_placement_group

doc/source/ray-core/api/utility.rst

@@ -0,0 +1,35 @@
+Utility
+=======
+
+.. autosummary::
+   :toctree: doc/
+
+   ray.util.ActorPool
+   ray.util.queue.Queue
+   ray.nodes
+   ray.cluster_resources
+   ray.available_resources
+
+.. _custom-metric-api-ref:
+
+Custom Metrics
+--------------
+
+.. autosummary::
+   :toctree: doc/
+
+   ray.util.metrics.Counter
+   ray.util.metrics.Gauge
+   ray.util.metrics.Histogram
+
+.. _package-ref-debugging-apis:
+
+Debugging
+---------
+
+.. autosummary::
+   :toctree: doc/
+
+   ray.util.pdb.set_trace
+   ray.util.inspect_serializability
+   ray.timeline

doc/source/ray-core/configure.rst

@@ -57,12 +57,12 @@ If using the command line, connect to the Ray cluster as follow:
 .. _omp-num-thread-note:
 
 .. note::
-    Ray sets the environment variable ``OMP_NUM_THREADS=<num_cpus>`` if ``num_cpus`` is set on 
-    the task/actor via :ref:`ray.remote() <ray-remote-ref>` and :ref:`.options() <ray-options-ref>`.
-    Ray sets ``OMP_NUM_THREADS=1`` if ``num_cpus`` is not specified; this 
+    Ray sets the environment variable ``OMP_NUM_THREADS=<num_cpus>`` if ``num_cpus`` is set on
+    the task/actor via :func:`ray.remote() <ray.remote>` and :meth:`task.options() <ray.remote_function.RemoteFunction.options>`/:meth:`actor.options() <ray.actor.ActorClass.options>`.
+    Ray sets ``OMP_NUM_THREADS=1`` if ``num_cpus`` is not specified; this
     is done to avoid performance degradation with many workers (issue #6998). You can
-    also override this by explicitly setting ``OMP_NUM_THREADS`` to override anything Ray sets by default. 
-    ``OMP_NUM_THREADS`` is commonly used in numpy, PyTorch, and Tensorflow to perform multi-threaded 
+    also override this by explicitly setting ``OMP_NUM_THREADS`` to override anything Ray sets by default.
+    ``OMP_NUM_THREADS`` is commonly used in numpy, PyTorch, and Tensorflow to perform multi-threaded
     linear algebra. In multi-worker setting, we want one thread per worker instead of many threads
     per worker to avoid contention. Some other libraries may have their own way to configure
     parallelism. For example, if you're using OpenCV, you should manually set the number of

doc/source/ray-core/examples/monte_carlo_pi.rst

@@ -19,7 +19,7 @@ To get started, install Ray via ``pip install -U ray``. See :ref:`Installing Ray
 
 Starting Ray
 ------------
-First, let's include all modules needed for this tutorial and start a local Ray cluster with :ref:`ray.init() <ray-init-ref>`:
+First, let's include all modules needed for this tutorial and start a local Ray cluster with :func:`ray.init() <ray.init>`:
 
 .. literalinclude:: ../doc_code/monte_carlo_pi.py
     :language: python
@@ -41,7 +41,7 @@ Ray actors are essentially stateful services that anyone with an instance (a han
     :start-after: __defining_actor_start__
     :end-before: __defining_actor_end__
 
-We define a Ray actor by decorating a normal Python class with :ref:`ray.remote <ray-remote-ref>`.
+We define a Ray actor by decorating a normal Python class with :func:`ray.remote <ray.remote>`.
 The progress actor has ``report_progress()`` method that will be called by sampling tasks to update their progress individually
 and ``get_progress()`` method to get the overall progress.
 
@@ -55,7 +55,7 @@ Ray tasks are stateless functions. They execute asynchronously, and run in paral
     :start-after: __defining_task_start__
     :end-before: __defining_task_end__
 
-To convert a normal Python function as a Ray task, we decorate the function with :ref:`ray.remote <ray-remote-ref>`.
+To convert a normal Python function as a Ray task, we decorate the function with :func:`ray.remote <ray.remote>`.
 The sampling task takes a progress actor handle as an input and reports progress to it.
 The above code shows an example of calling actor methods from tasks.
 
@@ -97,7 +97,7 @@ While sampling tasks are running, we can periodically query the progress by call
 To call an actor method, use ``actor_handle.method.remote()``.
 This invocation immediately returns an ``ObjectRef`` as a future
 and then executes the method asynchronously on the remote actor process.
-To fetch the actual returned value of ``ObjectRef``, we use the blocking :ref:`ray.get() <ray-get-ref>`.
+To fetch the actual returned value of ``ObjectRef``, we use the blocking :func:`ray.get() <ray.get>`.
 
 Calculating π
 -------------
@@ -108,7 +108,7 @@ Finally, we get number of samples inside the circle from the remote sampling tas
     :start-after: __calculating_pi_start__
     :end-before: __calculating_pi_end__
 
-As we can see from the above code, besides a single ``ObjectRef``, :ref:`ray.get() <ray-get-ref>` can also take a list of ``ObjectRef`` and return a list of results.
+As we can see from the above code, besides a single ``ObjectRef``, :func:`ray.get() <ray.get>` can also take a list of ``ObjectRef`` and return a list of results.
 
 If you run this tutorial, you will see output like:
 

doc/source/ray-core/fault_tolerance/actors.rst

@@ -24,7 +24,7 @@ After the specified number of restarts, subsequent actor methods will
 raise a ``RayActorError``.
 
 By default, actor tasks execute with at-most-once semantics
-(``max_task_retries=0`` in the ``@ray.remote`` :ref:`decorator <ray-remote-ref>`). This means that if an
+(``max_task_retries=0`` in the ``@ray.remote`` :func:`decorator <ray.remote>`). This means that if an
 actor task is submitted to an actor that is unreachable, Ray will report the
 error with ``RayActorError``, a Python-level exception that is thrown when
 ``ray.get`` is called on the future returned by the task. Note that this

doc/source/ray-core/fault_tolerance/tasks.rst

@@ -71,7 +71,7 @@ If a task is hanging, you may want to cancel the task to continue to make
 progress. You can do this by calling ``ray.cancel`` on an ``ObjectRef``
 returned by the task. By default, this will send a KeyboardInterrupt to the
 task's worker if it is mid-execution.  Passing ``force=True`` to ``ray.cancel``
-will force-exit the worker. See :ref:`the API reference <ray-cancel-ref>` for
+will force-exit the worker. See :func:`the API reference <ray.cancel>` for
 ``ray.cancel`` for more details.
 
 Note that currently, Ray will not automatically retry tasks that have been

doc/source/ray-core/objects.rst

@@ -12,7 +12,7 @@ similar.
 Object refs can be created in two ways.
 
   1. They are returned by remote function calls.
-  2. They are returned by ``put`` (:ref:`docstring <ray-put-ref>`).
+  2. They are returned by :func:`ray.put() <ray.put>`.
 
 .. tabbed:: Python
 
@@ -48,7 +48,7 @@ Object refs can be created in two ways.
 Fetching Object Data
 --------------------
 
-You can use the ``get`` method (:ref:`docstring <ray-get-ref>`) to fetch the result of a remote object from an object ref.
+You can use the :func:`ray.get() <ray.get>` method to fetch the result of a remote object from an object ref.
 If the current node's object store does not contain the object, the object is downloaded.
 
 .. tabbed:: Python