Update documents and check configuration

gigony · gigony · commit 12c65e57930b · 2021-09-17T13:52:11.000-07:00
Signed-off-by: Gigon Bae &lt;gbae@nvidia.com&gt;
diff --git a/docs/_static/custom.css b/docs/_static/custom.css
@@ -105,3 +105,10 @@ blockquote {
 blockquote p {
     margin-bottom: 0px;
 }
+
+/* Mermaid
+to avoid the lable text being cut off
+*/
+.edgeTerminals {
+    font-size: 9px !important;
+}
diff --git a/docs/source/developing_with_sdk/core_concepts.md b/docs/source/developing_with_sdk/core_concepts.md
@@ -1,3 +1,82 @@
 # Core concepts
 
-This document introduces the basic concepts of the MONAI Deploy App SDK. If you are eager to try out the SDK in practice, you can start with the tutorial. After the tutorial, you can return to this document to learn more about how MONAI Deploy App SDK works.
+This document introduces the basic concepts of the MONAI Deploy App SDK. If you are eager to try out the SDK in practice, you can start with the [tutorial](/getting_started/tutorials/index). After the tutorial, you can return to this document to learn more about how MONAI Deploy App SDK works.
+
+As described in the [Architecture of MONAI Deploy App SDK](/introduction/architecture), an application is implemented by subclassing [Application](/modules/_autosummary/monai.deploy.core.Application) class.
+
+```{mermaid}
+:alt: Application class diagram
+:align: center
+
+%%{init: {"theme": "base", "themeVariables": { "fontSize": "12px"}} }%%
+
+classDiagram
+    direction TB
+    class Application {
+    }
+
+    class Graph {
+    }
+
+    class Operator {
+    }
+
+    <<abstract>> Application
+    <<abstract>> Graph
+    <<abstract>> Operator
+
+    Application --> Graph : makes use of
+    Graph "1" --> "many" Operator : contains
+```
+
+[Application](/modules/_autosummary/monai.deploy.core.Application) represents a workflow as a [Graph](/modules/_autosummary/monai.deploy.core.graphs.Graph) and the graph handles [Operator](/modules/_autosummary/monai.deploy.core.Operator)s which are computational tasks.
+
+To develop and deploy your MONAI App, you can follow the steps below (click a node to see the detail):
+
+## 1. Developing Application
+
+```{mermaid}
+:alt: Developing Application
+:align: center
+:caption: ⠀⠀Steps to developing application
+
+%%{init: {"theme": "base", "themeVariables": { "fontSize": "12px"}} }%%
+
+graph TD
+
+    Design(Design Workflow)
+    --> Operator(Creating Operator classes)
+    --> App(Creating Application class)
+    --> ExecApp(Executing app locally)
+
+
+    click Design "./designing_workflow.html" "Go to the document" _self
+    click Operator "./creating_operator_classes.html" "Go to the document" _self
+    click App "./creating_application_class.html" "Go to the document" _self
+    click ExecApp "./executing_app_locally.html" "Go to the document" _self
+```
+
+First, you will need to design the workflow of your application that defines [Operator](/modules/_autosummary/monai.deploy.core.Operator)s (tasks) and flows among them. Once the workflow is designed, you can start implementing operator classes for those that you cannot use existing operators as they are. Then implement an [Application](/modules/_autosummary/monai.deploy.core.Application) class to make a workflow graph with the operators.
+
+You can execute and debug your application locally in a Jupyter notebook or through CLI.
+
+## 2. Packaging, Local-Running, and Deploying Application Package
+
+```{mermaid}
+:alt: Application class diagram
+:align: center
+:caption: ⠀⠀Steps to package, local-running, and deploying MONAI Application Package (MAP)
+
+%%{init: {"theme": "base", "themeVariables": { "fontSize": "12px"}} }%%
+
+graph TD
+
+    Package(Packaging app)
+    --> ExecPackage(Executing packaged app locally)
+    --> DeployPackage(Deploying to the remote server)
+
+    click Package "./packaging_app.html" "Go to the document" _self
+    click ExecPackage "./executing_packaged_app_locally.html" "Go to the document" _self
+    click DeployPackage "./deploying_to_the_remote_server.html" "Go to the document" _self
+```
+
diff --git a/docs/source/developing_with_sdk/designing_workflow.md b/docs/source/developing_with_sdk/designing_workflow.md
@@ -0,0 +1 @@
+# Designing Workflow
diff --git a/docs/source/developing_with_sdk/index.md b/docs/source/developing_with_sdk/index.md
@@ -5,6 +5,7 @@
 :maxdepth: 2
 
 core_concepts
+designing_workflow
 creating_operator_classes
 creating_application_class
 executing_app_locally
diff --git a/docs/source/introduction/architecture.md b/docs/source/introduction/architecture.md
@@ -16,7 +16,7 @@ The MONAI Deploy App SDK enables the designing of usable, reproducible, and prod
 
 Below is a high-level architecture diagram of the MONAI Deploy Application SDK.
 
-![image](https://user-images.githubusercontent.com/1928522/133539726-fd148075-c37e-485f-b2d5-672741488181.png)
+![image](https://user-images.githubusercontent.com/1928522/133837308-034cf542-d83d-4b0b-b668-1999bb7b3e30.png)
 
 In the following sections, the components of the architecture are described in more detail. To highlight the time-dimension of the architecture, the descriptions have been grouped by Development-time & Run-time.
 
@@ -26,18 +26,18 @@ The core development-time concept in the MONAI Deploy App SDK is an inference ap
 
 The user is encouraged to structure the application code in a way that enables reproducibility and debuggability. In contrast, the SDK itself is designed to minimize concerns related to production readiness during development time. Optimally, the user can write idiomatic Python code focusing on the logic itself and the guard rails of the framework enable the code to be production-ready.
 
-**Application**: An application represents a collection of computational tasks that together accomplish a meaningful goal in the healthcare domain. Typically, an app defines a workflow that reads medical imaging data from disk, processes it in one or more operators (some of which could be AI inference related), and produces output data. User implements an app by subclassing [monai.deploy.core.Application](/modules/_autosummary/monai.deploy.core.Application) class. An app makes use of instances of Operators as stages in the application.
+**Application**: An application represents a collection of computational tasks that together accomplish a meaningful goal in the healthcare domain. Typically, an app defines a workflow that reads medical imaging data from disk, processes it in one or more operators (some of which could be AI inference related), and produces output data. User implements an app by subclassing [Application](/modules/_autosummary/monai.deploy.core.Application) class. An app makes use of instances of Operators as stages in the application.
 
-**Graph**: The SDK provides a mechanism to define a directed acyclic graph which can be composed of operators. This acyclic property is important, as it prevents the framework from running into circular dependencies between operators. The graph consists of one or more vertices and edges, with each edge directed from one vertex to another, such that there is no way to start at any vertex and follow a consistently directed sequence of edges that eventually loops back to the same vertex again. Each vertex in the graph represents an Operator. The edge between two operators contains connectivity information.
+**Graph**: The SDK provides a mechanism to define a directed acyclic graph (through [Graph](/modules/_autosummary/monai.deploy.core.graphs.Graph) class) which can be composed of operators. This acyclic property is important, as it prevents the framework from running into circular dependencies between operators. The graph consists of one or more vertices and edges, with each edge directed from one vertex to another, such that there is no way to start at any vertex and follow a consistently directed sequence of edges that eventually loops back to the same vertex again. Each vertex in the graph represents an Operator. The edge between two operators contains connectivity information.
 
-**Operator**: An operator is the smallest unit of computation. It is implemented by the user by inheriting a class from the [monai.deploy.core.Operator](/modules/_autosummary/monai.deploy.core.Operator). An operator is an element of a MONAI Deploy Application. Each operator is typically designed to perform a specific function/analysis on incoming input data. Common examples of such functions are: reading images from disk, performing image processing, performing AI inference, writing images to disk, etc. The SDK comes with a bundled set of operators.
+**Operator**: An operator is the smallest unit of computation. It is implemented by the user by inheriting a class from the [Operator](/modules/_autosummary/monai.deploy.core.Operator). An operator is an element of a MONAI Deploy Application. Each operator is typically designed to perform a specific function/analysis on incoming input data. Common examples of such functions are: reading images from disk, performing image processing, performing AI inference, writing images to disk, etc. The SDK comes with a bundled set of operators.
 
 ### Run-Time Concepts
 
 The core runtime concepts in the MONAI Deploy App SDK are the MONAI Application Package (MAP) and a MONAI Application Runner (MAR). A key design decision of the SDK is to make the framework runtime-agnostic. The same code should be runnable in various environments, such as on a workstation during development or on a production-ready workflow orchestrator during production.
 
-**MONAI Application Packager**: Once an application is built using the MONAI App SDK, it can be packaged into a portable MONAI Application Package (MAP). A MAP contains an executable application & provides sufficient information to execute the application as intended. It consists of a single container and metadata that provides additional information about the application. A MAP is self-describing and provides a mechanism for extracting its description. It provides information about its expected inputs such that an external agent is able to determine if the MAP is capable of receiving a workload. The containerized portion of a MAP complies with Open Container Initiative (OCI) format standards. The MONAI Application Packager utility helps developers to package an app written using the SDK into a MAP.
+**MONAI Application Packager**: Once an application is built using the MONAI App SDK, it can be packaged into a portable MONAI Application Package (MAP). A MAP contains an executable application & provides sufficient information to execute the application as intended. It consists of a single container and metadata that provides additional information about the application. A MAP is self-describing and provides a mechanism for extracting its description. It provides information about its expected inputs such that an external agent is able to determine if the MAP is capable of receiving a workload. The containerized portion of a MAP complies with Open Container Initiative (OCI) format standards. The [MONAI Application Packager](/developing_with_sdk/packaging_app) utility helps developers to package an app written using the SDK into a MAP.
 
 **Executor**: An executor in the SDK is an entity that ingests an Application, parses the Directed Acyclic Graph inside it, and executes the operators in the specified order. The SDK has provisions to support multiple types of Executors depending on single/multi-process and execution order needs.
 
-**MONAI Application Runner**: The MONAI Application Runner (MAR) is a command-line utility that allows users to run and test their MONAI Application Package (MAP) locally. MAR is developed to make the running and testing of MAPs locally an easy process for developers by abstracting away the need to understand the internal details of the MAP. MAR allows users to specify input and output paths on the local file system which it maps to the input and output of MAP during execution.
+**MONAI Application Runner**: The [MONAI Application Runner (MAR)](/developing_with_sdk/executing_packaged_app_locally) is a command-line utility that allows users to run and test their MONAI Application Package (MAP) locally. MAR is developed to make the running and testing of MAPs locally an easy process for developers by abstracting away the need to understand the internal details of the MAP. MAR allows users to specify input and output paths on the local file system which it maps to the input and output of MAP during execution.
diff --git a/docs/source/modules/datastores.md b/docs/source/modules/datastores.md
@@ -0,0 +1,6 @@
+# Graphs
+
+```{eval-rst}
+.. automodule:: monai.deploy.core.datastores
+   :noindex:
+```
diff --git a/docs/source/modules/executors.md b/docs/source/modules/executors.md
@@ -0,0 +1,6 @@
+# Executors
+
+```{eval-rst}
+.. automodule:: monai.deploy.core.executors
+   :noindex:
+```
diff --git a/docs/source/modules/graphs.md b/docs/source/modules/graphs.md
@@ -0,0 +1,6 @@
+# Graphs
+
+```{eval-rst}
+.. automodule:: monai.deploy.core.graphs
+   :noindex:
+```
diff --git a/docs/source/modules/index.md b/docs/source/modules/index.md
@@ -32,3 +32,27 @@ operators
 :hidden:
 models
 ```
+
+## Graphs
+
+```{toctree}
+:maxdepth: 2
+:hidden:
+graphs
+```
+
+## Executors
+
+```{toctree}
+:maxdepth: 2
+:hidden:
+executors
+```
+
+## Datastores
+
+```{toctree}
+:maxdepth: 2
+:hidden:
+datastores
+```
diff --git a/monai/deploy/core/__init__.py b/monai/deploy/core/__init__.py
@@ -18,10 +18,10 @@
     input
     output
     resource
+    IOType
     ExecutionContext
     InputContext
     OutputContext
-    IOType
 """
 
 from .application import Application
diff --git a/monai/deploy/core/datastores/__init__.py b/monai/deploy/core/datastores/__init__.py
@@ -8,7 +8,14 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+"""
+.. autosummary::
+    :toctree: _autosummary
 
+    DatastoreFactory
+    Datastore
+    MemoryDatastore
+"""
 
 from .datastore import Datastore
 from .factory import DatastoreFactory
diff --git a/monai/deploy/core/executors/__init__.py b/monai/deploy/core/executors/__init__.py
@@ -1,9 +1,27 @@
+# Copyright 2021 MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+.. autosummary::
+    :toctree: _autosummary
+
+    ExecutorFactory
+    Executor
+    SingleProcessExecutor
+"""
+
 from .executor import Executor
 from .factory import ExecutorFactory
 
 # from .multi_process_executor import MultiProcessExecutor
 # from .multi_threaded_executor import MultiThreadedExecutor
 from .single_process_executor import SingleProcessExecutor
 
-# __all__ = ["Executor", "MultiProcessExecutor", "MultiThreadedExecutor", "SingleProcessExecutor"]
-__all__ = ["Executor", "SingleProcessExecutor", "ExecutorFactory"]
+# __all__ = ["Executor", "SingleProcessExecutor", "ExecutorFactory"]
diff --git a/monai/deploy/core/executors/executor.py b/monai/deploy/core/executors/executor.py
@@ -10,18 +10,17 @@
 # limitations under the License.
 
 from abc import ABC, abstractmethod
-from typing import TYPE_CHECKING, Dict, Optional
-
-if TYPE_CHECKING:
-    from monai.deploy.core import Application
+from typing import Dict, Optional
 
+# https://github.com/agronholm/sphinx-autodoc-typehints#dealing-with-circular-imports
+from monai.deploy.core import application
 from monai.deploy.core.datastores import Datastore, DatastoreFactory
 
 
 class Executor(ABC):
     """This is the base class that enables execution of an application."""
 
-    def __init__(self, app: "Application", datastore: Optional[Datastore] = None, **kwargs: Dict):
+    def __init__(self, app: "application.Application", datastore: Optional[Datastore] = None, **kwargs: Dict):
         """Constructor of the class.
 
         Given an application it invokes the compose method on the app, which
@@ -38,7 +37,7 @@ def __init__(self, app: "Application", datastore: Optional[Datastore] = None, **
             self._datastore = DatastoreFactory.create(DatastoreFactory.DEFAULT)
 
     @property
-    def app(self) -> "Application":
+    def app(self) -> "application.Application":
         """Returns the application that is executed by the executor."""
         return self._app
 
diff --git a/monai/deploy/core/graphs/__init__.py b/monai/deploy/core/graphs/__init__.py
@@ -1,3 +1,22 @@
+# Copyright 2021 MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+.. autosummary::
+    :toctree: _autosummary
+
+    GraphFactory
+    Graph
+    NetworkXGraph
+"""
+
 from .factory import GraphFactory
 from .graph import Graph
 from .nx_digraph import NetworkXGraph
diff --git a/monai/deploy/runner/utils.py b/monai/deploy/runner/utils.py
@@ -25,7 +25,8 @@ def get_requested_gpus(pkg_info: dict) -> int:
     Returns:
         int: requested number of gpus in the package manifest
     """
-    return pkg_info.get("resources", {}).get("gpu", 0)
+    num_gpu: int = pkg_info.get("resources", {}).get("gpu", 0)
+    return num_gpu
 
 
 def run_cmd(cmd: str) -> int:
diff --git a/pyproject.toml b/pyproject.toml
@@ -25,6 +25,8 @@ exclude = '''
     | buck-out
     | build
     | dist
+    | docs/source/notebooks
+    | notebooks
   )/
   # also separately exclude a file named versioneer.py and _version.py
   | versioneer.py
diff --git a/run b/run
@@ -915,6 +915,8 @@ setup_gen_docs() {
 
     # Remove existing files in dist/docs
     run_command rm -rf ${output_folder}/*
+    # Remove existing _autosummary folder
+    run command rm -rf ${TOP}/docs/source/modules/_autosummary
 
     # Symbolic link notebooks folder from 'docs/source/notebooks' to 'notebooks'
     run_command rm -rf ${TOP}/docs/source/notebooks
diff --git a/setup.cfg b/setup.cfg
@@ -48,13 +48,13 @@ per_file_ignores =
     # Allow using camel case for variable/argument names for the sake of readability.
     # 43 warnings with N803(2) N806(41) in dicom_seg_writer_operator.py
     dicom_seg_writer_operator.py: N803, N806
-exclude = *.pyi,.git,.eggs,monai/deploy/_version.py,versioneer.py,venv,.venv,_version.py
+exclude = *.pyi,.git,.eggs,monai/deploy/_version.py,dist,versioneer.py,venv,.venv,_version.py
 
 [isort]
 known_first_party = monai
 profile = black
 line_length = 120
-skip = .git, .eggs, venv, .venv, versioneer.py, _version.py, conf.py
+skip = .git, .eggs, venv, .venv, versioneer.py, _version.py, conf.py, dist
 skip_glob = *.pyi
 
 [versioneer]
@@ -84,6 +84,8 @@ show_column_numbers = True
 show_error_codes = True
 # Use visually nicer output in error messages: use soft word wrap, show source code snippets, and show error location markers.
 pretty = False
+# Exclude certail files/folders
+exclude = (dist|notebooks)/$
 
 [mypy-versioneer]
 # Ignores all non-fatal errors.
