Start and restart arbitrary processes

guide/content/en/guide/running/manager.md

@@ -239,6 +239,18 @@ The application instance has access to an object that provides access to interac
     }
     ```
 
+The possible states are:
+
+- `NONE` - The worker has been created, but there is no process yet
+- `IDLE` - The process has been created, but is not running yet
+- `STARTING` - The process is starting
+- `STARTED` - The process has started
+- `ACKED` - The process has started and sent an acknowledgement (usually only for server processes)
+- `JOINED` - The process has exited and joined the main process
+- `TERMINATED` - The process has exited and terminated
+- `RESTARTING` - The process is restarting
+- `FAILED` - The process encountered an exception and is no longer running
+- `COMPLETED` - The process has completed its work and exited successfully
 
 ## Built-in non-server processes
 
@@ -324,6 +336,122 @@ To run a managed custom process on Sanic, you must create a callable. If that pr
         app.manager.manage("MyProcess", my_process, {"foo": "bar"})
     ```
 
+### Transient v. durable processes
+
+.. column::
+
+    When you manage a process with the `manage` method, you have the option to make it transient or durable. A transient process will be restarted by the auto-reloader, and a durable process will not.
+
+    By default, all processes are durable.
+
+.. column::
+
+    ```python
+    @app.main_process_ready
+    async def ready(app: Sanic, _):
+        app.manager.manage(
+            "MyProcess",
+            my_process,
+            {"foo": "bar"},
+            transient=True,
+        )
+    ```
+
+
+### Tracked v. untracked processes
+
+.. new:: v23.12
+
+    Out of the box, Sanic will track the state of all processes. This means that you can access the state of the process from the [multiplexer](./manager#access-to-the-multiplexer) object, or from the [Inspector](./manager#inspector).
+
+    See [worker state](./manager#worker-state) for more information.
+
+    Sometimes it is helpful to run background processes that are not long-running. You run them once until completion and then they exit. Upon completion, they will either be in `FAILED` or `COMPLETED` state.
+
+
+.. column::
+
+    When you are running a non-long-running process, you can opt out of tracking it by setting `tracked=False` in the `manage` method. This means that upon completion of the process it will be removed from the list of tracked processes. You will only be able to check the state of the process while it is running.
+
+.. column::
+
+    ```python
+    @app.main_process_ready
+    async def ready(app: Sanic, _):
+        app.manager.manage(
+            "OneAndDone",
+            do_once,
+            {},
+            tracked=False,
+        )
+    ```
+
+*Added in v23.12*
+
+### Restartable custom processes
+
+.. new:: v23.12
+
+    A custom process that is transient will **always** be restartable. That means the auto-restart will work as expected. However, what if you want to be able to *manually* restart a process, but not have it be restarted by the auto-reloader?
+
+.. column::
+
+    In this scenario, you can set `restartable=True` in the `manage` method. This will allow you to manually restart the process, but it will not be restarted by the auto-reloader.
+
+.. column::
+
+    ```python
+    @app.main_process_ready
+    async def ready(app: Sanic, _):
+        app.manager.manage(
+            "MyProcess",
+            my_process,
+            {"foo": "bar"},
+            restartable=True,
+        )
+    ```
+
+.. column::
+
+    You could now manually restart that process from the multiplexer.
+
+.. column::
+
+    ```python
+    @app.get("/restart")
+    async def restart_handler(request: Request):
+        request.app.m.restart("Sanic-MyProcess-0")
+        return json({"foo": request.app.m.name})
+    ```
+
+*Added in v23.12*
+
+### On the fly process management
+
+.. new:: v23.12
+
+    Custom processes are usually added in the `main_process_ready` listener. However, there may be times when you want to add a process after the application has started. For example, you may want to add a process from a request handler. The multiplexer provides a method for doing this.
+
+.. column::
+
+    Once you have a reference to the multiplexer, you can call `manage` to add a process. It works the same as the `manage` method on the Manager.
+
+.. column::
+
+    ```python
+    @app.post("/start")
+    async def start_handler(request: Request):
+        request.app.m.manage(
+            "MyProcess",
+            my_process,
+            {"foo": "bar"},
+            workers=2,
+        )
+        return json({"foo": request.app.m.name})
+    ```
+
+*Added in v23.12*
+
 ## Single process mode
 
 .. column::

sanic/app.py

@@ -2510,7 +2510,10 @@ def inspector(self) -> Inspector:
         """
         if environ.get("SANIC_WORKER_PROCESS") or not self._inspector:
             raise SanicException(
-                "Can only access the inspector from the main process"
+                "Can only access the inspector from the main process "
+                "after main_process_start has run. For example, you most "
+                "likely want to use it inside the @app.main_process_ready "
+                "event listener."
             )
         return self._inspector
 
@@ -2545,6 +2548,9 @@ def manager(self) -> WorkerManager:
 
         if environ.get("SANIC_WORKER_PROCESS") or not self._manager:
             raise SanicException(
-                "Can only access the manager from the main process"
+                "Can only access the manager from the main process "
+                "after main_process_start has run. For example, you most "
+                "likely want to use it inside the @app.main_process_ready "
+                "event listener."
             )
         return self._manager

sanic/mixins/startup.py

@@ -1166,7 +1166,9 @@ def serve(
                     ...
                 sock.close()
             socks = []
+
             trigger_events(main_stop, loop, primary)
+
             loop.close()
             cls._cleanup_env_vars()
             cls._cleanup_apps()
@@ -1192,7 +1194,12 @@ def serve(
     @staticmethod
     def _get_process_states(worker_state) -> List[str]:
         return [
-            state for s in worker_state.values() if (state := s.get("state"))
+            state
+            for s in worker_state.values()
+            if (
+                (state := s.get("state"))
+                and state not in ("TERMINATED", "FAILED", "COMPLETED", "NONE")
+            )
         ]
 
     @classmethod

sanic/worker/constants.py

@@ -13,10 +13,13 @@ class RestartOrder(UpperStrEnum):
 class ProcessState(IntEnum):
     """Process states."""
 
+    NONE = auto()
     IDLE = auto()
     RESTARTING = auto()
     STARTING = auto()
     STARTED = auto()
     ACKED = auto()
     JOINED = auto()
     TERMINATED = auto()
+    FAILED = auto()
+    COMPLETED = auto()