Better exception on multiprocessing context conflicts

guide/content/en/guide/running/manager.md

@@ -345,3 +345,102 @@ To run a managed custom process on Sanic, you must create a callable. If that pr
         Sanic.serve_single()
     ```
 
+## Sanic and multiprocessing
+
+Sanic makes heavy use of the [`multiprocessing` module](https://docs.python.org/3/library/multiprocessing.html) to manage the worker processes. You should generally avoid lower level usage of this module (like setting the start method) as it may interfere with the functionality of Sanic.
+
+### Start methods in Python
+
+Before explaining what Sanic tries to do, it is important to understand what the `start_method` is and why it is important. Python generally allows for three different methods of starting a process:
+
+- `fork`
+- `spawn`
+- `forkserver`
+
+The `fork` and `forkserver` methods are only available on Unix systems, and `spawn` is the only method available on Windows. On Unix systems where you have a choice, `fork` is generally the default system method.
+
+You are encouraged to read the [Python documentation](https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods) to learn more about the differences between these methods. However, the important thing to know is that `fork` basically copies the entire memory of the parent process into the child process, whereas `spawn` will create a new process and then load the application into that process. This is the reason why you need to nest your Sanic `run` call inside of the `__name__ == "__main__"` block if you are not using the CLI.
+
+### Sanic and start methods
+
+By default, Sanic will try and use `spawn` as the start method. This is because it is the only method available on Windows, and it is the safest method on Unix systems. 
+
+.. column::
+
+    However, if you are running Sanic on a Unix system and you would like to use `fork` instead, you can do so by setting the `start_method` on the `Sanic` class. You will want to do this as early as possible in your application, and ideally in the global scope before you import any other modules.
+
+.. column::
+
+    ```python
+    from sanic import Sanic
+
+    Sanic.start_method = "fork"
+    ```
+
+### Overcoming a `RuntimeError`
+
+You might have received a `RuntimeError` that looks like this:
+
+```
+RuntimeError: Start method 'spawn' was requested, but 'fork' was already set.
+```
+
+If so, that means somewhere in your application you are trying to set the start method that conflicts with what Sanic is trying to do. You have a few options to resolve this:
+
+.. column::
+
+    **OPTION 1:** You can tell Sanic that the start method has been set and to not try and set it again.
+
+.. column::
+
+    ```python
+    from sanic import Sanic
+
+    Sanic.START_METHOD_SET = True
+    ```
+
+.. column::
+
+    **OPTION 2:** You could tell Sanic that you intend to use `fork` and to not try and set it to `spawn`.
+
+.. column::
+
+    ```python
+    from sanic import Sanic
+
+    Sanic.start_method = "fork"
+    ```
+
+.. column::
+
+    **OPTION 3:** You can tell Python to use `spawn` instead of `fork` by setting the `multiprocessing` start method.
+
+.. column::
+
+    ```python
+    import multiprocessing
+
+    multiprocessing.set_start_method("spawn")
+    ```
+
+In any of these options, you should run this code as early as possible in your application. Depending upon exactly what your specific scenario is, you may need to combine some of the options.
+
+.. note::
+
+    The potential issues that arise from this problem are usually easily solved by just allowing Sanic to be in charge of multiprocessing. This usually means making use of the `main_process_start` and `main_process_ready` listeners to deal with multiprocessing issues. For example, you should move instantiating multiprocessing primitives that do a lot of work under the hood from the global scope and into a listener.
+
+    ```python
+    # This is BAD; avoid the global scope
+    from multiprocessing import Queue
+
+    q = Queue()
+    ```
+
+    ```python
+    # This is GOOD; the queue is made in a listener and shared to all the processes on the shared_ctx
+    from multiprocessing import Queue
+
+    @app.main_process_start
+    async def main_process_start(app):
+        app.shared_ctx.q = Queue()
+    ```

guide/webapp/endpoint/search.py

@@ -18,7 +18,7 @@ async def _search(request: Request, language: str, searcher: Searcher):
     return html(str(builder))
 
 
-@bp.after_server_start
+@bp.before_server_start
 async def setup_search(app: Sanic):
     stemmer = Stemmer()
     pages: list[Page] = app.ctx.pages

guide/webapp/worker/factory.py

@@ -37,21 +37,17 @@ def create_app(root: Path) -> Sanic:
     app.config.STYLE_DIR = root / "style"
     app.config.NODE_MODULES_DIR = root / "node_modules"
     app.config.LANGUAGES = ["en"]
-    app.config.SIDEBAR = load_menu(
-        app.config.CONFIG_DIR / "en" / "sidebar.yaml"
-    )
+    app.config.SIDEBAR = load_menu(app.config.CONFIG_DIR / "en" / "sidebar.yaml")
     app.config.NAVBAR = load_menu(app.config.CONFIG_DIR / "en" / "navbar.yaml")
-    app.config.GENERAL = load_config(
-        app.config.CONFIG_DIR / "en" / "general.yaml"
-    )
+    app.config.GENERAL = load_config(app.config.CONFIG_DIR / "en" / "general.yaml")
 
     setup_livereload(app)
     setup_style(app)
     app.blueprint(bp)
 
     app.static("/assets/", app.config.PUBLIC_DIR / "assets")
 
-    @app.before_server_start
+    @app.before_server_start(priority=1)
     async def setup(app: Sanic):
         app.ext.dependency(PageRenderer(base_title="Sanic User Guide"))
         page_order = _compile_sidebar_order(app.config.SIDEBAR)
@@ -73,9 +69,7 @@ async def page(
     ):
         # TODO: Add more language support
         if language != "api" and language not in app.config.LANGUAGES:
-            return redirect(
-                request.app.url_for("page", language="en", path=path)
-            )
+            return redirect(request.app.url_for("page", language="en", path=path))
         if path in KNOWN_REDIRECTS:
             return redirect(
                 request.app.url_for(
@@ -95,8 +89,6 @@ async def page(
 
     @app.on_request
     async def set_language(request: Request):
-        request.ctx.language = request.match_info.get(
-            "language", Page.DEFAULT_LANGUAGE
-        )
+        request.ctx.language = request.match_info.get("language", Page.DEFAULT_LANGUAGE)
 
     return app

sanic/mixins/startup.py

@@ -818,9 +818,7 @@
             module_name = package_name.replace("-", "_")
             try:
                 module = import_module(module_name)
-                packages.append(
-                    f"{package_name}=={module.__version__}"  # type: ignore
-                )
+                packages.append(f"{package_name}=={module.__version__}")  # type: ignore
             except ImportError:  # no cov
                 ...
 
@@ -927,7 +925,19 @@
             return
 
         method = cls._get_startup_method()
-        set_start_method(method, force=cls.test_mode)
+        try:
+            set_start_method(method, force=cls.test_mode)
+        except RuntimeError:
+            ctx = get_context()
+            actual = ctx.get_start_method()
+            if actual != method:
+                raise RuntimeError(
+                    f"Start method '{method}' was requested, but '{actual}' "
+                    "was already set.\nFor more information, see: "
+                    "https://sanic.dev/en/guide/running/manager.html#overcoming-a-coderuntimeerrorcode"
+                ) from None
+            else:
+                raise
         cls.START_METHOD_SET = True
 
     @classmethod
@@ -938,8 +948,9 @@
         if method != actual:
             raise RuntimeError(
                 f"Start method '{method}' was requested, but '{actual}' "
-                "was actually set."
-            )
+                "was already set.\nFor more information, see: "
+                "https://sanic.dev/en/guide/running/manager.html#overcoming-a-coderuntimeerrorcode"
+            ) from None
         return get_context()
 
     @classmethod

tests/worker/test_startup.py

@@ -1,9 +1,11 @@
 from unittest.mock import patch
 
-import pytest
+import pytest, sys
 
 from sanic import Sanic
 
+from multiprocessing import Queue, set_start_method
+
 
 @pytest.mark.parametrize(
     "start_method,platform,expected",
@@ -23,3 +25,17 @@ def test_get_context(start_method, platform, expected):
         Sanic.start_method = start_method
     with patch("sys.platform", platform):
         assert Sanic._get_startup_method() == expected
+
+
+@pytest.mark.skipif(not sys.platform.startswith("linux"), reason="Only test on Linux")
+def test_set_startup_catch():
+    Sanic.START_METHOD_SET = False
+    set_start_method("fork", force=True)
+    Sanic.test_mode = False
+    message = (
+        "Start method 'spawn' was requested, but 'fork' was already set.\n"
+        "For more information, see: https://sanic.dev/en/guide/running/manager.html#overcoming-a-coderuntimeerrorcode"
+    )
+    with pytest.raises(RuntimeError, match=message):
+        Sanic._set_startup_method()
+    Sanic.test_mode = True