[Windows] OSError when multiple workers

[FrsECM]

🐛 Bug

When we create a server with multiple worker on windows, there is an OSError.

To Reproduce

On windows, run the following server :

import litserve as ls
class SimpleLitAPI(ls.LitAPI):
    def setup(self, device):
        self.model1 = lambda x: x**2
    def decode_request(self, request):
        return request["input"] 
    def predict(self, x):
        squared = self.model1(x)
        output = squared
        return {"output": output}

    def encode_response(self, output):
        return {"output": output} 

# (STEP 2) - START THE SERVER
if __name__ == "__main__":
    # scale with advanced features (batching, GPUs, etc...)
    server = ls.LitServer(SimpleLitAPI(), accelerator="auto", max_batch_size=1,workers_per_device=2)
    server.run(port=8000)

Attach a Lightning Studio which is fully reproducible (code, dependencies, environment, etc...) to reproduce this: NA - I cannot reproduce this because it's specific to windows environment.

Previous code leads to this error :

Swagger UI is available at http://0.0.0.0:8000/docs
INFO:     Started server process [3324]
INFO:     Started server process [3324]
INFO:     Waiting for application startup.
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Application startup complete.
Accept failed on a socket
socket: <asyncio.TransportSocket fd=964, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('0.0.0.0', 8000)>  
Traceback (most recent call last):
  File "c:\Users\F296849\AppData\Local\miniforge3\envs\iris2_win\lib\asyncio\proactor_events.py", line 841, in loop
    f = self._proactor.accept(sock)
  File "c:\Users\F296849\AppData\Local\miniforge3\envs\iris2_win\lib\asyncio\windows_events.py", line 563, in accept
    self._register_with_iocp(listener)
  File "c:\Users\F296849\AppData\Local\miniforge3\envs\iris2_win\lib\asyncio\windows_events.py", line 732, in _register_with_iocp      
    _overlapped.CreateIoCompletionPort(obj.fileno(), self._iocp, 0, 0)
OSError: [WinError 87] Paramètre incorrect
Task exception was never retrieved
future: <Task finished name='Task-7' coro=<IocpProactor.accept.<locals>.accept_coro() done, defined at c:\Users\F296849\AppData\Local\miniforge3\envs\iris2_win\lib\asyncio\windows_events.py:577> exception=OSError(22, 'L’opération d’entrée/sortie a été abandonnée en raison de l’arrêt d’un thread ou à la demande d’une application', None, 995, None)>
Traceback (most recent call last):
  File "c:\Users\F296849\AppData\Local\miniforge3\envs\iris2_win\lib\asyncio\windows_events.py", line 580, in accept_coro
    await future
OSError: [WinError 995] L’opération d’entrée/sortie a été abandonnée en raison de l’arrêt d’un thread ou à la demande d’une application

Expected behavior

I expect the server to run correctly on windows.

Environment

• OS (e.g., Linux): Windows
• How you installed PyTorch (conda, pip, source): conda

(iris2_win) PS C:\BUSDATA\General\IRIS2\IRIS2_STD\Models> pip list
Package                Version
---------------------- -----------------
annotated-types        0.7.0
anyio                  4.6.2.post1
certifi                2024.8.30
charset-normalizer     3.4.0
click                  8.1.7
colorama               0.4.6
domf_iris2_inference   0.0.2.post0+dirty
exceptiongroup         1.2.2
fastapi                0.115.5
filelock               3.13.1
fsspec                 2024.2.0
h11                    0.14.0
httpcore               1.0.7
httptools              0.6.4
httpx                  0.27.2
idna                   3.10
imagecodecs            2024.9.22
intel-openmp           2021.4.0
Jinja2                 3.1.3
litserve               0.2.4
MarkupSafe             2.1.5
mkl                    2021.4.0
mpmath                 1.3.0
networkx               3.2.1
numpy                  2.1.3
opencv-python-headless 4.10.0.84
pillow                 11.0.0
pip                    24.3.1
pydantic               2.10.1
pydantic_core          2.27.1
python-dotenv          1.0.1
python-multipart       0.0.17
PyYAML                 6.0.2
requests               2.32.3
setuptools             75.6.0
sniffio                1.3.1
starlette              0.41.3
sympy                  1.13.1
tbb                    2021.11.0
tifffile               2023.4.12
torch                  2.5.1+cu121
torchvision            0.20.1+cu121
tqdm                   4.67.1
typing_extensions      4.12.2
urllib3                2.2.3
uvicorn                0.32.1
watchfiles             1.0.0
websockets             14.1
wheel                  0.45.1