[Windows] Server Hangs while closing

[FrsECM]

Hello !

🐛 Bug

When we run and close the server on windows (ctrl+C), it's impossible to kill the process/thread.

The only solution is to close the terminal.

To Reproduce

Unfortunately it's not possible to reproduce in studio because the issue is only present on windows.

Code sample

The default server is causing the issue :

import litserve as ls
class SimpleLitAPI(ls.LitAPI):
    def setup(self, device):
        self.model1 = lambda x: x**2
    def decode_request(self, request):
        return request["input"] 
    def predict(self, x):
        squared = self.model1(x)
        output = squared
        return {"output": output}

    def encode_response(self, output):
        return {"output": output} 

# (STEP 2) - START THE SERVER
if __name__ == "__main__":
    # scale with advanced features (batching, GPUs, etc...)
    server = ls.LitServer(SimpleLitAPI(), accelerator="auto", max_batch_size=1)
    server.run(port=8000)

Expected behavior

When we close the server, it should give the hand on the terminal to the user, not hang.

Environment

• OS - Windows 11

$> pip list
---------------------- -----------------
 Package                Version
---------------------- -----------------
annotated-types        0.7.0
anyio                  4.6.2.post1
certifi                2024.8.30
charset-normalizer     3.4.0
click                  8.1.7
colorama               0.4.6
exceptiongroup         1.2.2
fastapi                0.115.5
filelock               3.13.1
fsspec                 2024.2.0
h11                    0.14.0
httpcore               1.0.7
httptools              0.6.4
httpx                  0.27.2
idna                   3.10
imagecodecs            2024.9.22
intel-openmp           2021.4.0
Jinja2                 3.1.3
litserve               0.2.4
MarkupSafe             2.1.5
mkl                    2021.4.0
mpmath                 1.3.0
networkx               3.2.1
numpy                  2.1.3
opencv-python-headless 4.10.0.84
pillow                 11.0.0
pip                    24.3.1
pydantic               2.10.1
pydantic_core          2.27.1
python-dotenv          1.0.1
python-multipart       0.0.17
PyYAML                 6.0.2
requests               2.32.3
setuptools             75.6.0
sniffio                1.3.1
starlette              0.41.3
sympy                  1.13.1
tbb                    2021.11.0
tifffile               2023.4.12
torch                  2.5.1+cu121
torchvision            0.20.1+cu121
tqdm                   4.67.1
typing_extensions      4.12.2
urllib3                2.2.3
uvicorn                0.32.1
watchfiles             1.0.0
websockets             14.1
wheel                  0.45.1

[aniketmaurya]

hi @FrsECM, may I know if the server shutsdown after multiple CTRL + C calls or does it stay alive waiting?

[FrsECM]

hi @FrsECM, may I know if the server shutsdown after multiple CTRL + C calls or does it stay alive waiting?

It seems that the server is no longer responding after the CTRL + C.