propagate Exception from inference workers to main process

[aniketmaurya]

🐛 Bug

Exceptions are not propagated from inference workers to main process when using OpenAISpec. This results in silent failure.

Code sample

import litserve as ls

class SimpleLitAPI(ls.LitAPI):
    def setup(self, device):
        self.model = None

    def decode_request(self, request):
        content = request.messages[-1].content
        if "BAD WORD" in content:
            raise Exception("Guardrail detected inappropriate content.")
        return [{"role": "user", "content": content}]

    def predict(self, prompt):
        yield "This is a sample generated text"

if __name__ == '__main__':
    api = SimpleLitAPI()
    server = ls.LitServer(api, spec=ls.OpenAISpec())
    server.run(port=8000)

This server will always give HTTP 200 response since the FastAPI StreamingResponse is sent before any actual computation is performed.

Expected behavior

Environment

If you published a Studio with your bug report, we can automatically get this information. Otherwise, please describe:

• PyTorch/Jax/Tensorflow Version (e.g., 1.0):
• OS (e.g., Linux):
• How you installed PyTorch (conda, pip, source):
• Build command you used (if compiling from source):
• Python version:
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information:

Additional context