Skip to content

Commit 303e781

Browse files
committed
fix: properly handle async service cleanup in LocalBackend.close()
The close() method was calling service.close() without checking if it was async, causing resource leaks when services had async cleanup methods. This fix: - Makes LocalBackend.close() async to match the base Backend class - Properly awaits async service close methods - Maintains backward compatibility for sync close methods - Handles __exit__ context manager compatibility Prevents GPU memory leaks and zombie processes in production deployments.
1 parent 538a662 commit 303e781

File tree

1 file changed

+14
-4
lines changed

1 file changed

+14
-4
lines changed

src/art/local/backend.py

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
import asyncio
12
import json
23
import math
34
from datetime import datetime
@@ -82,16 +83,25 @@ def __enter__(self):
8283
return self
8384

8485
def __exit__(self, *excinfo):
85-
self.close()
86-
87-
def close(self):
86+
try:
87+
asyncio.get_running_loop()
88+
# If we're already in an async context, we can't await here
89+
# The caller will need to handle cleanup manually
90+
except RuntimeError:
91+
# No event loop running, we can create one
92+
asyncio.run(self.close())
93+
94+
async def close(self):
8895
"""
8996
If running vLLM in a separate process, this will kill that process and close the communication threads.
9097
"""
9198
for _, service in self._services.items():
9299
close_method = getattr(service, "close", None)
93100
if callable(close_method):
94-
close_method()
101+
if asyncio.iscoroutinefunction(close_method):
102+
await close_method()
103+
else:
104+
close_method()
95105

96106
async def register(
97107
self,

0 commit comments

Comments
 (0)