Skip to content

Commit 2eed89c

Browse files
committed
doc: update doc
1 parent 7493c2c commit 2eed89c

File tree

1 file changed

+12
-13
lines changed

1 file changed

+12
-13
lines changed

docs/API/sdk.md

Lines changed: 12 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -83,12 +83,12 @@ class ServiceA:
8383
self.engine = await initialize_model_engine(self.model_name)
8484
print(f"ServiceA initialized with model: {self.model_name}")
8585

86-
@async_on_shutdown
87-
async def async_shutdown(self):
86+
@on_shutdown
87+
def shutdown(self):
8888
# Clean up resources
8989
if self.engine:
90-
await self.engine.shutdown()
91-
print("ServiceA engine shut down")
90+
self.engine.shutdown()
91+
print("ServiceA engine shut down")
9292

9393
@endpoint()
9494
async def generate(self, request: ChatCompletionRequest):
@@ -104,7 +104,7 @@ class ServiceA:
104104
Dynamo follows a class-based architecture similar to BentoML making it intuitive for users familiar with those frameworks. Each service is defined as a Python class, with the following components:
105105
1. Class attributes for dependencies using `depends()`
106106
2. An `__init__` method for standard initialization
107-
3. Optional lifecycle hooks like `@async_on_start` and `@async_on_shutdown`
107+
3. Optional lifecycle hooks like `@async_on_start` and `@on_shutdown`
108108
4. Endpoints defined with `@endpoint()`. Optionally, an endpoint can be given a name
109109
via `@endpoint("my_endpoint_name")`, but otherwise defaults to the name of the
110110
function being decorated if omitted.
@@ -170,15 +170,14 @@ This is especially useful for:
170170
- Initializing external connections
171171
- Setting up runtime resources that require async operations
172172

173-
#### `@async_on_shutdown`
174-
The `@async_on_shutdown` hook is called when the service is shutdown handles cleanup.
173+
#### `@on_shutdown`
174+
The `@on_shutdown` hook is called when the service is shutdown handles cleanup.
175175

176176
```python
177-
@async_on_shutdown
178-
async def async_shutdown(self):
179-
if self._engine_context is not None:
180-
await self._engine_context.__aexit__(None, None, None)
181-
print("VllmWorkerRouterLess shutting down")
177+
@on_shutdown
178+
def shutdown(self):
179+
# gracefully Handle shutdown / cleanup
180+
logger.info("worker shutting down")
182181
```
183182

184183
This ensures resources are properly released, preventing memory leaks and making sure external connections are properly closed. This is helpful to clean up vLLM engines that have been started outside of the main process.
@@ -471,4 +470,4 @@ Think of all the depends statements as the maximal set of edges for the processo
471470
Processor.link(Router)
472471
```
473472

474-
This removes the `worker` dependency from the Processor and only spin up the Router.
473+
This removes the `worker` dependency from the Processor and only spin up the Router.

0 commit comments

Comments
 (0)