You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Dynamo follows a class-based architecture similar to BentoML making it intuitive for users familiar with those frameworks. Each service is defined as a Python class, with the following components:
105
105
1. Class attributes for dependencies using `depends()`
106
106
2. An `__init__` method for standard initialization
107
-
3. Optional lifecycle hooks like `@async_on_start` and `@async_on_shutdown`
107
+
3. Optional lifecycle hooks like `@async_on_start` and `@on_shutdown`
108
108
4. Endpoints defined with `@endpoint()`. Optionally, an endpoint can be given a name
109
109
via `@endpoint("my_endpoint_name")`, but otherwise defaults to the name of the
110
110
function being decorated if omitted.
@@ -170,15 +170,14 @@ This is especially useful for:
170
170
- Initializing external connections
171
171
- Setting up runtime resources that require async operations
172
172
173
-
#### `@async_on_shutdown`
174
-
The `@async_on_shutdown` hook is called when the service is shutdown handles cleanup.
173
+
#### `@on_shutdown`
174
+
The `@on_shutdown` hook is called when the service is shutdown handles cleanup.
This ensures resources are properly released, preventing memory leaks and making sure external connections are properly closed. This is helpful to clean up vLLM engines that have been started outside of the main process.
@@ -471,4 +470,4 @@ Think of all the depends statements as the maximal set of edges for the processo
471
470
Processor.link(Router)
472
471
```
473
472
474
-
This removes the `worker` dependency from the Processor and only spin up the Router.
473
+
This removes the `worker` dependency from the Processor and only spin up the Router.
0 commit comments