Skip to content

Commit 7503955

Browse files
samuel-williams-shopifysamuel-williams-shopify
committed
Add documentation.
1 parent ebbdec8 commit 7503955

File tree

5 files changed

+349
-0
lines changed

5 files changed

+349
-0
lines changed

‎context/getting-started.md‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
# Getting Started
2+
3+
This guide explains how to get started with `async-container-supervisor` to supervise and monitor worker processes in your Ruby applications.
4+
5+
## Installation
6+
7+
Add the gem to your project:
8+
9+
```bash
10+
$ bundle add async-container-supervisor
11+
```
12+
13+
## Core Concepts
14+
15+
`async-container-supervisor` provides a robust process supervision system built on top of {ruby Async::Service::Generic}. The key components are:
16+
17+
- {ruby Async::Container::Supervisor::Environment}: An environment mixin that sets up a supervisor service in your application.
18+
- {ruby Async::Container::Supervisor::Supervised}: An environment mixin that enables workers to connect to and be supervised by the supervisor.
19+
- {ruby Async::Container::Supervisor::Server}: The server that handles communication with workers and performs monitoring.
20+
- {ruby Async::Container::Supervisor::Worker}: A client that connects workers to the supervisor for health monitoring and diagnostics.
21+
22+
### Process Architecture
23+
24+
The supervisor operates as a multi-process architecture with three layers:
25+
26+
```mermaid
27+
graph TD
28+
Controller[Async::Container::Controller<br/>Root Process]
29+
30+
Controller -->|spawns & manages| Supervisor[Supervisor Process<br/>async-container-supervisor]
31+
Controller -->|spawns & manages| Worker1[Worker Process 1]
32+
Controller -->|spawns & manages| Worker2[Worker Process 2]
33+
Controller -->|spawns & manages| WorkerN[Worker Process N...]
34+
35+
Worker1 -.->|connects via IPC| Supervisor
36+
Worker2 -.->|connects via IPC| Supervisor
37+
WorkerN -.->|connects via IPC| Supervisor
38+
39+
style Controller fill:#e1f5ff
40+
style Supervisor fill:#fff4e1
41+
style Worker1 fill:#e8f5e9
42+
style Worker2 fill:#e8f5e9
43+
style WorkerN fill:#e8f5e9
44+
```
45+
46+
**Important:** The supervisor process is itself just another process managed by the root controller. If the supervisor crashes, the controller will restart it, and all worker processes will automatically reconnect to the new supervisor. This design ensures high availability and fault tolerance.
47+
48+
## Usage
49+
50+
To use the supervisor, you need to define two services: one for the supervisor itself and one for your workers that will be supervised.
51+
52+
### Basic Example
53+
54+
Create a service configuration file (e.g., `service.rb`):
55+
56+
```ruby
57+
#!/usr/bin/env async-service
58+
# frozen_string_literal: true
59+
60+
require "async/container/supervisor"
61+
62+
class MyWorkerService < Async::Service::Generic
63+
def setup(container)
64+
super
65+
66+
container.run(name: self.class.name, count: 4, restart: true) do |instance|
67+
Async do
68+
# Connect to the supervisor if available:
69+
if @environment.implements?(Async::Container::Supervisor::Supervised)
70+
@evaluator.make_supervised_worker(instance).run
71+
end
72+
73+
# Mark the worker as ready:
74+
instance.ready!
75+
76+
# Your worker logic here:
77+
loop do
78+
# Do work...
79+
sleep 1
80+
81+
# Periodically update readiness:
82+
instance.ready!
83+
end
84+
end
85+
end
86+
end
87+
end
88+
89+
# Define the worker service:
90+
service "worker" do
91+
service_class MyWorkerService
92+
93+
# Enable supervision for this service:
94+
include Async::Container::Supervisor::Supervised
95+
end
96+
97+
# Define the supervisor service:
98+
service "supervisor" do
99+
include Async::Container::Supervisor::Environment
100+
end
101+
```
102+
103+
### Running the Service
104+
105+
Make the service executable and run it:
106+
107+
```bash
108+
$ chmod +x service.rb
109+
$ ./service.rb
110+
```
111+
112+
This will start:
113+
- A supervisor process listening on a Unix socket
114+
- Four worker processes that connect to the supervisor
115+
116+
### Adding Health Monitors
117+
118+
You can add monitors to detect and respond to unhealthy conditions. For example, to add a memory monitor:
119+
120+
```ruby
121+
service "supervisor" do
122+
include Async::Container::Supervisor::Environment
123+
124+
monitors do
125+
[
126+
# Restart workers that exceed 500MB of memory:
127+
Async::Container::Supervisor::MemoryMonitor.new(
128+
interval: 10, # Check every 10 seconds
129+
limit: 1024 * 1024 * 500 # 500MB limit
130+
)
131+
]
132+
end
133+
end
134+
```
135+
136+
The {ruby Async::Container::Supervisor::MemoryMonitor} will periodically check worker memory usage and restart any workers that exceed the configured limit.
137+
138+
### Collecting Diagnostics
139+
140+
The supervisor can collect various diagnostics from workers on demand:
141+
142+
- **Memory dumps**: Full heap dumps for memory analysis
143+
- **Thread dumps**: Stack traces of all threads
144+
- **Scheduler dumps**: Async fiber hierarchy
145+
- **Garbage collection profiles**: GC performance data
146+
147+
These can be triggered programmatically or via command-line tools (when available).
148+
149+
## Advanced Usage
150+
151+
### Custom Monitors
152+
153+
You can create custom monitors by implementing the monitor interface. A monitor should:
154+
155+
1. Accept connections and periodically check worker health
156+
2. Take action (like restarting workers) when unhealthy conditions are detected
157+
158+
### Fault Tolerance
159+
160+
The supervisor architecture is designed for fault tolerance:
161+
162+
- **Supervisor crashes**: When the supervisor process crashes, the root controller automatically restarts it. Workers detect the disconnection and reconnect to the new supervisor.
163+
- **Worker crashes**: The container automatically restarts crashed workers based on the `restart: true` configuration.
164+
- **Communication failures**: Workers gracefully handle supervisor unavailability and will attempt to reconnect.
165+
166+
This design ensures your application remains operational even when individual processes fail.

‎context/index.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# Automatically generated context index for Utopia::Project guides.
2+
# Do not edit then files in this directory directly, instead edit the guides and then run `bake utopia:project:agent:context:update`.
3+
---
4+
description: A supervisor for managing multiple container processes.
5+
metadata:
6+
documentation_uri: https://socketry.github.io/async-container-supervisor/
7+
source_code_uri: https://github.com/socketry/async-container-supervisor.git
8+
files:
9+
- path: getting-started.md
10+
title: Getting Started
11+
description: This guide explains how to get started with `async-container-supervisor`
12+
to supervise and monitor worker processes in your Ruby applications.

‎guides/getting-started/readme.md‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
# Getting Started
2+
3+
This guide explains how to get started with `async-container-supervisor` to supervise and monitor worker processes in your Ruby applications.
4+
5+
## Installation
6+
7+
Add the gem to your project:
8+
9+
```bash
10+
$ bundle add async-container-supervisor
11+
```
12+
13+
## Core Concepts
14+
15+
`async-container-supervisor` provides a robust process supervision system built on top of {ruby Async::Service::Generic}. The key components are:
16+
17+
- {ruby Async::Container::Supervisor::Environment}: An environment mixin that sets up a supervisor service in your application.
18+
- {ruby Async::Container::Supervisor::Supervised}: An environment mixin that enables workers to connect to and be supervised by the supervisor.
19+
- {ruby Async::Container::Supervisor::Server}: The server that handles communication with workers and performs monitoring.
20+
- {ruby Async::Container::Supervisor::Worker}: A client that connects workers to the supervisor for health monitoring and diagnostics.
21+
22+
### Process Architecture
23+
24+
The supervisor operates as a multi-process architecture with three layers:
25+
26+
```mermaid
27+
graph TD
28+
Controller[Async::Container::Controller<br/>Root Process]
29+
30+
Controller -->|spawns & manages| Supervisor[Supervisor Process<br/>async-container-supervisor]
31+
Controller -->|spawns & manages| Worker1[Worker Process 1]
32+
Controller -->|spawns & manages| Worker2[Worker Process 2]
33+
Controller -->|spawns & manages| WorkerN[Worker Process N...]
34+
35+
Worker1 -.->|connects via IPC| Supervisor
36+
Worker2 -.->|connects via IPC| Supervisor
37+
WorkerN -.->|connects via IPC| Supervisor
38+
39+
style Controller fill:#e1f5ff
40+
style Supervisor fill:#fff4e1
41+
style Worker1 fill:#e8f5e9
42+
style Worker2 fill:#e8f5e9
43+
style WorkerN fill:#e8f5e9
44+
```
45+
46+
**Important:** The supervisor process is itself just another process managed by the root controller. If the supervisor crashes, the controller will restart it, and all worker processes will automatically reconnect to the new supervisor. This design ensures high availability and fault tolerance.
47+
48+
## Usage
49+
50+
To use the supervisor, you need to define two services: one for the supervisor itself and one for your workers that will be supervised.
51+
52+
### Basic Example
53+
54+
Create a service configuration file (e.g., `service.rb`):
55+
56+
```ruby
57+
#!/usr/bin/env async-service
58+
# frozen_string_literal: true
59+
60+
require "async/container/supervisor"
61+
62+
class MyWorkerService < Async::Service::Generic
63+
def setup(container)
64+
super
65+
66+
container.run(name: self.class.name, count: 4, restart: true) do |instance|
67+
Async do
68+
# Connect to the supervisor if available:
69+
if @environment.implements?(Async::Container::Supervisor::Supervised)
70+
@evaluator.make_supervised_worker(instance).run
71+
end
72+
73+
# Mark the worker as ready:
74+
instance.ready!
75+
76+
# Your worker logic here:
77+
loop do
78+
# Do work...
79+
sleep 1
80+
81+
# Periodically update readiness:
82+
instance.ready!
83+
end
84+
end
85+
end
86+
end
87+
end
88+
89+
# Define the worker service:
90+
service "worker" do
91+
service_class MyWorkerService
92+
93+
# Enable supervision for this service:
94+
include Async::Container::Supervisor::Supervised
95+
end
96+
97+
# Define the supervisor service:
98+
service "supervisor" do
99+
include Async::Container::Supervisor::Environment
100+
end
101+
```
102+
103+
### Running the Service
104+
105+
Make the service executable and run it:
106+
107+
```bash
108+
$ chmod +x service.rb
109+
$ ./service.rb
110+
```
111+
112+
This will start:
113+
- A supervisor process listening on a Unix socket
114+
- Four worker processes that connect to the supervisor
115+
116+
### Adding Health Monitors
117+
118+
You can add monitors to detect and respond to unhealthy conditions. For example, to add a memory monitor:
119+
120+
```ruby
121+
service "supervisor" do
122+
include Async::Container::Supervisor::Environment
123+
124+
monitors do
125+
[
126+
# Restart workers that exceed 500MB of memory:
127+
Async::Container::Supervisor::MemoryMonitor.new(
128+
interval: 10, # Check every 10 seconds
129+
limit: 1024 * 1024 * 500 # 500MB limit
130+
)
131+
]
132+
end
133+
end
134+
```
135+
136+
The {ruby Async::Container::Supervisor::MemoryMonitor} will periodically check worker memory usage and restart any workers that exceed the configured limit.
137+
138+
### Collecting Diagnostics
139+
140+
The supervisor can collect various diagnostics from workers on demand:
141+
142+
- **Memory dumps**: Full heap dumps for memory analysis
143+
- **Thread dumps**: Stack traces of all threads
144+
- **Scheduler dumps**: Async fiber hierarchy
145+
- **Garbage collection profiles**: GC performance data
146+
147+
These can be triggered programmatically or via command-line tools (when available).
148+
149+
## Advanced Usage
150+
151+
### Custom Monitors
152+
153+
You can create custom monitors by implementing the monitor interface. A monitor should:
154+
155+
1. Accept connections and periodically check worker health
156+
2. Take action (like restarting workers) when unhealthy conditions are detected
157+
158+
### Fault Tolerance
159+
160+
The supervisor architecture is designed for fault tolerance:
161+
162+
- **Supervisor crashes**: When the supervisor process crashes, the root controller automatically restarts it. Workers detect the disconnection and reconnect to the new supervisor.
163+
- **Worker crashes**: The container automatically restarts crashed workers based on the `restart: true` configuration.
164+
- **Communication failures**: Workers gracefully handle supervisor unavailability and will attempt to reconnect.
165+
166+
This design ensures your application remains operational even when individual processes fail.

‎guides/links.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
getting-started:
2+
order: 1
3+

‎readme.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ Provides a supervisor service for
1616

1717
Please see the [project documentation](https://socketry.github.io/async-container-supervisor/) for more details.
1818

19+
- [Getting Started](https://socketry.github.io/async-container-supervisor/guides/getting-started/index) - This guide explains how to get started with `async-container-supervisor` to supervise and monitor worker processes in your Ruby applications.
20+
1921
## Releases
2022

2123
Please see the [project releases](https://socketry.github.io/async-container-supervisor/releases/index) for all releases.

0 commit comments

Comments
 (0)