Skip to content

Commit bef7f13

Browse files
vmoensvmoens
committed
Update
[ghstack-poisoned]
1 parent 5f9f206 commit bef7f13

File tree

1 file changed

+54
-54
lines changed

1 file changed

+54
-54
lines changed

‎docs/source/reference/collectors.rst‎

Lines changed: 54 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -243,13 +243,13 @@ When you create a collector with weight sync schemes, the following initializati
243243
| |
244244
| Configuration: |
245245
| - strategy |
246-
| - transport_type |
246+
| - transport type |
247247
| |
248248
+--------+---------+
249249
|
250250
+------------+-------------+
251251
| |
252-
creates creates
252+
| creates | creates
253253
| |
254254
v v
255255
Main Process Worker Process
@@ -258,26 +258,26 @@ When you create a collector with weight sync schemes, the following initializati
258258
| | | |
259259
| - strategy | | - strategy |
260260
| - transports | | - transport |
261-
| - model_ref | | - model_ref |
261+
| - model ref | | - model ref |
262262
| | | |
263263
| Registers: | | Registers: |
264264
| - model | | - model |
265265
| - workers | | - transport |
266266
+--------------+ +---------------+
267267
| |
268268
| Transport Layer |
269-
| +----------------+ |
270-
+-->+ MPTransport |<------+
271-
| | (pipes) | |
272-
| +----------------+ |
269+
| +----------------+ |
270+
+--> | MPTransport | <----+
271+
| | (pipes) | |
272+
| +----------------+ |
273273
| |
274274
| +--------------------+ |
275-
+-->+ SharedMemTransport |<--+
276-
| | (shared mem) | |
275+
+-> | SharedMemTransport | <-+
276+
| | (shared memory) | |
277277
| +--------------------+ |
278278
| |
279279
| +----------------+ |
280-
+-->+ RayTransport |<------+
280+
+-> | RayTransport | <-----+
281281
| (Ray store) |
282282
+----------------+
283283

@@ -299,29 +299,29 @@ When you call ``collector.update_policy_weights_()``, the weight synchronization
299299

300300
Main Process Worker Process
301301

302-
+-------------------+ +-------------------+
303-
| WeightSender | | WeightReceiver |
304-
| | | |
305-
| 1. Extract | | 4. Poll transport |
306-
| weights from | | for weights |
307-
| model using | | |
308-
| strategy | | |
309-
| | 2. Send via | |
310-
| +-------------+ | Transport | +--------------+ |
311-
| | Strategy | | +------------+ | | Strategy | |
312-
| | extract() | | | | | | apply() | |
313-
| +-------------+ +----+ Transport +-------->+ +--------------+ |
314-
| | | | | | | |
315-
| v | +------------+ | v |
316-
| +-------------+ | | +--------------+ |
317-
| | Model | | | | Model | |
318-
| | (source) | | 3. Ack (optional) | | (dest) | |
319-
| +-------------+ | <-----------------------+ | +--------------+ |
320-
| | | |
321-
+-------------------+ | 5. Apply weights |
322-
| to model using |
323-
| strategy |
324-
+-------------------+
302+
+-------------------+ +-------------------+
303+
| WeightSender | | WeightReceiver |
304+
| | | |
305+
| 1. Extract | | 4. Poll transport |
306+
| weights from | | |
307+
| model using | | for weights |
308+
| strategy | | |
309+
| | 2. Send via | |
310+
| +-------------+ | Transport | +--------------+ |
311+
| | Strategy | | +------------+ | | Strategy | |
312+
| | extract() | | | | | | apply() | |
313+
| +-------------+ +----+ Transport +------------>+ +--------------+ |
314+
| | | | | | | |
315+
| v | +------------+ | v |
316+
| +-------------+ | | +--------------+ |
317+
| | Model | | | | Model (dest) | |
318+
| | (source) | | 3. Acknowledgment (optional) | | | |
319+
| +-------------+ | <---------------------------+ | +--------------+ |
320+
| | | |
321+
+-------------------+ | 5. Apply weights |
322+
| to model using |
323+
| strategy |
324+
+-------------------+
325325

326326
1. **Extract**: Sender extracts weights from the source model (state_dict or TensorDict)
327327
2. **Send**: Sender broadcasts weights through all registered transports
@@ -339,27 +339,27 @@ One of the key features is support for synchronizing multiple models independent
339339
:scale: 130
340340
:textual:
341341

342-
Main Process Worker Process 1 Worker Process 2
343-
344-
+-----------------+ +---------------+ +---------------+
345-
| Collector | | Collector | | Collector |
346-
| | | | | |
347-
| Models: | | Models: | | Models: |
348-
| +----------+ | | +--------+ | | +--------+ |
349-
| | Policy A | | | |Policy A| | | |Policy A| |
350-
| +----------+ | | +--------+ | | +--------+ |
351-
| +----------+ | | +--------+ | | +--------+ |
352-
| | Model B | | | |Model B| | | |Model B| |
353-
| +----------+ | | +--------+ | | +--------+ |
354-
| | | | | |
355-
| Weight Senders: | | Weight | | Weight |
356-
| +----------+ | | Receivers: | | Receivers: |
357-
| | Sender A +---+------------+->Receiver A | | Receiver A |
358-
| +----------+ | | | | |
359-
| +----------+ | | +--------+ | | +--------+ |
360-
| | Sender B +---+------------+->Receiver B | | Receiver B |
361-
| +----------+ | Pipes | | Pipes | |
362-
+-----------------+ +-------+-------+ +-------+-------+
342+
Main Process Worker Processes (1, 2, ..., B)
343+
344+
+-----------------+ +-------------------+
345+
| Orchestrator | | Collector |
346+
| | | |
347+
| Models: | | Models: |
348+
| +----------+ | | +--------+ |
349+
| | Policy A | | | |Policy A| |
350+
| +----------+ | | +--------+ |
351+
| +----------+ | | +--------+ |
352+
| | Model B | | | |Model B| |
353+
| +----------+ | | +--------+ |
354+
| | | |
355+
| Weight Senders: | | Weight Receivers: |
356+
| +----------+ | | |
357+
| | Sender A +---+------------+->Receiver A |
358+
| +----------+ | | |
359+
| +----------+ | | +--------+ |
360+
| | Sender B +---+------------+->Receiver B |
361+
| +----------+ | Pipes | |
362+
+-----------------+ +-------------------+
363363

364364
Each model gets its own sender/receiver pair, allowing independent synchronization frequencies,
365365
different transport mechanisms per model, and model-specific strategies.

0 commit comments

Comments
 (0)