🔷 [Epic] Indexer Lifecycle Management

[morgsmccauley]

Description

Currently, Indexers will run indefinitely. There is no way for developers to control or interrupt this process. Similarly, the system itself cannot intervene when needed, meaning we cannot stop indexers which are perpetually failing.

The current architecture isn't ideal for managing these lifecycle events. Runner needs to poll for configuration updates creating delays as well as additional latency during executions.

The goal of this epic is to refactor the current architecture to create a dedicated control layer, paving a way for lifecycle management of indexers.

Indexer Lifecycle

Preview Give feedback

1. TBC Failure modes #205
   +0
   
2. Stop historical processing on indexer redeployment #200
   +0
   
3. Add exponential retries for intermittent errors #467
   component: Runner +1
4. Indexer Status frozen as RUNNING if unexpectedly crashes #533
   bug +1
   
5. Indexers occasionally crash due to timeout on Hasura credentials fetch when creating DmlHandler #532
   bug +1
   

Control/Data Plane Separation

Preview Give feedback

1. Spike Control/Data plane separation and create fine grained issues #399
   +0
   
2. Created dedicated block stream per Indexer #418
   1 of 1
   component: Streamer +1
   
3. Expose endpoint to control streams #420
   component: Streamer +1
   
4. Expose endpoint to control executors #416
   component: Runner +1
   
5. Create Control service #421
   component: Coordinator +1
   
6. Add created/updated at timestamps #415
   component: Registry +1
   
7. Enable Toggle between Endpoint and Redis Control of Executors #478
   component: Runner +1
   
8. Selectively start indexers #501
   component: Coordinator +1
   
9. Setup infra for Coordinator V2 #455
   component: Coordinator +1
   
10. Create Rust GRPC client for Runner #468
    component: Runner +1
    
11. Setup infra for Block Streamer #443
    component: Streamer +1
    
12. Control Plane rollout plan #422
    +0
    
13. Enable dedicated streams to start from interruption #417
    component: Coordinator +1
    
14. Design Doc: Separation of Control and Data planes #397
    +0
    
15. Migrate Indexers to Control Plane #520
    component: Coordinator +1
    
16. Temporarilty disable historical backfill during migration #512
    component: Frontend +1
17. Setup network between Coordinator and Streamer/Runner #427
    component: Coordinator +1
    
18. Handle new StartBlock config options #521
    component: Coordinator +1
    
19. Expand start block types to include "Continue" #536
    component: Registry +1
    
20. Allow developers to configure StartBlock::Continue via UI #535
    2 of 2
    component: Frontend +1
    
21. Add Instrumentation to Runner #530
    component: Runner +1
    
22. Coordinator not respecting allowlist #543
    bug component: Coordinator +2
    
23. Create production infrastructure #560
    +0
    
24. Remove allowlist #568
    component: Coordinator +1