op-supervisor: local-safe reorg support

[protolambda]

Description

Start of reorg support. This handles the reorg case where a local-safe block is not actually cross-safe.

When a local-safe block is detected to conflict:

1. Send update to chains-DB to roll back, and insert a replacement block entry into local/cross-safe DB.
2. Use feed to have connected sync nodes proactively follow the replacement block, rather than finding cross-safe conflicting later (at which point we then also need to handle replacement)
3. Sync nodes should insert the replacement block with a reorg, mark it as safe, and then then trigger a reset, so the nodes continues to derive on top of this cross-safe block.
4. Sync nodes should publish the replacement-block as event. Instead of a derived-from event. This can then be watched by the supervisor, and marked as cross-safe.

Tests

Additional context

Metadata

After this we still have two types of reorgs:

• L1 reorgs, and we need to roll back the supervisor (simple, already keyed by L1 blocks)
• L2 unsafe block conflicts: we won't know what is and isn't valid. We might need some heuristics here, to choose an unsafe chain that can be recovered, and remove the rest asap, so it's not batch-submitted.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 138 lines in your changes missing coverage. Please review.

Project coverage is 43.71%. Comparing base (9697e37) to head (fb74e75).
Report is 27 commits behind head on develop.

Files with missing lines	Patch %	Lines
op-supervisor/supervisor/backend/db/query.go	0.00%	56 Missing ⚠️
op-supervisor/supervisor/backend/db/update.go	0.00%	55 Missing ⚠️
op-node/rollup/interop/managed/system.go	0.00%	9 Missing ⚠️
op-supervisor/supervisor/backend/db/feeds.go	0.00%	6 Missing ⚠️
op-supervisor/supervisor/backend/db/db.go	0.00%	5 Missing ⚠️
op-node/rollup/engine/events.go	0.00%	4 Missing ⚠️
op-node/rollup/interop/managed/api.go	0.00%	2 Missing ⚠️
...r/supervisor/backend/processors/chain_processor.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop   #13645      +/-   ##
===========================================
- Coverage    47.15%   43.71%   -3.44%     
===========================================
  Files          951      864      -87     
  Lines        79645    72004    -7641     
  Branches       766      766              
===========================================
- Hits         37555    31480    -6075     
+ Misses       39228    38111    -1117     
+ Partials      2862     2413     -449     

Flag	Coverage Δ
cannon-go-tests-32	?
cannon-go-tests-64	?
contracts-bedrock-tests	87.90% <ø> (-2.60%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
op-supervisor/supervisor/types/types.go	6.74% <ø> (ø)
...r/supervisor/backend/processors/chain_processor.go	0.00% <0.00%> (ø)
op-node/rollup/interop/managed/api.go	0.00% <0.00%> (-90.91%)	⬇️
op-node/rollup/engine/events.go	70.70% <0.00%> (-6.08%)	⬇️
op-supervisor/supervisor/backend/db/db.go	0.00% <0.00%> (ø)
op-supervisor/supervisor/backend/db/feeds.go	0.00% <0.00%> (ø)
op-node/rollup/interop/managed/system.go	0.00% <0.00%> (-59.22%)	⬇️
op-supervisor/supervisor/backend/db/update.go	0.00% <0.00%> (ø)
op-supervisor/supervisor/backend/db/query.go	0.00% <0.00%> (ø)

... and 133 files with indirect coverage changes