op-supervisor: Add log db

[ajsutton]

Description

Introduces an append only database to record logs for a single chain. Reorgs can be handled by rewinding the chain back to a specific block then adding the new canonical logs.

This implements the basic log storage and search functionality but has two key pieces not yet implemented:

1. Storing and returning executing message dependencies for logs
2. Recovering the database after write errors, particularly when a partial write occurs. An error is returned to the caller to indicate the write failed, but there's no code to handle truncating any partially written data or ensuring the data cached in memory is rolled back.

These items will be added in a follow up PR.

Also note that this doesn't attempt to track different levels of safety. Ideally an outer wrapper can track that based on block number. Storing those head levels to disk may require them being handled inside this struct though ideally I'd like them to just be a prefix and manage to keep this struct completely focussed on storing the log data.

A simple RWMutex is used to provide thread safety, it may be better to remove that again in the future and have thread safety be handled by the wrapper. Or if we want to maximise performance it may be possible to have more fine-grained locking so that checking if logs exist can be safely done in parallel with adding blocks, with only rewinds needing exclusive access. Going to avoid adding that complexity until we know it's needed.

Tests

Added unit tests. Each test is run first by writing and reading from the same DB instance, then writes and performs the reads from a separate instance to confirm it can be reread purely from the information on disk. Also implemented a set of invariant checks that are run after each test to verify key properties of the database format.

Metadata

• Interop: op-supervisor logdb #10857

[semgrep-app]

Semgrep found 8 todos_require_linear findings:

• op-supervisor/supervisor/backend/db/db.go
  • L143 - Triage
  • L146 - Triage
  • L206 - Triage
  • L208 - Triage
  • L354 - Triage
  • L393 - Triage
  • L395 - Triage
  • L396 - Triage

Please create a GitHub ticket for this TODO.

_{Ignore this finding from todos_require_linear.}

[protolambda]

Functionality is good, thank you! I have some suggestions to introduce more fine typing and separate the file-read/write layer from the entry-read/write layer, to make things more readable.

[Inphi]

just some quick notes. I'll take a closer look later.

[protolambda]

Very nice!