design-doc: supersim #34

fainashalts · 2024-06-20T04:56:06Z

Description

Design document for supersim, a local multichain development environment for app developers building on the Superchain.

Change formatting Update formatting

protolambda · 2024-07-17T15:59:59Z

ecosystem/supersim.md

+## Usage
+
+```bash
+// brings up two vanilla op chains with each other in the dep set with the same L1 chain


Besides two L2s, does it also include running the L1 chain itself? Or are there existing different preferred tools to test a L1 <> L2 chain combination in this context?

Yes it does - supersim's orchestrator spins up an L1 and multiple L2s.

protolambda · 2024-07-17T16:04:49Z

ecosystem/supersim.md

+
+
+
+          D1[Check Interop Message Invariants] -->|If Pass|D2


Clear debug messages, so the user understands how/why their interop executing message is invalid, would be really useful.

yes! will add this to be an explicit requirement.

protolambda · 2024-07-17T16:08:37Z

ecosystem/supersim.md

+    * Orchestrator should also run an L1 instance, though L1 withdrawals won’t initially be supported.
+    * Eventually supersim can be extended to be used with HardHat and other local simulation tools as well.
+
+3. **Genesis-applier** : The `genesis-applier` takes genesis files output by the monorepo and applies them to anvil instances.


The monorepo currently outputs allocs-X.json files, where X is a feature-set like a hardfork. E.g. allocs-ecotone.json. We might just want to output an additional allocs-supersim.json with any pre-configured things for the supersim, if more internal EVM things have to change.

Also note that some L2 contracts contain references to the L1, which would need to be updated, to make it work e.g. with different L1 deployments, chain-IDs, etc. Any improvements here, to make templating easier, like not using immutable but storage (so contract bytecode never has to be touched) would be really helpful, but has not been prioritized yet (even though it's currently affecting the monorepo testing, the superchain-registry, and the supersim).

Ooh the allocs approach is interesting! We have been talking about the best ways to identify what exactly should be used for genesis, and this could be a really helpful option.

We'll look into the L1 references in the L2 contracts. Maybe we can help make the changes if it makes sense. I'd like to be able to use the contracts as is rather than have to edit them on our end, if at all possible.

It would be useful to directly use the JSON allocs files to populate anvil instances in vanilla mode, would be surprised if they don't already support a CLI flag for this, where you can just pass a path to an allocs file and its applied. Then we can avoid the need to write another tool that converts an allocs file into JSON anvil_setCode/anvil_setStorage calls. Ideally we reuse as much of our existing tooling as possible here, so the deploy script with --sig runWithStateDump() so that the same genesis is created as in the production setup

@tynes makes sense! we're using anvil --init to pass a pre-made genesis.json

in terms of generating the state dump, definitely alligned on trying to use the deploy script as is. basically copied the logic in the python script in the monorepo and added some functionality to handle for multiple l2s https://github.com/ethereum-optimism/supersim/pull/53/files.

protolambda · 2024-07-17T16:09:49Z

ecosystem/supersim.md

+    * Can be multi-step: multiple genesis files can be applied idempotently, particularly in forking mode (when you fork a network that has bedrock contracts but not interop contracts, for instance).
+    * Genesis-applier should work as a standalone tool: if a developer is using Anvil outside the context of the orchestrator, they should be able to use genesis-applier to apply genesis state to their instance. In the context of supersim, the `orchestrator` will use the `genesis-applier` to apply the needed genesis state to the anvil instances it is running. 
+
+4. **Chain-config-loader**: Downloads the latest chain config from the superchain registry to allow the orchestrator to apply needed configs to anvil instances. Should be a standalone tool that works with the orchestrator but could be used directly on local testing nodes. 


If it's part of the other tooling, and written in Go or rust, then it should be relatively straight-forward to just import the superchain-registry as library. Like the op-node does.

protolambda · 2024-07-17T16:10:36Z

ecosystem/supersim.md

+
+### Services
+
+Given the application flow, the following services will be part of supersim: 


Will this all still be packaged into the same executable? I don't see a user installing and running 6 different tools

Yes 100%, the orchestrator spins these up but it is all "supersim."

mslipper · 2024-07-17T22:05:47Z

ecosystem/supersim.md

+
+Supersim will be a lightweight tool that simulates an interoperable Superchain environment locally. It will not require a complicated devnet setup and will be run using cli commands with configuration options that fall back to sensible defaults if they are not specified. Predefined state can be passed in at startup or the state can be forked based on specified block height.
+
+From a developer perspective, supersim will offer a command line interface through which to easily start up multiple local testnets that can be used interoperably via the [Optimism Interop Protocol](https://github.com/ethereum-optimism/specs/tree/main/specs/interop). 


How does it orchestrate each instance? Through Docker or something similar?

mslipper · 2024-07-17T22:06:26Z

ecosystem/supersim.md

+
+    1. Orchestrator acquires L1 reference block to get the latest L2 height for forking using l2-fork-state-finder. 
+    2. Orchestrator gets appropriate chain configs from the superchain registry using the chain-config-loader.
+    3. Orchestrator starts the requested chains at determined L2 block height.


Presumably this means that you'd need RPCs for all chains in your forked interop set, correct?

For forking, yes I don't see a way around that. We were planning to default to public RPCs where possible and allow users to override.

tynes · 2024-07-17T22:11:44Z

ecosystem/supersim.md

+
+3. **Genesis-applier** : The `genesis-applier` takes genesis files output by the monorepo and applies them to anvil instances.
+
+    * The genesis files needed will differ depending on whether the user is in vanilla or forked mode, and forked mode is unlikely to need anything other than the interop contracts initially. 


Could create a small script to install the interop contracts in the short term

tynes · 2024-07-17T22:14:53Z

ecosystem/supersim.md

+
+6. **Op-simulator**: Proxy server in front of anvil instances that simulates op-stack services without the real derivation pipeline.
+
+    * Simulates deposits by listening to the `optimismPortal` and forwarding deposit txs via `eth_sendRawTransaction`.


The foundry maintainers don't want to support deposit txs via RPC like this

fainashalts added 5 commits June 19, 2024 15:32

design-doc: supersim

2ac9cc0

Update op-simulator interface chart

efccce6

Update op-simulator interface description

78c47f8

Update sample commands to use supersim name

d2d17f1

Update doc to clarify some points

aea9d66

fainashalts mentioned this pull request Jun 24, 2024

Project: Supersim MVP ethereum-optimism/supersim#4

Closed

2 tasks

fainashalts requested review from nickbalestra, nitaliano, jakim929, hamdiallam and tremarkley June 24, 2024 16:53

fainashalts added 2 commits June 24, 2024 14:12

Change test script description

76972c0

Fix list formatting

7fce2c9

Change formatting Update formatting

fainashalts force-pushed the supersim branch from 5a86ba7 to 7fce2c9 Compare June 24, 2024 21:58

fainashalts marked this pull request as ready for review June 27, 2024 21:19

fainashalts mentioned this pull request Jul 12, 2024

Design Review: 7/17/24 #47

Closed

protolambda reviewed Jul 17, 2024

View reviewed changes

mslipper reviewed Jul 17, 2024

View reviewed changes

tynes reviewed Jul 17, 2024

View reviewed changes

mslipper approved these changes Jul 31, 2024

View reviewed changes

mslipper merged commit e183bf6 into main Jul 31, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

design-doc: supersim #34

design-doc: supersim #34

fainashalts commented Jun 20, 2024

protolambda Jul 17, 2024

fainashalts Jul 17, 2024

protolambda Jul 17, 2024

fainashalts Jul 17, 2024

protolambda Jul 17, 2024 •

edited

Loading

fainashalts Jul 17, 2024

tynes Jul 17, 2024

jakim929 Jul 17, 2024 •

edited

Loading

protolambda Jul 17, 2024

protolambda Jul 17, 2024

fainashalts Jul 17, 2024

mslipper Jul 17, 2024

mslipper Jul 17, 2024

fainashalts Jul 18, 2024

tynes Jul 17, 2024

tynes Jul 17, 2024


		### Services

		Given the application flow, the following services will be part of supersim:


		Supersim will be a lightweight tool that simulates an interoperable Superchain environment locally. It will not require a complicated devnet setup and will be run using cli commands with configuration options that fall back to sensible defaults if they are not specified. Predefined state can be passed in at startup or the state can be forked based on specified block height.

		From a developer perspective, supersim will offer a command line interface through which to easily start up multiple local testnets that can be used interoperably via the [Optimism Interop Protocol](https://github.com/ethereum-optimism/specs/tree/main/specs/interop).


		3. Genesis-applier : The `genesis-applier` takes genesis files output by the monorepo and applies them to anvil instances.

		* The genesis files needed will differ depending on whether the user is in vanilla or forked mode, and forked mode is unlikely to need anything other than the interop contracts initially.


		6. Op-simulator: Proxy server in front of anvil instances that simulates op-stack services without the real derivation pipeline.

		* Simulates deposits by listening to the `optimismPortal` and forwarding deposit txs via `eth_sendRawTransaction`.

design-doc: supersim #34

design-doc: supersim #34

Conversation

fainashalts commented Jun 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

protolambda Jul 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakim929 Jul 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

protolambda Jul 17, 2024 •

edited

Loading

jakim929 Jul 17, 2024 •

edited

Loading