feat(forge): support multiple forks #1715

mattsse · 2022-05-24T17:08:04Z

Motivation

support multiple forks during tests

hotswap state via cheatcode
snapshot + revert
change execution context between forks

Ref #1649 #834 #1162 #748 #939

Closes #834
Closes #1162

Solution

This introduces new cheatcodes:

foundry/testdata/cheats/Cheats.sol

Lines 106 to 118 in 9088be5

    
           // Snapshot the current state of the evm. 
        
           // Returns the id of the snapshot that was created. 
        
           // To revert a snapshot use `evmRevert` 
        
           function snapshot() external returns(uint256); 
        
           // Revert the state of the evm to a previous snapshot 
        
           // takes the snapshot id to revert to. This deletes the snapshot and all snapshots taken after the given snapshot id. 
        
           function revertTo(uint256) external; 
        
           // manually enables forking mode for the current test 
        
           function setFork(string,uint256) external; 
        
           // manually enables forking mode for the current test with the latest block number 
        
           function setFork(string) external; 
        
           // forks the `block` variable from the given endpoint 
        
           function forkBlockVariable(string, uint256) external;

forkBlockVariable basically addresses this #748

atm it's expected that fork related cheat codes are called in setup and snapshots from within the test.

another idea would be to add support for storing multiple RPC endpoints along with their aliases so that RPC endpoints can be used by an alias. Probably best to add this in the configuration

hexonaut · 2022-06-02T16:22:58Z

Copying this over from Discord. Basically local storage, memory and stack should be persistent and external calls use latest state of a particular defined fork.

Here is my intuition for how this would work:

function test_my_xchain_e2e_test() public {
	// Ran forge test without rpc url
	uint256 stackVar = 123;
	MyStruct memory memoryVar = a;
	MyStruct storage storageVar = a;

	address contract1 = address(new MyContract());

	uint256 myLocalMainnetForkId = vm.defineFork("MAINNET RPC");
	uint256 myLocalOptimismForkId = vm.defineFork("OPTIMISM RPC");

	// Still on local testing environment
	// stackVar == 123 && memoryVar == a && storageVar == a

	contract1.someFunc();	// Valid

	vm.switchFork(myLocalMainnetForkId);

	// Switched to mainnet latest block - fresh fork
	// stackVar == 123 && memoryVar == a && storageVar == a

	address contract2 = address(new MyContract());

	contract1.someFunc();	// Contract deployed on non-forked env - this will fail - address is still set
	contract2.someFunc();	// Valid

	vm.switchFork(myLocalOptimismForkId);

	address contract3 = address(new MyContract());

	// Run code on optimism latest block - fresh fork
	// stackVar == 123 && memoryVar == a && storageVar == a

	contract1.someFunc();	// Fail
	contract2.someFunc();	// Fail
	contract3.someFunc();	// Valid

	vm.switchFork(myLocalMainnetForkId);

	// Back on mainnet with local changes active
	// stackVar == 123 && memoryVar == a && storageVar == a

	contract1.someFunc();	// Fail
	contract2.someFunc();	// Valid
	contract3.someFunc();	// Fail

	vm.switchFork(myLocalOptimismForkId);

	// Back on optimism with latest local changes active
	// stackVar == 123 && memoryVar == a && storageVar == a
	contract1.someFunc();	// Fail
	contract2.someFunc();	// Fail
	contract3.someFunc();	// Valid
}

For a more concrete implementation we are currently simulating xchain integration tests like this: https://github.com/makerdao/xdomain/blob/ed0435803597ccb4a2328713c0ef8fc0093b2093/packages/dss-bridge/src/tests/Integration.t.sol#L129

mcd is a storage variable referencing currently active MCD on Ethereum. Fork supplied by ETH_RPC_URL
rmcd is a simulated xchain deploy of the remote chain (say on Optimism). It's still all in the same environment as we can't do this yet.

We would like to be able to swap between forks like in the example above and call into mcd or rmcd accordingly.

gakonst · 2022-07-09T21:43:10Z

Per @hexonaut's feedback / user testing I vote we get this merged and iterate on any bugs discovered, wdyt?

onbjerg · 2022-07-10T20:12:20Z

testdata/cheats/Cheats.sol

+    function selectFork(uint256) external;
+    // Updates the currently active fork to given block number
+    // This is similar to `roll` but for the fork
+    function rollFork(uint256) external;


Just a smol question, how does this work if you do not use any of the fork cheatcodes, but just use the CLI args? Still as if you had used one of the cheatcodes?

exactly, initializing with CLI args will be the same as creating and selecting during setup

onbjerg · 2022-07-10T20:13:23Z

testdata/cheats/Cheats.sol

+    // Creates a new fork with the given endpoint and the latest block and returns the identifier of the fork
+    function createFork(string calldata) external returns(uint256);
+    // takes a fork identifier created by `createFork` and changes the state
+    function selectFork(uint256) external;


Another question here actually: if you pass a fork URL using the CLI, then do createFork etc., how would you revert back to the initial fork passed via CLI? Is it possible? If not, should we discourage passing --rpc-url?

if we launch in fork mode (via CLI) this fork will always have id 0, I added another cheatcode activeFork()(uint256) that returns the currently active fork, so the id of the CLI fork can be retrieved as well

Great! 😄

onbjerg

Two small questions but LGTM and I think we should merge and iterate. No need to make any changes to the PR to address the questions I posed, just important to keep in mind re: documentation and so on

mattsse · 2022-07-11T12:57:20Z

Per @hexonaut's feedback / user testing I vote we get this merged and iterate on any bugs discovered, wdyt?

+1 on that, writing docs for the book now

gakonst

LGTM - some nits/qs, beyond that feel free to merge whenever.

gakonst · 2022-07-11T19:40:14Z

evm/src/executor/fork/multi.rs

+        // sending and receiving data from the remote client(s)
+        let _ = std::thread::Builder::new()
+            .name("multi-fork-backend-thread".to_string())
+            .spawn(move || {
+                let rt = tokio::runtime::Builder::new_current_thread()
+                    .enable_all()
+                    .build()
+                    .expect("failed to create multi-fork-backend-thread tokio runtime");


what's the difference between this and a tokio::spawn?

the runs handler in a standalone runtime on a separate thread

ya i guess what's the benefit? spawning a tokio future in a multithreaded runtime vs spawning a thread and launching a new runtime (is it just better? cuz you launch a new multithreaded runtime in a new thread?)

that's a dedicated thread in which the runtime's driving the single future. this rt doesn't spawn any threads but uses the current thread it's in.

tokio::spawn requires an existing runtime and we don't actually have any other async stuff when tests are executed.

gakonst · 2022-07-11T19:50:48Z

testdata/cheats/Fork.t.sol

+        // modify state
+        bytes32 value = bytes32(uint(1));
+        // "0x3617319a054d772f909f7c479a2cebe5066e836a939412e32403c99029b92eff" is the slot storing the balance of zero address for the weth contract
+        // `cast index address uint 0x0000000000000000000000000000000000000000 3`
+        bytes32 zero_address_balance_slot = 0x3617319a054d772f909f7c479a2cebe5066e836a939412e32403c99029b92eff;
+        cheats.store(WETH_TOKEN_ADDR, zero_address_balance_slot, value);
+        assertEq(WETH.balanceOf(0x0000000000000000000000000000000000000000), 1, "Cheatcode did not change value at the storage slot.");
+
+        // switch forks and ensure local modified state is persistent
+        cheats.selectFork(forkB);
+        assertEq(WETH.balanceOf(0x0000000000000000000000000000000000000000), 1, "Cheatcode did not change value at the storage slot.");


confused a bit here, why is the WETH balance the same in both forks? shouldn't it be 1 in just forkA and still be neq 1 in forkB?

writes are persistent across fork swaps, only the remote half is swapped.

I tried to explain this here https://github.com/foundry-rs/book/pull/442/files#diff-ef0012936dece3a24d02ea8d8e27bf208801f8be2e0689cb74a053c36351c05aR35

cc @hexonaut

any chance of changing the behaviour here? would be great if it's possible to have separate states for writes as well.

looking at this currently, should be possible.

testdata/cheats/RpcUrls.t.sol

forge/src/multi_runner.rs

gakonst · 2022-07-12T17:01:57Z

lfg

mattsse added 2 commits May 24, 2022 18:27

refactor: move backed to separate module

0139114

refactor: move fork db to evm crate

74059f4

mattsse added T-feature Type: feature C-forge Command: forge A-cheatcodes Area: cheatcodes labels May 24, 2022

mattsse added 4 commits May 24, 2022 19:51

feat: design multifork

b04b3fb

feat: more multi handler work

90e626b

Merge branch 'master' into matt/multifork

eca3a1e

fix: use new paths

e065ad7

mattsse mentioned this pull request May 25, 2022

Restructure evm crate #1725

Closed

3 tasks

mattsse added 5 commits May 28, 2022 18:58

Merge branch 'master' into matt/multifork

b430bd6

Merge branch 'master' into matt/multifork

558b755

Merge branch 'master' into matt/multifork

85f9757

Merge branch 'master' into matt/multifork

63ed4aa

describe cheatcodes

9088be5

mattsse added 14 commits June 3, 2022 13:09

chore: tune cheatcodes

3db5c4b

refactor: move in memory db to evm

1e81169

docs: add some docs

2426feb

refactor: redesign multifork backend

ae54fb0

feat: api improvements

f10b2c0

chore: bump revm

d7b1119

docs: more backend docs

7c7928c

Merge branch 'master' into matt/multifork

ddfd409

feat: implement multifork creation

c27657b

style: simplify locking

2a3507f

feat: add spawning

28769a4

feat: refactor backend types

a35a5e6

Merge branch 'master' into matt/multifork

242f1a3

Merge branch 'master' into matt/multifork

3a809ca

This was referenced Jul 8, 2022

feat(config): extend rpc endpoint support #2245

Merged

chore: formatting and typos #2246

Merged

mattsse added 2 commits July 8, 2022 15:23

Merge branch 'master' into matt/multifork

7bf0d16

Merge branch 'master' into matt/multifork

edf76a0

onbjerg reviewed Jul 10, 2022

View reviewed changes

onbjerg approved these changes Jul 10, 2022

View reviewed changes

mattsse added 6 commits July 11, 2022 14:12

Merge branch 'master' into matt/multifork

cb69234

feat: add more util cheat codes

60bce64

style: simplify create select

1201afc

docs: update docs

a0a7b8d

test: more fork tests

fbdf0e2

add active fork test

b5c6cbd

docs: update cheatcode docs

1a5b7ad

mattsse mentioned this pull request Jul 11, 2022

docs: document forking related cheatcodes foundry-rs/book#442

Merged

mattsse added 2 commits July 11, 2022 17:14

Merge branch 'master' into matt/multifork

77a209b

Merge branch 'master' into matt/multifork

c5dc174

gakonst approved these changes Jul 11, 2022

View reviewed changes

mattsse added 3 commits July 12, 2022 11:31

Merge branch 'master' into matt/multifork

7b80d43

fix: capture env in snapshot

5fe1c2b

test: add snapshot tests

b9b91b5

mattsse merged commit 4ae40da into foundry-rs:master Jul 12, 2022

This was referenced Jul 12, 2022

feat: add new fork + snapshot cheatcodes foundry-rs/forge-std#112

Merged

fix: use platform agnostic filter paths for cheat tests #2281

Merged

This was referenced Jul 13, 2022

feat(evm): use completely separated storage sections in multifork #2301

Merged

RPC cache possibly flaky since #1715 #2383

Closed

fix: properly enable storage caching #2385

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(forge): support multiple forks #1715

feat(forge): support multiple forks #1715

mattsse commented May 24, 2022 •

edited

Loading

hexonaut commented Jun 2, 2022 •

edited

Loading

gakonst commented Jul 9, 2022 •

edited

Loading

onbjerg Jul 10, 2022

mattsse Jul 11, 2022

onbjerg Jul 10, 2022

mattsse Jul 11, 2022

onbjerg Jul 11, 2022

onbjerg left a comment

mattsse commented Jul 11, 2022

gakonst left a comment

gakonst Jul 11, 2022

mattsse Jul 11, 2022

gakonst Jul 11, 2022 •

edited

Loading

mattsse Jul 11, 2022

gakonst Jul 11, 2022

mattsse Jul 11, 2022 •

edited

Loading

gakonst Jul 11, 2022

carmenjiawenc Jul 13, 2022

mattsse Jul 13, 2022

gakonst commented Jul 12, 2022

	// Snapshot the current state of the evm.
	// Returns the id of the snapshot that was created.
	// To revert a snapshot use `evmRevert`
	function snapshot() external returns(uint256);
	// Revert the state of the evm to a previous snapshot
	// takes the snapshot id to revert to. This deletes the snapshot and all snapshots taken after the given snapshot id.
	function revertTo(uint256) external;
	// manually enables forking mode for the current test
	function setFork(string,uint256) external;
	// manually enables forking mode for the current test with the latest block number
	function setFork(string) external;
	// forks the `block` variable from the given endpoint
	function forkBlockVariable(string, uint256) external;

feat(forge): support multiple forks #1715

feat(forge): support multiple forks #1715

Conversation

mattsse commented May 24, 2022 • edited Loading

Motivation

Solution

hexonaut commented Jun 2, 2022 • edited Loading

gakonst commented Jul 9, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

onbjerg left a comment

Choose a reason for hiding this comment

mattsse commented Jul 11, 2022

gakonst left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gakonst Jul 11, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattsse Jul 11, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gakonst commented Jul 12, 2022

mattsse commented May 24, 2022 •

edited

Loading

hexonaut commented Jun 2, 2022 •

edited

Loading

gakonst commented Jul 9, 2022 •

edited

Loading

gakonst Jul 11, 2022 •

edited

Loading

mattsse Jul 11, 2022 •

edited

Loading