Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(forge): support multiple forks #1715

Merged
merged 148 commits into from
Jul 12, 2022
Merged
Show file tree
Hide file tree
Changes from 136 commits
Commits
Show all changes
148 commits
Select commit Hold shift + click to select a range
0139114
refactor: move backed to separate module
mattsse May 24, 2022
74059f4
refactor: move fork db to evm crate
mattsse May 24, 2022
b04b3fb
feat: design multifork
mattsse May 24, 2022
90e626b
feat: more multi handler work
mattsse May 24, 2022
eca3a1e
Merge branch 'master' into matt/multifork
mattsse May 25, 2022
e065ad7
fix: use new paths
mattsse May 25, 2022
b430bd6
Merge branch 'master' into matt/multifork
mattsse May 28, 2022
558b755
Merge branch 'master' into matt/multifork
mattsse May 31, 2022
85f9757
Merge branch 'master' into matt/multifork
mattsse Jun 1, 2022
63ed4aa
Merge branch 'master' into matt/multifork
mattsse Jun 2, 2022
9088be5
describe cheatcodes
mattsse Jun 2, 2022
3db5c4b
chore: tune cheatcodes
mattsse Jun 3, 2022
1e81169
refactor: move in memory db to evm
mattsse Jun 3, 2022
2426feb
docs: add some docs
mattsse Jun 3, 2022
ae54fb0
refactor: redesign multifork backend
mattsse Jun 3, 2022
f10b2c0
feat: api improvements
mattsse Jun 3, 2022
d7b1119
chore: bump revm
mattsse Jun 3, 2022
7c7928c
docs: more backend docs
mattsse Jun 3, 2022
ddfd409
Merge branch 'master' into matt/multifork
mattsse Jun 4, 2022
c27657b
feat: implement multifork creation
mattsse Jun 4, 2022
2a3507f
style: simplify locking
mattsse Jun 4, 2022
28769a4
feat: add spawning
mattsse Jun 4, 2022
a35a5e6
feat: refactor backend types
mattsse Jun 4, 2022
242f1a3
Merge branch 'master' into matt/multifork
mattsse Jun 5, 2022
3a809ca
Merge branch 'master' into matt/multifork
mattsse Jun 5, 2022
8d8839f
feat: complete fork api
mattsse Jun 5, 2022
7b32b73
refactor: simplify TestFilter
mattsse Jun 5, 2022
35ddffe
refactor: extract helper types
mattsse Jun 5, 2022
c0d4c34
refactor: restructure runner
mattsse Jun 5, 2022
37e5210
chore(clippy): make clippy happy
mattsse Jun 5, 2022
6eb09b5
refactor: extract types
mattsse Jun 5, 2022
da478bb
refactor: simplify create2 deployer fn
mattsse Jun 5, 2022
d46a075
Merge branch 'master' into matt/multifork
mattsse Jun 5, 2022
9ae19f5
cleanup
mattsse Jun 5, 2022
fa36c68
fix: failing tests
mattsse Jun 5, 2022
6acbfce
test: ensure solc finished successfully
mattsse Jun 5, 2022
ca75693
refactor: introduce more types
mattsse Jun 6, 2022
0cd76e1
Merge branch 'master' into matt/multifork
mattsse Jun 6, 2022
a1e3f72
feat: add a bunch of revm trait impls
mattsse Jun 6, 2022
b1a9833
clean up types
mattsse Jun 6, 2022
a63c141
refactor: remove generic Inspector impl
mattsse Jun 6, 2022
968c71a
feat: introduce Backendtrait
mattsse Jun 6, 2022
09e9b53
chore: remove old types
mattsse Jun 6, 2022
13fadcf
refactor: rename Backend type
mattsse Jun 6, 2022
6f1ebcc
refactor: rename DatbaseExt trait
mattsse Jun 6, 2022
30de433
feat: integrate new Backend type
mattsse Jun 6, 2022
fa4e694
revertuse refcell again
mattsse Jun 6, 2022
7e1e241
Merge branch 'master' into matt/multifork
mattsse Jun 7, 2022
752312b
refactor: change to dedicated fuzz backend
mattsse Jun 7, 2022
b3f955b
refactor: refactor conversion
mattsse Jun 7, 2022
f102cf1
chore: some cleanup
mattsse Jun 7, 2022
d047200
Merge branch 'master' into matt/multifork
mattsse Jun 8, 2022
5258a5b
refactor: extract Fuzzbackend wrapper
mattsse Jun 8, 2022
112d05b
feat: implement cheat codes
mattsse Jun 8, 2022
30e0b84
feat: implement fork cheat codes
mattsse Jun 8, 2022
3f667c1
refactor: make it compile again
mattsse Jun 8, 2022
ffc433c
Merge branch 'master' into matt/multifork
mattsse Jun 8, 2022
c4b4002
refactor: add shutdown routine to Multifork
mattsse Jun 8, 2022
fa47f9c
refactor: improve backend
mattsse Jun 8, 2022
d9c7ac9
Merge branch 'master' into matt/multifork
mattsse Jun 8, 2022
5d0758c
Merge branch 'master' into matt/multifork
mattsse Jun 8, 2022
7a4792a
make everything compile again
mattsse Jun 8, 2022
8a31ba6
Merge branch 'master' into matt/multifork
mattsse Jun 10, 2022
ded855b
add auto impl
mattsse Jun 10, 2022
54f4282
add config
mattsse Jun 10, 2022
b2eb70b
refactor: update outdated code
mattsse Jun 10, 2022
243d0dd
chore: cleanup some code
mattsse Jun 10, 2022
df9cb01
Merge branch 'master' into matt/multifork
mattsse Jun 13, 2022
5703dee
Merge branch 'master' into matt/multifork
mattsse Jun 13, 2022
e1e65d8
chore: make it compile again
mattsse Jun 13, 2022
2edda13
test: update failing tests
mattsse Jun 13, 2022
61e563a
chore(clippy): make clippy happy
mattsse Jun 13, 2022
c14473d
test: add simple fork cheatcode tests
mattsse Jun 13, 2022
032931f
refactor: use execute function
mattsse Jun 13, 2022
727a0ea
refactor: move snapshot cheatcode impl to separate mod
mattsse Jun 13, 2022
c7c3940
feat: store subroutine with snapshot
mattsse Jun 13, 2022
24f4374
Merge branch 'master' into matt/multifork
mattsse Jun 13, 2022
c4d2097
feat: add subroutine to revert call
mattsse Jun 13, 2022
b737aeb
Merge branch 'master' into matt/multifork
mattsse Jun 14, 2022
eeb8a80
Merge branch 'master' into matt/multifork
mattsse Jun 14, 2022
fcb7ba5
Merge branch 'master' into matt/multifork
mattsse Jun 15, 2022
192e122
Merge branch 'master' into matt/multifork
mattsse Jun 17, 2022
c0563f3
feat: add Cheats config type
mattsse Jun 17, 2022
5f0d598
work on snapshots
mattsse Jun 17, 2022
596dff5
Merge branch 'master' into matt/multifork
mattsse Jun 17, 2022
a7e5d5a
docs: write additional revert docs
mattsse Jun 17, 2022
8e82b65
Merge branch 'master' into matt/multifork
mattsse Jun 20, 2022
ee5f6c8
Merge branch 'master' into matt/multifork
mattsse Jun 21, 2022
f740d87
Merge branch 'master' into matt/multifork
mattsse Jun 22, 2022
bb669c5
Merge branch 'master' into matt/multifork
mattsse Jun 23, 2022
0b9b6f7
feat: check for failures
mattsse Jun 23, 2022
08ea3cc
Merge branch 'master' into matt/multifork
mattsse Jun 23, 2022
0ef157a
fix: make compile again
mattsse Jun 23, 2022
c39cf43
Merge branch 'master' into matt/multifork
mattsse Jun 23, 2022
5dd540c
refactor: rename
mattsse Jun 24, 2022
23e404c
Merge branch 'master' into matt/multifork
mattsse Jun 25, 2022
7c486ea
Merge branch 'master' into matt/multifork
mattsse Jul 1, 2022
0ed3708
refactor: unify backend code
mattsse Jul 1, 2022
758f94b
Merge branch 'master' into matt/multifork
mattsse Jul 1, 2022
e629551
feat: resolve rpc aliases
mattsse Jul 1, 2022
2a105ed
feat: periodically flush rpc cache
mattsse Jul 1, 2022
5ba6f8a
chore: derive default
mattsse Jul 1, 2022
0470107
chore: use revm head
mattsse Jun 15, 2022
cc48ce8
chore: bump revm
gakonst Jul 1, 2022
987bbd9
build: use revm 1.6.0
onbjerg Jul 3, 2022
6a50138
fix: use new revm data structures
onbjerg Jul 3, 2022
0ca3add
Merge branch 'master' into matt/multifork
mattsse Jul 4, 2022
03054d6
feat: add roll fork cheat codes
mattsse Jul 4, 2022
16a5f39
feat: add rpc helper functions
mattsse Jul 4, 2022
5a7c641
docs: document rpc endpoints table
mattsse Jul 4, 2022
7745ecd
test: add rpc endpoint tests
mattsse Jul 4, 2022
0c30cd1
Delete run.rs
gakonst Jul 4, 2022
ad1ac2b
Merge branch 'master' into matt/multifork
mattsse Jul 6, 2022
e9b707d
work on roll fork
mattsse Jul 6, 2022
16a5435
refactor: use local fork ids as ints
mattsse Jul 6, 2022
30457b7
Merge branch 'master' into matt/multifork
mattsse Jul 7, 2022
8479b7d
test: update fork test
mattsse Jul 7, 2022
c19f0c8
extend trait
mattsse Jul 7, 2022
68d067d
Merge branch 'master' into matt/check-latest-revm
mattsse Jul 7, 2022
8d8a8db
fix: migrate new revm api
mattsse Jul 7, 2022
cf05836
patch revm git
mattsse Jul 7, 2022
5783c6c
Merge branch 'matt/check-latest-revm' into matt/multifork
mattsse Jul 7, 2022
fd65aa9
use revm naming
mattsse Jul 7, 2022
49920d8
fix: rpc urls api
mattsse Jul 7, 2022
611f50c
fix: return encoded errors
mattsse Jul 7, 2022
1efefe2
chore: rustfmt
mattsse Jul 7, 2022
42e5c86
Merge branch 'master' into matt/multifork
mattsse Jul 7, 2022
c5059d1
chore: rustfmt
mattsse Jul 7, 2022
c7e377c
feat: update env when selecting fork
mattsse Jul 7, 2022
36389a8
fix: fix a ton of bugs
mattsse Jul 7, 2022
86ddd0a
fix: failing tests
mattsse Jul 7, 2022
d9b2296
Merge branch 'master' into matt/multifork
mattsse Jul 8, 2022
ce7c601
chore: rm unused cheat
mattsse Jul 8, 2022
42d5485
chore: rm unused types
mattsse Jul 8, 2022
7bf0d16
Merge branch 'master' into matt/multifork
mattsse Jul 8, 2022
edf76a0
Merge branch 'master' into matt/multifork
mattsse Jul 8, 2022
cb69234
Merge branch 'master' into matt/multifork
mattsse Jul 11, 2022
60bce64
feat: add more util cheat codes
mattsse Jul 11, 2022
1201afc
style: simplify create select
mattsse Jul 11, 2022
a0a7b8d
docs: update docs
mattsse Jul 11, 2022
fbdf0e2
test: more fork tests
mattsse Jul 11, 2022
b5c6cbd
add active fork test
mattsse Jul 11, 2022
1a5b7ad
docs: update cheatcode docs
mattsse Jul 11, 2022
77a209b
Merge branch 'master' into matt/multifork
mattsse Jul 11, 2022
c5dc174
Merge branch 'master' into matt/multifork
mattsse Jul 11, 2022
7b80d43
Merge branch 'master' into matt/multifork
mattsse Jul 12, 2022
5fe1c2b
fix: capture env in snapshot
mattsse Jul 12, 2022
b9b91b5
test: add snapshot tests
mattsse Jul 12, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 16 additions & 16 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -64,4 +64,4 @@ debug = 0
#ethers-solc = { path = "../ethers-rs/ethers-solc" }

[patch.crates-io]
revm = { git = "https://github.com/bluealloy/revm", package = "revm" }
revm = { git = "https://github.com/bluealloy/revm", package = "revm" }
13 changes: 4 additions & 9 deletions cli/src/cmd/cast/run.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use crate::{cmd::Cmd, utils, utils::consume_config_rpc_url};
use crate::{cmd::Cmd, utils::consume_config_rpc_url};
use cast::trace::{identifier::SignaturesIdentifier, CallTraceDecoder};
use clap::Parser;
use ethers::{
Expand All @@ -10,7 +10,7 @@ use ethers::{
use forge::{
debug::DebugArena,
executor::{
builder::Backend, inspector::CheatsConfig, opts::EvmOpts, DeployResult, ExecutorBuilder,
inspector::CheatsConfig, opts::EvmOpts, Backend, DeployResult, ExecutorBuilder,
RawCallResult,
},
trace::{identifier::EtherscanIdentifier, CallTraceArena, CallTraceDecoderBuilder, TraceKind},
Expand Down Expand Up @@ -75,8 +75,7 @@ impl RunArgs {

// Set up the execution environment
let env = evm_opts.evm_env().await;
let db =
Backend::new(utils::get_fork(&evm_opts, &config.rpc_storage_caching), &env).await;
let db = Backend::spawn(evm_opts.get_fork(env.clone()));

let builder = ExecutorBuilder::default()
.with_config(env)
Expand Down Expand Up @@ -112,11 +111,7 @@ impl RunArgs {

// Execute our transaction
let mut result = {
executor.set_tracing(true).set_gas_limit(tx.gas);

if self.debug {
executor.set_debugger(true);
}
executor.set_tracing(true).set_gas_limit(tx.gas).set_debugger(self.debug);

if let Some(to) = tx.to {
let RawCallResult { reverted, gas, traces, debug: run_debug, .. } =
Expand Down
10 changes: 7 additions & 3 deletions cli/src/cmd/forge/coverage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ use forge::{
coverage::{
CoverageMap, CoverageReporter, DebugReporter, LcovReporter, SummaryReporter, Visitor,
},
executor::opts::EvmOpts,
executor::{inspector::CheatsConfig, opts::EvmOpts},
result::SuiteResult,
trace::identifier::LocalTraceIdentifier,
MultiContractRunnerBuilder,
Expand Down Expand Up @@ -236,16 +236,20 @@ impl CoverageArgs {
let fuzzer = proptest::test_runner::TestRunner::new(cfg);
let root = project.paths.root;

let env = evm_opts.evm_env_blocking();

// Build the contract runner
let evm_spec = utils::evm_spec(&config.evm_version);
let mut runner = MultiContractRunnerBuilder::default()
.fuzzer(fuzzer)
.initial_balance(evm_opts.initial_balance)
.evm_spec(evm_spec)
.sender(evm_opts.sender)
.with_fork(utils::get_fork(&evm_opts, &config.rpc_storage_caching))
.with_fork(evm_opts.get_fork(env.clone()))
.with_cheats_config(CheatsConfig::new(&config, &evm_opts))
.set_coverage(true)
.build(root.clone(), output, evm_opts)?;
.build(root.clone(), output, env, evm_opts)?;

let (tx, rx) = channel::<(String, SuiteResult)>();

// Set up identifier
Expand Down
23 changes: 7 additions & 16 deletions cli/src/cmd/forge/script/executor.rs
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
use super::*;
use crate::{
cmd::{
forge::script::{sequence::TransactionWithMetadata, *},
needs_setup,
},
cmd::{forge::script::sequence::TransactionWithMetadata, needs_setup},
utils,
};
use cast::executor::inspector::CheatsConfig;
use ethers::{
solc::artifacts::CompactContractBytecode,
types::{transaction::eip2718::TypedTransaction, Address, U256},
};
use forge::{
executor::{builder::Backend, inspector::CheatsConfig, ExecutorBuilder},
executor::{Backend, ExecutorBuilder},
trace::CallTraceDecoder,
};
use std::collections::VecDeque;
Expand Down Expand Up @@ -146,24 +145,16 @@ impl ScriptArgs {
}

/// Creates the Runner that drives script execution
async fn prepare_runner(
&self,
script_config: &ScriptConfig,
sender: Address,
) -> ScriptRunner<Backend> {
async fn prepare_runner(&self, script_config: &ScriptConfig, sender: Address) -> ScriptRunner {
let env = script_config.evm_opts.evm_env().await;

// the db backend that serves all the data
let db = Backend::new(
utils::get_fork(&script_config.evm_opts, &script_config.config.rpc_storage_caching),
&env,
)
.await;
let db = Backend::spawn(script_config.evm_opts.get_fork(env.clone()));

let executor = ExecutorBuilder::default()
.with_cheatcodes(CheatsConfig::new(&script_config.config, &script_config.evm_opts))
.with_config(env)
.with_spec(crate::utils::evm_spec(&script_config.config.evm_version))
.with_spec(utils::evm_spec(&script_config.config.evm_version))
.with_gas_limit(script_config.evm_opts.gas_limit())
.set_tracing(script_config.evm_opts.verbosity >= 3 || self.debug)
.set_debugger(self.debug)
Expand Down
10 changes: 5 additions & 5 deletions cli/src/cmd/forge/script/runner.rs
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
use super::*;
use ethers::types::{Address, Bytes, NameOrAddress, U256};
use forge::{
executor::{CallResult, DatabaseRef, DeployResult, EvmError, Executor, RawCallResult},
executor::{CallResult, DeployResult, EvmError, Executor, RawCallResult},
trace::{CallTraceArena, TraceKind},
CALLER,
};

/// Drives script execution
pub struct ScriptRunner<DB: DatabaseRef> {
pub executor: Executor<DB>,
pub struct ScriptRunner {
pub executor: Executor,
pub initial_balance: U256,
pub sender: Address,
}

impl<DB: DatabaseRef> ScriptRunner<DB> {
pub fn new(executor: Executor<DB>, initial_balance: U256, sender: Address) -> Self {
impl ScriptRunner {
pub fn new(executor: Executor, initial_balance: U256, sender: Address) -> Self {
Self { executor, initial_balance, sender }
}

Expand Down
7 changes: 5 additions & 2 deletions cli/src/cmd/forge/test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -490,16 +490,19 @@ pub fn custom_run(args: TestArgs, include_fuzz_tests: bool) -> eyre::Result<Test
evm_opts.verbosity = 3;
}

let env = evm_opts.evm_env_blocking();

// Prepare the test builder
let evm_spec = utils::evm_spec(&config.evm_version);

let mut runner = MultiContractRunnerBuilder::default()
.fuzzer(fuzzer)
.initial_balance(evm_opts.initial_balance)
.evm_spec(evm_spec)
.sender(evm_opts.sender)
.with_fork(utils::get_fork(&evm_opts, &config.rpc_storage_caching))
.with_fork(evm_opts.get_fork(env.clone()))
.with_cheats_config(CheatsConfig::new(&config, &evm_opts))
.build(project.paths.root, output, evm_opts)?;
.build(project.paths.root, output, env, evm_opts)?;

if args.debug.is_some() {
filter.test_pattern = args.debug;
Expand Down
61 changes: 3 additions & 58 deletions cli/src/utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@ use ethers::{
types::U256,
utils::format_units,
};
use forge::executor::{opts::EvmOpts, Fork, SpecId};
use foundry_config::{cache::StorageCachingConfig, Config};
use forge::executor::SpecId;
use foundry_config::Config;
use std::{
future::Future,
ops::Mul,
path::{Path, PathBuf},
path::Path,
process::{Command, Output},
str::FromStr,
sync::Arc,
Expand Down Expand Up @@ -171,61 +171,6 @@ pub fn block_on<F: Future>(future: F) -> F::Output {
rt.block_on(future)
}

/// Helper function that returns the [Fork] to use, if any.
///
/// storage caching for the [Fork] will be enabled if
/// - `fork_url` is present
/// - `fork_block_number` is present
/// - [StorageCachingConfig] allows the `fork_url` + chain id pair
/// - storage is allowed (`no_storage_caching = false`)
///
/// If all these criteria are met, then storage caching is enabled and storage info will be written
/// to [Config::foundry_cache_dir()]/<str(chainid)>/<block>/storage.json
///
/// for `mainnet` and `--fork-block-number 14435000` on mac the corresponding storage cache will be
/// at `~/.foundry/cache/mainnet/14435000/storage.json`
pub fn get_fork(evm_opts: &EvmOpts, config: &StorageCachingConfig) -> Option<Fork> {
/// Returns the path where the cache file should be stored
///
/// or `None` if caching should not be enabled
///
/// See also [ Config::foundry_block_cache_file()]
fn get_block_storage_path(
evm_opts: &EvmOpts,
config: &StorageCachingConfig,
chain_id: u64,
) -> Option<PathBuf> {
if evm_opts.no_storage_caching {
// storage caching explicitly opted out of
return None
}
let url = evm_opts.fork_url.as_ref()?;
// cache only if block explicitly pinned
let block = evm_opts.fork_block_number?;

if config.enable_for_endpoint(url) && config.enable_for_chain_id(chain_id) {
return Config::foundry_block_cache_file(chain_id, block)
}

None
}

if let Some(ref url) = evm_opts.fork_url {
let chain_id = evm_opts.get_chain_id();
let cache_storage = get_block_storage_path(evm_opts, config, chain_id);
let fork = Fork {
url: url.clone(),
pin_block: evm_opts.fork_block_number,
cache_path: cache_storage,
chain_id,
initial_backoff: evm_opts.fork_retry_backoff.unwrap_or(50),
};
return Some(fork)
}

None
}

/// Conditionally print a message
///
/// This macro accepts a predicate and the message to print if the predicate is tru
Expand Down
Loading