feat: implement EIP-4844 #668

DaniPopes · 2023-08-30T20:20:41Z

Implements EIP-4844.

Closes #526.
Supersedes #536.

Rjected

just adding some comments on how we might be able to configure the trusted setup from reth

Rjected · 2023-08-30T23:19:11Z

crates/precompile/build.rs

+fn generate_kzg_settings() {
+    let in_path = "src/kzg/trusted_setup.txt";
+    let out_path = "src/kzg/generated_settings.rs";
+
+    let in_path = Path::new(in_path);
+    println!("cargo:rerun-if-changed={}", in_path.display());
+    assert!(in_path.exists());
+    let contents = format_kzg_settings(in_path);
+    fs::write(out_path, contents).unwrap();
+}


makes sense - generating consts here makes creating the default kzgsettings easy, and does not require reading the file on startup etc

Rjected · 2023-08-30T23:19:21Z

crates/precompile/src/kzg/mod.rs

+fn get_kzg_settings() -> &'static KzgSettings {
+    static SETTINGS: OnceCell<KzgSettings> = OnceCell::new();
+    SETTINGS.get_or_init(|| {
+        c_kzg::KzgSettings::load_trusted_setup(
+            generated_settings::G1_POINTS,
+            generated_settings::G2_POINTS,
+        )
+        .expect("failed to load trusted setup")
+    })
+}


This makes sense given how the precompiles have to access the kzgsettings, but we need to be able to configure this from reth as well somehow.

Maybe we could have a static mut and use a pattern similar to this:
https://github.com/tokio-rs/tracing/blob/bf05c61b7234eb1a196ef8d30f773d3ea6b60ce9/tracing-core/src/dispatch.rs#L329-L366

To provide a way to set the KzgSettings in reth?

I believe they use static mut and manual atomic operations because of MSRV restrictions, once_cell is probably better

mattsse · 2023-08-31T00:38:39Z

just flagging that it probably should be Arc because KzgSettings is not clone

ref

https://github.com/paradigmxyz/reth/blob/893f4cf2a27034f4550d3f4443a0fa7c2cd2444b/bin/reth/src/node/mod.rs#L575-L585

mattsse · 2023-08-31T00:45:36Z

crates/precompile/src/lib.rs

+        static INSTANCE: OnceCell<Precompiles> = OnceCell::new();
+        INSTANCE.get_or_init(|| {
+            let mut precompiles = Self::berlin().clone();


for precompiles, we should consider either an enum

enum Precompile { Fun(fn(&[u8], u64) -> PrecompileResult), Fn(Arc<dyn Send + Sync + Fn(&[u8], u64) -> PrecompileResult) }

or always use the Arc variant,

and a static mut Precompile list that supports configuring global defaults

This is in my TODO list :)

hm we can simplify a lot of things about initialization by adding function signature that has CfgEnv input
No global initialization at all, and we set KzgSettings inside CfgEnv.

@DaniPopes @mattsse @Rjected wdyt?

No global initialization at all, and we set KzgSettings inside CfgEnv.

I am open to that, as long as we can set a default that's hard-coded to Cancun/Sepolia/.. mainnet?

No global initialization at all, and we set KzgSettings inside CfgEnv.

I am open to that, as long as we can set a default that's hard-coded to Cancun/Sepolia/.. mainnet?

yeah this seems much nicer than messing around with globals

DaniPopes · 2023-08-31T00:46:50Z

@mattsse
What's clone needed for? AFAIK you just need to pass the instance pointer into C, so I'm thinking just OnceCell<KzgSettings>

mattsse · 2023-08-31T00:49:06Z

What's clone needed for?

we need the KzgSettings in multiple locations (for example tx pool validation) and ideally, we just want to load it once.

mattsse · 2023-08-31T00:53:53Z

hmm, the way precompiles are currently used

let mut precompiles = Self::berlin().clone();

it's only really useful to allow custom default precompiles starting with cancun hardfork, pre-cancun this doesn't really make sense.

so actually not sure yet what the ideal static mut should look like.
perhaps just a single static mut specifically for the kzg precompile?

or perhaps just use the kzg settings in a static mut

DaniPopes · 2023-08-31T01:06:48Z

we need the KzgSettings in multiple locations (for example tx pool validation) and ideally, we just want to load it once.

You can store a local &'static KzgSettings or get it when needed with a global getter function. I don't think Arcs are needed here.

See #668 (comment) for static muts.

mattsse · 2023-08-31T01:17:35Z

okay @Rjected I figured out how the hardfork precompiles are currently instantiated, and since we only need to access the KzgSettings once, a static Lazy<Mutex<Arc<KzgSettings>> should be enough to support custom KzgSettings if set at the start of the program

DaniPopes · 2023-08-31T03:19:39Z

crates/precompile/src/blob/kzg_settings/mod.rs

@@ -0,0 +1,33 @@
+use c_kzg::KzgSettings;


This module is exported, lmk wyt @Rjected @mattsse

I tend not to usemod inside revm

I think it makes sense here for now, kzg_settings::set_global like tracing::subscriber::set_global_default. It's equivalent of a ZST struct with the module functions as static methods on it. In the future we can think of a better solution.

With EnvCfg path of setting kzg setting, this is unnecessary.

crates/interpreter/src/instructions/host_env.rs

crates/precompile/build.rs

gakonst · 2023-08-31T16:51:13Z

crates/precompile/build.rs

+    // TODO: We can use `hex_literal` to reduce file size
+    // load g1 points
+    let mut g1_points = Vec::with_capacity(n_g1);
+    for _ in 0..n_g1 {
+        let line = lines.next().unwrap();
+        let mut bytes = [0; BYTES_PER_G1_POINT];
+        hex::decode_to_slice(line, &mut bytes).unwrap();
+        g1_points.push(bytes);
+    }
+
+    // load g2 points
+    let mut g2_points = Vec::with_capacity(n_g2);
+    for _ in 0..n_g2 {
+        let line = lines.next().unwrap();
+        let mut bytes = [0; BYTES_PER_G2_POINT];
+        hex::decode_to_slice(line, &mut bytes).unwrap();
+        g2_points.push(bytes);
+    }


Nit but as discussed you don't need to read all the points right?

It's done at compile-time. There's only 2 methods exposed to init a KzgSettings, with the points bytes, or with a file path.

gakonst · 2023-08-31T16:51:51Z

crates/precompile/build.rs

+/*
+/// Pros:
+/// - no need to call `load_trusted_setup` at runtime
+///
+/// Cons:
+/// - larger runtime static size (722K = `4096*32 + 4096*144 + 65*288`)
+/// - larger generated file size (2M)
+/// - possibly unsafe? `as_ptr() as *mut _`
+/// - depends on `c_kzg`; this might not matter much
+fn format_kzg_settings_large(in_path: &Path) -> String {


We should check about this w/ the c-kzg folks.

I think I'll delete the second option, it's way too big to have at runtime

crates/precompile/src/blob/mod.rs

gakonst · 2023-08-31T16:54:35Z

crates/precompile/src/lib.rs

+        static INSTANCE: OnceCell<Precompiles> = OnceCell::new();
+        INSTANCE.get_or_init(|| {
+            let mut precompiles = Self::berlin().clone();


No global initialization at all, and we set KzgSettings inside CfgEnv.

I am open to that, as long as we can set a default that's hard-coded to Cancun/Sepolia/.. mainnet?

bins/revme/src/statetest/runner.rs

crates/precompile/src/kzg_point_evaluation.rs

* feat: point evaluation precompile * feat: BLOBHASH opcode * refactor: revme runner * renames * export global kzg settings * feat: include kzg settings bytes with `include_bytes!` * build.rs: remove second option, update docs * revme: remove unused files and dead code * feat: implement remaining block and tx env fields * Add tests for helper functions, update constants * Add EIP-4844 EF tests to CI, skip outdated ones * chore: make skip_test more readable * Fix tests * Fix fmt * Fix lints, review * fix: validate new tx, block fields; add to balance check * Restore `load_access_list` * chore: drop c-kzg fork * test: update tests from Geth See: <ethereum/go-ethereum#28026> * chore: revert `is_create` change * chore: fmt toml * chore: unnecessary import * remove unsafe from fake_exponential * Remove kzg global settings, and move it to CfgEnv * enable kzg only in std. main README updated * fmt and clippy * Update README.md Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * nits and docs * disable exception eip4844 tests, small refactor * revert back last commit refactor --------- Co-authored-by: rakita <dragan0rakita@gmail.com> Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> Co-authored-by: Waylon Jepsen <57912727+0xJepsen@users.noreply.github.com>

DaniPopes added 4 commits August 30, 2023 22:12

feat: point evaluation precompile

aefca60

feat: BLOBHASH opcode

3342e99

refactor: revme runner

92e627b

renames

37feea7

Rjected reviewed Aug 30, 2023

View reviewed changes

mattsse reviewed Aug 31, 2023

View reviewed changes

export global kzg settings

9c048b4

DaniPopes force-pushed the eip-4844 branch from dc2094a to 9c048b4 Compare August 31, 2023 03:17

DaniPopes commented Aug 31, 2023

View reviewed changes

crates/interpreter/src/instructions/host_env.rs Show resolved Hide resolved

rakita mentioned this pull request Aug 31, 2023

Waylon/eip4844 #536

Closed

feat: include kzg settings bytes with include_bytes!

bb0a0e6

gakonst reviewed Aug 31, 2023

View reviewed changes

DaniPopes added 3 commits August 31, 2023 19:38

build.rs: remove second option, update docs

f4dbaf9

revme: remove unused files and dead code

7792a91

feat: implement remaining block and tx env fields

71a29cb

mattsse mentioned this pull request Aug 31, 2023

Reuse revm eip-4844 impls and constants paradigmxyz/reth#4444

Closed

DaniPopes added 6 commits September 1, 2023 18:29

Add tests for helper functions, update constants

16ff92c

Add EIP-4844 EF tests to CI, skip outdated ones

8d94cc5

chore: make skip_test more readable

44e5bea

Merge branch 'main' into eip-4844

25f0b83

Fix tests

85d25bc

Fix fmt

8c02e4a

rakita reviewed Sep 1, 2023

View reviewed changes

bins/revme/src/statetest/runner.rs Outdated Show resolved Hide resolved

DaniPopes commented Sep 15, 2023

View reviewed changes

crates/precompile/src/kzg_point_evaluation.rs Outdated Show resolved Hide resolved

revert back last commit refactor

eb8b223

rakita approved these changes Sep 15, 2023

View reviewed changes

rakita merged commit fa13fea into bluealloy:main Sep 15, 2023

DaniPopes deleted the eip-4844 branch September 28, 2023 15:43

github-actions bot mentioned this pull request Jan 12, 2024

chore: release #973

Closed

shekhirin mentioned this pull request Apr 18, 2024

feat(membership): add DaniPopes from Reth protocolguild/documentation#159

Merged

This was referenced Sep 9, 2024

chore: release #1765

Closed

chore: release #1775

Closed

chore: release v1.0.0 #1781

Closed

github-actions bot mentioned this pull request Nov 20, 2024

chore: release v1.0.0 #1866

Closed

This was referenced Dec 6, 2024

chore: release v1.0.0 #1897

Closed

chore: release v1.0.0 #1905

Closed

This was referenced Dec 16, 2024

chore: release v1.0.0 #1918

Closed

chore: release v1.0.0 lancorer/revm#1

Open

chore: release v1.0.0 leruaa/revm#1

Open

chore: release imotai/revm#78

Open

This was referenced Dec 23, 2024

chore: release v1.0.0 stevencartavia/revm#1

Open

chore: release v1.0.0 #1947

Closed

chore: release v1.0.0 ZzPoLariszZ/revm#1

Open

chore: release v1.0.0 mrLSD/revm#1

Closed

chore: release v1.0.0 leovct/revm#1

Open

This was referenced Dec 31, 2024

chore: release v1.0.0 FredCoen/revm#2

Open

chore: release v1.0.0 NomicFoundation/revm#1

Open

chore: release v1.0.0 dacozai/revm#1

Open

chore: release v1.0.0 all-forks/revm#1

Open

chore: release v1.0.0 elijahhampton/revm#1

Open

This was referenced Jan 7, 2025

chore: release v1.0.0 mrLSD/revm#2

Open

chore: release #1978

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: implement EIP-4844 #668

feat: implement EIP-4844 #668

DaniPopes commented Aug 30, 2023

Rjected left a comment

Rjected Aug 30, 2023

Rjected Aug 30, 2023

DaniPopes Aug 31, 2023

mattsse commented Aug 31, 2023

mattsse Aug 31, 2023

DaniPopes Aug 31, 2023

rakita Aug 31, 2023

gakonst Aug 31, 2023

Rjected Aug 31, 2023

DaniPopes commented Aug 31, 2023 •

edited

Loading

mattsse commented Aug 31, 2023

mattsse commented Aug 31, 2023 •

edited

Loading

DaniPopes commented Aug 31, 2023 •

edited

Loading

mattsse commented Aug 31, 2023 •

edited

Loading

DaniPopes Aug 31, 2023

rakita Aug 31, 2023

DaniPopes Aug 31, 2023

rakita Sep 5, 2023

gakonst Aug 31, 2023

DaniPopes Aug 31, 2023

gakonst Aug 31, 2023

DaniPopes Aug 31, 2023

gakonst Aug 31, 2023

feat: implement EIP-4844 #668

feat: implement EIP-4844 #668

Conversation

DaniPopes commented Aug 30, 2023

Rjected left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattsse commented Aug 31, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DaniPopes commented Aug 31, 2023 • edited Loading

mattsse commented Aug 31, 2023

mattsse commented Aug 31, 2023 • edited Loading

DaniPopes commented Aug 31, 2023 • edited Loading

mattsse commented Aug 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DaniPopes commented Aug 31, 2023 •

edited

Loading

mattsse commented Aug 31, 2023 •

edited

Loading

DaniPopes commented Aug 31, 2023 •

edited

Loading

mattsse commented Aug 31, 2023 •

edited

Loading