Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forkless state transition with upgradable WASM executor #1716

Merged
merged 77 commits into from
Mar 26, 2024
Merged
Changes from 1 commit
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
3783579
Extraction of the `FuelBlockSecondaryKeyBlockHeights` table to an off…
xgreenx Feb 15, 2024
3c2d424
Updated CHANGELOG.md
xgreenx Feb 15, 2024
7e8689c
Merge branch 'master' into feature/blocks-second-key-extraction
xgreenx Feb 17, 2024
1d95e63
Merge with `master`
xgreenx Feb 17, 2024
95dfa58
Fix CI
xgreenx Feb 17, 2024
111efbd
Apply comments from the PR
xgreenx Feb 18, 2024
cabca80
Merge branch 'master' into feature/blocks-second-key-extraction
xgreenx Feb 18, 2024
0e20633
Merge branch 'master' into feature/blocks-second-key-extraction
xgreenx Feb 18, 2024
4e8ebc6
Merge branch 'master' into feature/blocks-second-key-extraction
Voxelot Feb 20, 2024
ad09341
Moved `StorageTransaction` to the `fuel-core-storage` crate
xgreenx Feb 21, 2024
b619743
Merge branch 'feature/blocks-second-key-extraction' into feature/comm…
xgreenx Feb 21, 2024
71dbc41
Merge branch 'master' into feature/blocks-second-key-extraction
xgreenx Feb 21, 2024
593ad52
Apply comments from the review
xgreenx Feb 21, 2024
97a96d5
Renamed `FuelBlockSecondaryKeyBlockHeights`
xgreenx Feb 21, 2024
3ce8c17
Remove contstrain
xgreenx Feb 21, 2024
802578e
Merge branch 'master' into feature/blocks-second-key-extraction
xgreenx Feb 21, 2024
5cc6c33
Merge branch 'feature/blocks-second-key-extraction' into feature/comm…
xgreenx Feb 22, 2024
0137658
Merge conflicts
xgreenx Feb 22, 2024
cc86bb6
Merge branch 'master' into feature/commit-changes
xgreenx Feb 22, 2024
5b9bd78
Generalized the implementation of the iterator functionality.
xgreenx Feb 22, 2024
7b1141a
Implemented tracking of the height during committing of the changes.
xgreenx Feb 23, 2024
305e567
Small nits
xgreenx Feb 23, 2024
4716c15
Make clippy happy
xgreenx Feb 23, 2024
518a842
Updated CHANGELOG.md
xgreenx Feb 23, 2024
253d188
Merge branch 'master' into feature/commit-changes
xgreenx Feb 27, 2024
f44b903
Removed TODOs
xgreenx Feb 27, 2024
00ef712
Apply comments from the PR
xgreenx Feb 27, 2024
0c27ff9
Apply comments from the PR
xgreenx Feb 28, 2024
fb5bc50
Merge branch 'master' into feature/commit-changes
xgreenx Feb 28, 2024
0c63393
Forkless state transition with upgradable WASM executor
xgreenx Mar 1, 2024
25b9f57
Merge branch 'master' into feature/commit-changes
xgreenx Mar 6, 2024
309a8bf
Merge with `master` before regenesis
xgreenx Mar 6, 2024
2e64af8
Merge branch 'master' into feature/commit-changes
xgreenx Mar 7, 2024
d682dd8
Merged with the `master`
xgreenx Mar 7, 2024
9344069
Address comemnts form the PR
xgreenx Mar 7, 2024
154328c
Merge branch 'master' into feature/commit-changes
xgreenx Mar 11, 2024
f8b9c07
Merge branch 'master' into feature/commit-changes
xgreenx Mar 11, 2024
d0b95df
Merge branch 'master' into feature/commit-changes
xgreenx Mar 11, 2024
ec92011
Merged `master`
xgreenx Mar 11, 2024
ffecd1d
Fixed tests
xgreenx Mar 11, 2024
dd42659
Some configuration for performance
xgreenx Mar 12, 2024
87e9524
Merge branch 'feature/commit-changes' into feature/forkless-state-tra…
xgreenx Mar 12, 2024
8b3be42
Applied comments from the PR
xgreenx Mar 13, 2024
3899b2b
Merge branch 'feature/commit-changes' into feature/forkless-state-tra…
xgreenx Mar 13, 2024
e8295ea
Merged with parent branch
xgreenx Mar 13, 2024
e12cb77
Remove patch section
xgreenx Mar 13, 2024
944a273
Make CI happy
xgreenx Mar 13, 2024
106df96
Make CI happy
xgreenx Mar 13, 2024
d92380a
Added `wasm32-unknown-unknown` and removed `protobuf`, since libp2p u…
xgreenx Mar 13, 2024
551baf6
Updated the implementation to show the relationship to the semaphore.
xgreenx Mar 13, 2024
47d6df8
Merge branch 'feature/commit-changes' into feature/forkless-state-tra…
xgreenx Mar 13, 2024
f909992
Small nits
xgreenx Mar 13, 2024
e21e58e
Hidded the wasm executor udnder the feature flag.
xgreenx Mar 13, 2024
dd45a89
Merge branch 'feature/commit-changes' into feature/forkless-state-tra…
xgreenx Mar 13, 2024
a94af64
Merge branch 'master' into feature/forkless-state-transition
xgreenx Mar 13, 2024
0916f64
Reverted changes in the cargo lock file.
xgreenx Mar 14, 2024
c37b5bb
Make CI happy
xgreenx Mar 14, 2024
15b921a
Self review
xgreenx Mar 14, 2024
e584453
Added comments, fixed clippy, udpated chagnelog
xgreenx Mar 15, 2024
98bbe6b
Merge branch 'master' into feature/forkless-state-transition
xgreenx Mar 16, 2024
b7da355
Merge branch 'master' into feature/forkless-state-transition
xgreenx Mar 18, 2024
b8ac0e8
Merge branch 'master' into feature/forkless-state-transition
xgreenx Mar 18, 2024
91a7c59
Merged with `master`
xgreenx Mar 18, 2024
e789e4e
Merge branch 'master' into feature/forkless-state-transition
xgreenx Mar 18, 2024
6999c47
Merge branch 'master' into feature/forkless-state-transition
xgreenx Mar 19, 2024
243ac6c
Applied suggestions from the PR.
xgreenx Mar 19, 2024
967289f
Merge branch 'master' into feature/forkless-state-transition
xgreenx Mar 19, 2024
2698382
Merge branch 'master' into feature/forkless-state-transition
xgreenx Mar 20, 2024
a0d99e1
Addressed comments. Make clippy happy
xgreenx Mar 21, 2024
95128e3
Rerun for any changes
xgreenx Mar 21, 2024
8ce8ac0
Up Rust in CI to 1.77
xgreenx Mar 21, 2024
09d3d96
Merge branch 'master' into feature/forkless-state-transition
xgreenx Mar 22, 2024
6328d2d
Revert "Up Rust in CI to 1.77"
xgreenx Mar 22, 2024
06d1826
Addressed comments in the PR
xgreenx Mar 22, 2024
072ec0e
Merge remote-tracking branch 'origin/feature/forkless-state-transitio…
xgreenx Mar 22, 2024
a53dd24
Fixed compilation
xgreenx Mar 22, 2024
325afec
Merge branch 'master' into feature/forkless-state-transition
Voxelot Mar 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Self review
xgreenx committed Mar 14, 2024
commit 15b921ae738a26ccd435c2641dcbd4f722c84d4d
1 change: 0 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions crates/fuel-core/src/p2p_test_helpers.rs
Original file line number Diff line number Diff line change
@@ -378,11 +378,11 @@ pub fn make_config(name: String, mut node_config: Config) -> Config {
pub async fn make_node(node_config: Config, test_txs: Vec<Transaction>) -> Node {
let db = Database::in_memory();
let node = tokio::time::timeout(
Duration::from_secs(30),
Duration::from_secs(2),
FuelService::from_database(db.clone(), node_config),
)
.await
.expect("All services should start in less than 30 seconds")
.expect("All services should start in less than 2 seconds")
.expect("The `FuelService should start without error");

let config = node.shared.config.clone();
14 changes: 10 additions & 4 deletions crates/services/upgradable-executor/Cargo.toml
Original file line number Diff line number Diff line change
@@ -11,13 +11,13 @@ description = "Fuel Block Upgradable Executor"
build = "build.rs"

[dependencies]
anyhow = { workspace = true }
anyhow = { workspace = true, optional = true }
fuel-core-executor = { workspace = true }
fuel-core-storage = { workspace = true }
fuel-core-types = { workspace = true }
fuel-core-wasm-executor = { workspace = true, optional = true }
lazy_static = { version = "1.4" }
postcard = { workspace = true }
lazy_static = { version = "1.4", optional = true }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
lazy_static = { version = "1.4", optional = true }
lazy_static = { workspace = true, optional = true }

postcard = { workspace = true, optional = true }
wasmtime = { version = "18.0.1", default-features = false, features = [
"cache",
"cranelift",
@@ -33,7 +33,13 @@ fuel-core-types = { workspace = true, features = ["test-helpers"] }
[features]
default = ["std"]
std = []
wasm-executor = ["dep:fuel-core-wasm-executor", "dep:wasmtime"]
wasm-executor = [
"dep:anyhow",
"dep:lazy_static",
"dep:postcard",
"dep:fuel-core-wasm-executor",
"dep:wasmtime",
]
test-helpers = [
"fuel-core-storage/test-helpers",
"fuel-core-types/test-helpers",
79 changes: 0 additions & 79 deletions crates/services/upgradable-executor/src/executor.rs
Original file line number Diff line number Diff line change
@@ -269,82 +269,3 @@ where
instance.execute_without_commit(block)
}
}

#[cfg(test)]
mod tests {
use super::*;
use fuel_core_storage::{
structured_storage::test::InMemoryStorage,
Result as StorageResult,
};
use fuel_core_types::{
blockchain::block::Block,
fuel_types::BlockHeight,
services::{
executor::{
Error as ExecutorError,
ExecutionTypes,
},
relayer::Event,
},
};

struct Storage;

impl AtomicView for Storage {
type View = InMemoryStorage<Column>;
type Height = BlockHeight;

fn latest_height(&self) -> Option<Self::Height> {
Some(0u32.into())
}

fn view_at(&self, _: &Self::Height) -> StorageResult<Self::View> {
Ok(self.latest_view())
}

fn latest_view(&self) -> Self::View {
InMemoryStorage::<Column>::default()
}
}

struct DisabledRelayer;

impl AtomicView for DisabledRelayer {
type View = DisabledRelayer;
type Height = DaBlockHeight;

fn latest_height(&self) -> Option<Self::Height> {
unimplemented!()
}

fn view_at(&self, _: &Self::Height) -> StorageResult<Self::View> {
unimplemented!()
}

fn latest_view(&self) -> Self::View {
DisabledRelayer
}
}

impl RelayerPort for DisabledRelayer {
fn enabled(&self) -> bool {
false
}

fn get_events(&self, _: &DaBlockHeight) -> anyhow::Result<Vec<Event>> {
unimplemented!()
}
}

#[test]
fn test_executor() {
let executor = Executor::new(Storage, DisabledRelayer, Default::default());
let result =
executor.execute_without_commit(ExecutionTypes::Validation(Block::default()));
assert_eq!(
result.expect_err("Should be error"),
ExecutorError::MintMissing
);
}
}
4 changes: 4 additions & 0 deletions crates/services/upgradable-executor/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
#![deny(clippy::arithmetic_side_effects)]
#![deny(clippy::cast_possible_truncation)]
#![deny(unused_crate_dependencies)]

#[cfg(feature = "wasm-executor")]
use wasmtime::{
Config,
Original file line number Diff line number Diff line change
@@ -22,7 +22,6 @@ fuel-core-executor = { workspace = true, default-features = false }
fuel-core-storage = { workspace = true, default-features = false }
fuel-core-types = { workspace = true, default-features = false }
postcard = { workspace = true }
serde = { workspace = true }

[features]
default = ["std"]
16 changes: 10 additions & 6 deletions crates/services/upgradable-executor/wasm-executor/src/ext.rs
Original file line number Diff line number Diff line change
@@ -142,7 +142,7 @@ pub fn input_component(size: usize) -> anyhow::Result<InputType> {
unsafe {
host::input_component(
Ptr32Mut::from_slice(encoded_block.as_mut_slice()),
size as u32,
u32::try_from(size).expect("We only support wasm32 target; qed"),
)
};

@@ -158,7 +158,7 @@ pub fn input_options(size: usize) -> anyhow::Result<ExecutionOptions> {
unsafe {
host::input_options(
Ptr32Mut::from_slice(encoded_block.as_mut_slice()),
size as u32,
u32::try_from(size).expect("We only support wasm32 target; qed"),
)
};

@@ -182,7 +182,7 @@ pub fn next_transactions(gas_limit: u64) -> anyhow::Result<Vec<MaybeCheckedTrans
unsafe {
host::consume_next_txs(
Ptr32Mut::from_slice(encoded_block.as_mut_slice()),
size as u32,
u32::try_from(size).expect("We only support wasm32 target; qed"),
)
};

@@ -197,7 +197,11 @@ pub fn next_transactions(gas_limit: u64) -> anyhow::Result<Vec<MaybeCheckedTrans

pub fn size_of_value(key: &[u8], column: u32) -> anyhow::Result<Option<usize>> {
let val = unsafe {
host::storage_size_of_value(Ptr32::from_slice(key), key.len() as u32, column)
host::storage_size_of_value(
Ptr32::from_slice(key),
u32::try_from(key.len()).expect("We only support wasm32 target; qed"),
column,
)
};

let (exists, size, result) = unpack_exists_size_result(val);
@@ -220,10 +224,10 @@ pub fn get(key: &[u8], column: u32, out: &mut [u8]) -> anyhow::Result<()> {
let result = unsafe {
host::storage_get(
Ptr32::from_slice(key),
key.len() as u32,
u32::try_from(key.len()).expect("We only support wasm32 target; qed"),
column,
Ptr32Mut::from_slice(out),
output_size as u32,
u32::try_from(output_size).expect("We only support wasm32 target; qed"),
)
};

Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
#![deny(clippy::arithmetic_side_effects)]
#![deny(clippy::cast_possible_truncation)]

pub use fuel_core_executor;
pub mod utils;
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
#![deny(clippy::arithmetic_side_effects)]
#![deny(clippy::cast_possible_truncation)]
#![deny(unused_crate_dependencies)]

use crate::{
relayer::WasmRelayer,
storage::WasmStorage,
@@ -19,6 +23,7 @@ use fuel_core_types::services::{
},
Uncommitted,
};
use fuel_core_wasm_executor as _;

mod ext;
mod relayer;
@@ -31,7 +36,10 @@ pub extern "C" fn execute(component_len: u32, options_len: u32) -> u64 {
let result = execute_without_commit(component_len, options_len);
let encoded = postcard::to_allocvec(&result).expect("Failed to encode the result");
let static_slice = encoded.leak();
pack_ptr_and_len(static_slice.as_ptr() as u32, static_slice.len() as u32)
pack_ptr_and_len(
static_slice.as_ptr() as u32,
u32::try_from(static_slice.len()).expect("We only support wasm32 target; qed"),
)
}

pub fn execute_without_commit(component_len: u32, options_len: u32) -> ReturnType {
11 changes: 7 additions & 4 deletions crates/services/upgradable-executor/wasm-executor/src/utils.rs
Original file line number Diff line number Diff line change
@@ -15,8 +15,9 @@ pub fn pack_ptr_and_len(ptr: u32, len: u32) -> u64 {

/// Unpacks an `u64` into the pointer and length.
pub fn unpack_ptr_and_len(val: u64) -> (u32, u32) {
let ptr = (val & (u32::MAX as u64)) as u32;
let len = (val >> 32) as u32;
let ptr = u32::try_from(val & (u32::MAX as u64))
.expect("It ony contains first 32 bytes; qed");
let len = u32::try_from(val >> 32).expect("It ony contains first 32 bytes; qed");

(ptr, len)
}
@@ -29,8 +30,10 @@ pub fn pack_exists_size_result(exists: bool, size: u32, result: u16) -> u64 {
/// Unpacks an `u64` into `exists`, `size` and `result`.
pub fn unpack_exists_size_result(val: u64) -> (bool, u32, u16) {
let exists = (val & 1u64) != 0;
let size = ((val >> 1) & (u32::MAX as u64)) as u32;
let result = (val >> 33) as u16;
let size = u32::try_from((val >> 1) & (u32::MAX as u64))
.expect("It only contains first 32 bytes; qed");
let result = u16::try_from(val >> 33 & (u16::MAX as u64))
.expect("It only contains first 16 bytes; qed");

(exists, size, result)
}
1 change: 0 additions & 1 deletion crates/storage/src/lib.rs
Original file line number Diff line number Diff line change
@@ -4,7 +4,6 @@
//! defined here are used by services but are flexible enough to customize the
//! logic when the `Database` is known.

// #![cfg_attr(not(feature = "std"), no_std)]
#![deny(clippy::arithmetic_side_effects)]
#![deny(clippy::cast_possible_truncation)]
#![deny(unused_crate_dependencies)]