no-std implementation #167

dharjeezy · 2022-12-27T21:38:48Z

Implement alloc crate
Implement hashbrown crate
Made use of milagro crate for the BLS scheme.

closes #166

started implementing display for errors

Wizdave97 · 2022-12-28T21:38:23Z

Looking good.

builds successfully

Cargo.toml

wasm compilation successful

Cargo.toml

src/clock.rs

src/crypto.rs

src/altair/block_processing.rs

dharjeezy · 2023-01-09T12:49:43Z

@ralexstokes can you please, help review this. Thanks

ralexstokes

sorry for the delay on this and thanks for opening the PR!

there are a few high-level things to note before moving further:

use of blst or some alternative
note that you edited some code that falls under the spec code gen you can read about here: https://github.com/ralexstokes/ethereum-consensus#notes and the point is that we shouldn't be edited those files manually

otherwise I left some comments on the PR

ralexstokes · 2023-01-19T02:06:32Z

src/altair/block_processing.rs

@@ -202,8 +201,9 @@ pub fn process_deposit<

 let public_key = &deposit.data.public_key;
 let amount = deposit.data.amount;
+ let cloned_validators = state.validators.clone();


can we avoid a full clone here?

For this, if i don't clone, mutable borrow occurs when i try to iterate over the validators and put them into a HashSet

ralexstokes · 2023-01-19T02:06:52Z

src/altair/block_processing.rs

@@ -315,9 +315,9 @@ pub fn process_sync_aggregate<
 let proposer_reward =
 participant_reward * PROPOSER_WEIGHT / (WEIGHT_DENOMINATOR - PROPOSER_WEIGHT);

+ let cloned_validators = state.validators.clone();


same -- why do we need to change this in the first place?

This is due to the iteration and collecting into the Hashbrown's Hashset crate

src/altair/state_transition/block_processing.rs

ralexstokes · 2023-01-19T02:08:21Z

src/altair/state_transition/helpers.rs

+ #[cfg(not(feature = "serde"))]
+ let i_bytes: [u8; 4] = (i.saturating_div(32)).to_le_bytes();
+
+ #[cfg(feature = "serde")]
+ let i_bytes: [u8; 8] = (i.saturating_div(32)).to_le_bytes();
+


what is happening here?

So, I am trying to make this repo compile to target --target=wasm32-unknown-unknown
I get this error when i try to compile this section of code to that target

let i_bytes: [u8; 8] = (i/32).to_le_bytes(); | ------- ^^^^^^^^^^^^^^^^^^^^ expected an array with a fixed size of 8 elements, found one with 4 elements | | | expected due to this

Hence the reason why i used the feature tag to separate the std and no-std implementation

ralexstokes · 2023-01-19T02:10:45Z

src/crypto.rs

 use crate::primitives::Bytes32;
 #[cfg(feature = "serde")]
 use crate::serde::{try_bytes_from_hex_str, HexError};
 use crate::ssz::ByteVector;
-use blst::{min_pk as bls_impl, BLST_ERROR};
+use error_chain::ChainedError;
+use milagro_bls::{AggregatePublicKey, AggregateSignature, AmclError};


this may be its own sidequest but I would strongly prefer to use blst

do you know why it is not no-std compatible?

According to the blst crate, it is not no-std compatible hence the usage of milagro_bls crate which is no-std compatible

blst crate is only supports compilation to wasm32-unknown-emscripten because of it's dependence on some FFIs, but wasm32-unknown-unknown is the target we'll need in the future.
More details here rustwasm/team#291

ralexstokes · 2023-01-19T02:12:09Z

src/phase0/block_processing.rs

@@ -364,8 +364,9 @@ pub fn process_deposit<

 let public_key = &deposit.data.public_key;
 let amount = deposit.data.amount;
+ let cloned_validators = state.validators.clone();


same question here

For this, if i don't clone, mutable borrow occurs when i try to iterate over the validators and put them into a HashSet

ralexstokes · 2023-01-19T02:12:55Z

src/phase0/mod.rs

-pub const DEPOSIT_CONTRACT_TREE_DEPTH: usize = 2usize.pow(5);
+pub const DEPOSIT_CONTRACT_TREE_DEPTH: usize = 2usize.saturating_pow(5);


seems unnecessary? if there is a compatibility issue with pow then we can just write 32usize here

ralexstokes · 2023-01-19T02:13:28Z

src/phase0/mod.rs

-pub const DEPOSIT_DATA_LIST_BOUND: usize = 2usize.pow(DEPOSIT_CONTRACT_TREE_DEPTH as u32);
+pub const DEPOSIT_DATA_LIST_BOUND: usize =
+ 2usize.saturating_pow(DEPOSIT_CONTRACT_TREE_DEPTH as u32);


if we can't just use pow then we can just write out the integer as usize

Yh! so this is due to the compilation target wasm32-unknown-unknown
But if i do 2^32 which is 4294967296, I am going to end up with an overflowing literal

ralexstokes · 2023-01-19T02:14:13Z

src/phase0/presets/mainnet.rs

-pub const VALIDATOR_REGISTRY_LIMIT: usize = 2usize.pow(40);
+pub const VALIDATOR_REGISTRY_LIMIT: usize = 2usize.saturating_pow(40);
 pub const BASE_REWARD_FACTOR: u64 = 64;
 pub const WHISTLEBLOWER_REWARD_QUOTIENT: u64 = 512;
 pub const PROPOSER_REWARD_QUOTIENT: u64 = 8;
-pub const INACTIVITY_PENALTY_QUOTIENT: u64 = 2u64.pow(26);
+pub const INACTIVITY_PENALTY_QUOTIENT: u64 = 2u64.saturating_pow(26);


not sure why we changed these

Yh! so this is due to the compilation target wasm32-unknown-unknown

ralexstokes · 2023-01-19T02:15:25Z

src/prelude.rs

@@ -0,0 +1,47 @@
+mod core {


for consistency's sake, I'd prefer to follow the pattern from ralexstokes/ssz-rs#25

i.e. a mod lib with this inside and then we can use crate::lib::*; elsewhere

I feel this prelude looks cleaner

Gone ahead to do as adviced. @ralexstokes

use 32usize

…ensus into dami/no-std-support � Conflicts: � src/altair/state_transition/block_processing.rs � src/altair/state_transition/epoch_processing.rs � src/state_transition/error.rs

use lib for no-std importss

…ensus into dami/no-std-support � Conflicts: � Cargo.toml � src/altair/genesis.rs � src/altair/helpers.rs � src/altair/state_transition/block_processing.rs � src/altair/state_transition/epoch_processing.rs � src/bellatrix/genesis.rs � src/crypto.rs � src/lib.rs � src/phase0/genesis.rs

willemolding · 2023-02-24T05:55:08Z

src/altair/state_transition/helpers.rs

@@ -117,7 +117,12 @@ pub fn compute_proposer_index<
 loop {
 let shuffled_index = compute_shuffled_index(i % total, total, seed, context)?;
 let candidate_index = indices[shuffled_index];
+ #[cfg(not(feature = "serde"))]
+ let i_bytes: [u8; 4] = (i / 32).to_le_bytes();


I'm curious what is the purpose of this? It seems to be broken at the moment if the "serde" feature is not enabled

@dharjeezy why is there a serde feature flag here?

So, our target compilation is wasm32-unknown-unknown
I had to include the feature flag because of that else it errors out

@Wizdave97 @willemolding

Collecting into a dynamic type like a Vec might work better in this case.

Maybe better to use #[cfg(target_arch = "wasm32")] if it is a wasm specific thing? As it stands it errors out building for x86 with serde disabled

Oh wait an even better solution is to remove the intermediate variable entirely e.g.

hash_input[32..].copy_from_slice(&((i / 32) as u64).to_le_bytes());

This will build either way regardless of the size of a u64 on the given target

…ensus into dami/no-std-support � Conflicts: � Cargo.toml � src/bytes.rs � src/crypto.rs � src/serde.rs � src/ssz/byte_list.rs � src/ssz/byte_vector.rs � src/state_transition/error.rs

std related clock features

started no-std support

779a1e7

started implementing display for errors

dharjeezy marked this pull request as draft December 27, 2022 21:39

dharjeezy added 4 commits December 27, 2022 23:03

invalid operation error

3e66ff7

introduce prelude which contains the no_std imports

0b79485

transform all state_transition errors to no_std

7492dde

introduce BTreeSet and BTreeMap in prelude

8221123

dharjeezy added 2 commits December 28, 2022 23:56

further no_std imports in prelude

ac1b571

introduce hashbrown crate

cd52ded

builds successfully

Wizdave97 reviewed Dec 29, 2022

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

dharjeezy added 3 commits January 3, 2023 23:58

serde features on structs

0d05be0

make use of milagro_bls for no_std support

54f8133

wasm compilation successful

make ssz-rs default feature

eeb1d5e

Wizdave97 reviewed Jan 4, 2023

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

Wizdave97 reviewed Jan 4, 2023

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

Wizdave97 reviewed Jan 4, 2023

View reviewed changes

src/clock.rs Outdated Show resolved Hide resolved

Wizdave97 reviewed Jan 4, 2023

View reviewed changes

src/crypto.rs Show resolved Hide resolved

dharjeezy added 5 commits January 5, 2023 00:02

fix on comments

8849fcb

use signature for aggregate verify

2c0d3dd

use signature for aggregate verify

32883ab

introduce key validation failed

04f9bcc

build successful in std and no_std context

a6c3687

dharjeezy requested a review from Wizdave97 January 5, 2023 11:54

all test passes

1f0e6e2

dharjeezy marked this pull request as ready for review January 5, 2023 23:18

dharjeezy changed the title ~~started implementing no-std support~~ no-std implementation Jan 5, 2023

TomAFrench reviewed Jan 8, 2023

View reviewed changes

src/altair/block_processing.rs Outdated Show resolved Hide resolved

dharjeezy mentioned this pull request Jan 8, 2023

Create constants and data types polytope-labs/sync-committee-rs#2

Closed

remove duplication

71710e9

implement Eq and PartialEq traits for DomainType enum

df115f5

ralexstokes reviewed Jan 19, 2023

View reviewed changes

TomAFrench mentioned this pull request Jan 19, 2023

Correct typo in InvalidVoluntaryExit variant #174

Merged

dharjeezy added 4 commits January 19, 2023 11:30

remove saturating_div

7875009

use 32usize

Merge branch 'main' of https://github.com/polytope-labs/ethereum-cons…

33f236a

…ensus into dami/no-std-support � Conflicts: � src/altair/state_transition/block_processing.rs � src/altair/state_transition/epoch_processing.rs � src/state_transition/error.rs

merge changes

8c50522

fix warnings

e5c0d21

dharjeezy requested review from ralexstokes and removed request for Wizdave97 January 19, 2023 14:58

dharjeezy and others added 9 commits January 22, 2023 00:23

remove prelude.rs

2f46baf

use lib for no-std importss

remove excess clones

248b5d1

introduce default macro for execution payload

63b847d

remove default

9177be3

include default and alias

6b971e2

use rev

918fda3

no-std for capella fork

8be43ea

change ssz-rs repo temporarily

9862cab

Wizdave97 force-pushed the dami/no-std-support branch from 9862cab to 8be43ea Compare February 22, 2023 12:46

willemolding reviewed Feb 24, 2023

View reviewed changes

willemolding mentioned this pull request Feb 24, 2023

Fix building with no 'serde' feature #191

Closed

dharjeezy and others added 6 commits February 25, 2023 11:39

merge conflict fixed

6b7f33d

std related clock features

fix std and no std builds

b6ced43

extra fixes

23cd342

point ssz-rs to original crate

bd19d7c

point ssz-rs to polytope labs

34bb57f

ralexstokes mentioned this pull request Sep 30, 2023

remove cfg_attr for serde feature #273

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

no-std implementation #167

no-std implementation #167

dharjeezy commented Dec 27, 2022 •

edited

Loading

Wizdave97 commented Dec 28, 2022

dharjeezy commented Jan 9, 2023

ralexstokes left a comment

ralexstokes Jan 19, 2023

dharjeezy Jan 19, 2023

ralexstokes Jan 19, 2023

dharjeezy Jan 19, 2023

ralexstokes Jan 19, 2023

dharjeezy Jan 19, 2023 •

edited

Loading

ralexstokes Jan 19, 2023

dharjeezy Jan 19, 2023 •

edited

Loading

Wizdave97 Jan 24, 2023

ralexstokes Jan 19, 2023

dharjeezy Jan 19, 2023

ralexstokes Jan 19, 2023

ralexstokes Jan 19, 2023

dharjeezy Jan 19, 2023 •

edited

Loading

ralexstokes Jan 19, 2023

dharjeezy Jan 19, 2023 •

edited

Loading

ralexstokes Jan 19, 2023

dharjeezy Jan 19, 2023

dharjeezy Jan 22, 2023

willemolding Feb 24, 2023

Wizdave97 Feb 24, 2023

dharjeezy Feb 24, 2023

Wizdave97 Feb 24, 2023

willemolding Feb 25, 2023 •

edited

Loading

willemolding Feb 27, 2023

		pub const DEPOSIT_CONTRACT_TREE_DEPTH: usize = 2usize.pow(5);
		pub const DEPOSIT_CONTRACT_TREE_DEPTH: usize = 2usize.saturating_pow(5);

no-std implementation #167

Are you sure you want to change the base?

no-std implementation #167

Conversation

dharjeezy commented Dec 27, 2022 • edited Loading

Wizdave97 commented Dec 28, 2022

dharjeezy commented Jan 9, 2023

ralexstokes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dharjeezy Jan 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dharjeezy Jan 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dharjeezy Jan 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dharjeezy Jan 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

willemolding Feb 25, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dharjeezy commented Dec 27, 2022 •

edited

Loading

dharjeezy Jan 19, 2023 •

edited

Loading

dharjeezy Jan 19, 2023 •

edited

Loading

dharjeezy Jan 19, 2023 •

edited

Loading

dharjeezy Jan 19, 2023 •

edited

Loading

willemolding Feb 25, 2023 •

edited

Loading