Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rpc state implementation #618

Merged
merged 109 commits into from
Aug 19, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
c1d83d8
implemented state calls
StaticallyTypedAnxiety Jun 22, 2020
5b658e8
code complete
StaticallyTypedAnxiety Jun 30, 2020
46f0370
Merge branch 'master' into ashanti/state_api_rpc_methods
StaticallyTypedAnxiety Jun 30, 2020
c1a0310
state changes and fixes
StaticallyTypedAnxiety Jul 2, 2020
2ddf7c6
added test changes
StaticallyTypedAnxiety Jul 2, 2020
9f63d27
Merge branch 'main' into ashanti/state_api_rpc_methods
StaticallyTypedAnxiety Jul 2, 2020
78276f2
removed unused imports
StaticallyTypedAnxiety Jul 2, 2020
36d8d6d
fixed merge conflicts
StaticallyTypedAnxiety Jul 2, 2020
7c641ce
state manager changes
StaticallyTypedAnxiety Jul 2, 2020
fe9dd6e
Merge branch 'main' into ashanti/state_api_rpc_methods
StaticallyTypedAnxiety Jul 3, 2020
c0bbf33
added licenses
StaticallyTypedAnxiety Jul 3, 2020
06ea6fe
Merge branch 'ashanti/state_api_rpc_methods' of https://github.com/ch…
StaticallyTypedAnxiety Jul 3, 2020
9b8d656
applied cargo update
StaticallyTypedAnxiety Jul 6, 2020
ae87f0c
removed halt
StaticallyTypedAnxiety Jul 6, 2020
e9ae665
clean ups
StaticallyTypedAnxiety Jul 6, 2020
1bf85dc
comment added
StaticallyTypedAnxiety Jul 6, 2020
aca7b55
upgrade submodule
StaticallyTypedAnxiety Jul 6, 2020
049b0c7
Update blockchain/chain/src/store/chain_store.rs
StaticallyTypedAnxiety Jul 13, 2020
cec57e2
Update blockchain/chain/src/store/chain_store.rs
StaticallyTypedAnxiety Jul 13, 2020
a71bcdd
Update blockchain/chain/src/store/chain_store.rs
StaticallyTypedAnxiety Jul 13, 2020
c56d74c
Update blockchain/chain/src/store/chain_store.rs
StaticallyTypedAnxiety Jul 13, 2020
e4b0327
Update blockchain/chain/src/store/chain_store.rs
StaticallyTypedAnxiety Jul 13, 2020
0d1eec0
Update blockchain/chain/src/store/chain_store.rs
StaticallyTypedAnxiety Jul 13, 2020
9f9396b
Update blockchain/chain/src/store/chain_store.rs
StaticallyTypedAnxiety Jul 13, 2020
a79ef24
changes to pr
StaticallyTypedAnxiety Jul 13, 2020
f2db9d5
fixed merge conflicts
StaticallyTypedAnxiety Jul 13, 2020
884958f
applied cargo fmt
StaticallyTypedAnxiety Jul 13, 2020
4305f23
added more rustdoc comments
StaticallyTypedAnxiety Jul 13, 2020
b9b826d
Update blockchain/chain/src/store/chain_store.rs
StaticallyTypedAnxiety Jul 15, 2020
7698dea
Update node/state_api/src/lib.rs
StaticallyTypedAnxiety Jul 15, 2020
43c874a
Update blockchain/state_manager/src/call.rs
StaticallyTypedAnxiety Jul 15, 2020
7e73b95
json changes
StaticallyTypedAnxiety Jul 16, 2020
16f9481
pr changes
StaticallyTypedAnxiety Jul 16, 2020
60af581
Merge branch 'ashanti/state_api_rpc_methods' of https://github.com/Ch…
StaticallyTypedAnxiety Jul 16, 2020
75964b0
applied merge conflict changes
StaticallyTypedAnxiety Jul 16, 2020
dcc900a
cargo fmt
StaticallyTypedAnxiety Jul 16, 2020
36d0f80
fixed tests
StaticallyTypedAnxiety Jul 16, 2020
035b354
removed unneeded import
StaticallyTypedAnxiety Jul 16, 2020
4573c3d
Merge branch 'main' into ashanti/state_api_rpc_methods
StaticallyTypedAnxiety Jul 17, 2020
61e60b7
Merge branch 'main' into ashanti/state_api_rpc_methods
StaticallyTypedAnxiety Jul 17, 2020
251a820
Update blockchain/state_manager/src/lib.rs
StaticallyTypedAnxiety Jul 17, 2020
e746b7a
Update blockchain/state_manager/src/lib.rs
StaticallyTypedAnxiety Jul 17, 2020
8f6b963
Update blockchain/state_manager/src/lib.rs
StaticallyTypedAnxiety Jul 17, 2020
ed2f69e
Update blockchain/chain/src/store/chain_store.rs
StaticallyTypedAnxiety Jul 17, 2020
28fbfef
changes based on PR
StaticallyTypedAnxiety Jul 17, 2020
17d56fa
Merge branch 'ashanti/state_api_rpc_methods' of https://github.com/Ch…
StaticallyTypedAnxiety Jul 17, 2020
f7fbcba
changes to invoc_result
StaticallyTypedAnxiety Jul 17, 2020
d87ddef
state_api changes
StaticallyTypedAnxiety Jul 17, 2020
3890f03
removed for loop
StaticallyTypedAnxiety Jul 17, 2020
639829e
fixed merge confict
StaticallyTypedAnxiety Jul 17, 2020
f106d23
Merge branch 'ashanti/state_api_rpc_methods' into rpc_state_implement…
StaticallyTypedAnxiety Jul 20, 2020
91b36c4
Merge branch 'main' into ashanti/state_api_rpc_methods
StaticallyTypedAnxiety Jul 20, 2020
dc50d23
Update vm/message/src/lib.rs
StaticallyTypedAnxiety Jul 21, 2020
c4ea7a0
Update blockchain/state_manager/src/lib.rs
StaticallyTypedAnxiety Jul 21, 2020
76dd011
Update vm/message/src/lib.rs
StaticallyTypedAnxiety Jul 21, 2020
de424cd
Added fixes and types
StaticallyTypedAnxiety Jul 21, 2020
fd6a0d3
Update blockchain/state_manager/src/lib.rs
StaticallyTypedAnxiety Jul 21, 2020
5dd6269
chain message introduced
StaticallyTypedAnxiety Jul 21, 2020
61741b0
added comment
StaticallyTypedAnxiety Jul 21, 2020
cdbc29a
Merge branch 'ashanti/state_api_rpc_methods' of https://github.com/Ch…
StaticallyTypedAnxiety Jul 21, 2020
00021c5
cargo fmt
StaticallyTypedAnxiety Jul 21, 2020
708b012
moved some comments
StaticallyTypedAnxiety Jul 21, 2020
d25df74
added license
StaticallyTypedAnxiety Jul 21, 2020
3dba797
changed error messages
StaticallyTypedAnxiety Jul 21, 2020
9dcb457
changed wait_for_msg algorithmn
StaticallyTypedAnxiety Jul 22, 2020
ee0bcdc
merge conflict fixed
StaticallyTypedAnxiety Jul 22, 2020
8186b29
Merge branch 'ashanti/state_api_rpc_methods' into rpc_state_implement…
StaticallyTypedAnxiety Jul 22, 2020
8b2eb89
refactor in progress
StaticallyTypedAnxiety Jul 23, 2020
f69db09
refactored to use block_store instead
StaticallyTypedAnxiety Jul 23, 2020
864bc3b
removed fluff
StaticallyTypedAnxiety Jul 23, 2020
1b6fe24
fixed error
StaticallyTypedAnxiety Jul 23, 2020
4f8053a
Merge branch 'ashanti/state_api_rpc_methods' into rpc_state_implement…
StaticallyTypedAnxiety Jul 24, 2020
e15074b
refactor in progress
StaticallyTypedAnxiety Jul 28, 2020
79090ab
Merge branch 'main' into rpc_state_implementation
StaticallyTypedAnxiety Jul 28, 2020
a78015a
more changes
StaticallyTypedAnxiety Aug 9, 2020
55981aa
changes made
StaticallyTypedAnxiety Aug 10, 2020
290c173
changed to take in statemanager
StaticallyTypedAnxiety Aug 10, 2020
adc8be1
cargo fmt
StaticallyTypedAnxiety Aug 10, 2020
18f7b91
serialization changes
StaticallyTypedAnxiety Aug 11, 2020
cc00bde
Merged main and cargo fmt
StaticallyTypedAnxiety Aug 11, 2020
d515522
Delete randomness.rs.orig
StaticallyTypedAnxiety Aug 11, 2020
7b6fd9b
minor fixes
StaticallyTypedAnxiety Aug 11, 2020
3edeae5
Merge branch 'rpc_state_implementation' of https://github.com/ChainSa…
StaticallyTypedAnxiety Aug 11, 2020
d11938d
deleted files
StaticallyTypedAnxiety Aug 11, 2020
af92d39
Delete chain_store.rs.orig
StaticallyTypedAnxiety Aug 11, 2020
52dc280
add license headers
StaticallyTypedAnxiety Aug 11, 2020
02c1e89
removed type file
StaticallyTypedAnxiety Aug 11, 2020
f5f2762
Merge branch 'rpc_state_implementation' of https://github.com/ChainSa…
StaticallyTypedAnxiety Aug 11, 2020
2395f9d
Merge branch 'main' into rpc_state_implementation
StaticallyTypedAnxiety Aug 11, 2020
c0e2f35
clippy fixes
StaticallyTypedAnxiety Aug 11, 2020
daa894a
Merge branch 'rpc_state_implementation' of https://github.com/ChainSa…
StaticallyTypedAnxiety Aug 11, 2020
d860bdf
fixed tests
StaticallyTypedAnxiety Aug 11, 2020
06be1e2
Update node/rpc/Cargo.toml
StaticallyTypedAnxiety Aug 12, 2020
9c92a15
Update node/rpc/src/lib.rs
StaticallyTypedAnxiety Aug 12, 2020
2b6ea70
Update vm/address/Cargo.toml
StaticallyTypedAnxiety Aug 12, 2020
ee93ca5
made changes to name and serialization
StaticallyTypedAnxiety Aug 12, 2020
2d96eef
Merge branch 'rpc_state_implementation' of https://github.com/ChainSa…
StaticallyTypedAnxiety Aug 12, 2020
90064eb
changes to use tipset_from_keys function
StaticallyTypedAnxiety Aug 12, 2020
6a206ce
BitField Serialization Fixed
StaticallyTypedAnxiety Aug 13, 2020
85f3e52
changes to PR
StaticallyTypedAnxiety Aug 13, 2020
8d8977d
bumped address version
StaticallyTypedAnxiety Aug 13, 2020
6eb1408
Update vm/address/Cargo.toml
StaticallyTypedAnxiety Aug 17, 2020
0b99184
pascal case serialize/deserialize
StaticallyTypedAnxiety Aug 17, 2020
fbce258
Merge branch 'rpc_state_implementation' of https://github.com/ChainSa…
StaticallyTypedAnxiety Aug 17, 2020
a7c7cc3
Update utils/bitfield/src/lib.rs
StaticallyTypedAnxiety Aug 18, 2020
8d913ea
Update utils/bitfield/src/lib.rs
StaticallyTypedAnxiety Aug 18, 2020
6899663
Update utils/bitfield/src/lib.rs
StaticallyTypedAnxiety Aug 18, 2020
94dd2d6
Made cleanups on PR
StaticallyTypedAnxiety Aug 18, 2020
9452c8b
Merge branch 'main' into rpc_state_implementation
StaticallyTypedAnxiety Aug 18, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 11 additions & 26 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ members = [
"vm/interpreter",
"node/clock",
"node/rpc",
"node/state_api",
"crypto",
"encoding",
"ipld/cid",
Expand Down
6 changes: 6 additions & 0 deletions blockchain/blocks/src/tipset.rs
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,12 @@ pub mod tipset_json {
}
}

impl From<Tipset> for TipsetJson {
fn from(wrapper: Tipset) -> Self {
TipsetJson(wrapper)
}
}

pub fn serialize<S>(m: &Tipset, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
Expand Down
129 changes: 63 additions & 66 deletions blockchain/state_manager/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ use interpreter::{resolve_to_key_addr, ApplyRet, ChainRand, DefaultSyscalls, VM}
use ipld_amt::Amt;
use log::{trace, warn};
use message::{Message, MessageReceipt, UnsignedMessage};
use num_bigint::BigInt;
use num_bigint::{bigint_ser, BigInt};
use serde::{Deserialize, Serialize};
use state_tree::StateTree;
use std::collections::HashMap;
use std::error::Error as StdError;
Expand All @@ -37,22 +38,23 @@ use std::sync::Arc;
pub type CidPair = (Cid, Cid);

/// Type to represent invocation of state call results
pub struct InvocResult<Msg>
where
Msg: Message,
{
pub msg: Msg,
#[derive(Serialize, Deserialize)]
#[serde(rename_all = "PascalCase")]
pub struct InvocResult {
pub msg: UnsignedMessage,
pub msg_rct: Option<MessageReceipt>,
pub actor_error: Option<String>,
pub error: Option<String>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You didn't have to change this one, but doesn't matter

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made sense if wanted to be consistent with lotus

}

// An alias Result that represents an InvocResult and an Error
pub type StateCallResult<T> = Result<InvocResult<T>, Error>;
pub type StateCallResult = Result<InvocResult, Error>;

#[allow(dead_code)]
#[derive(Default)]
#[derive(Default, Serialize, Deserialize)]
#[serde(rename_all = "PascalCase")]
pub struct MarketBalance {
#[serde(with = "bigint_ser")]
escrow: BigInt,
#[serde(with = "bigint_ser")]
locked: BigInt,
}

Expand Down Expand Up @@ -233,7 +235,7 @@ where
bstate: &Cid,
rand: &ChainRand,
bheight: &ChainEpoch,
) -> StateCallResult<UnsignedMessage>
) -> StateCallResult
where
DB: BlockStore,
{
Expand Down Expand Up @@ -270,17 +272,13 @@ where
Ok(InvocResult {
msg: msg.clone(),
msg_rct: Some(apply_ret.msg_receipt.clone()),
actor_error: apply_ret.act_error.map(|e| e.to_string()),
error: apply_ret.act_error.map(|e| e.to_string()),
})
})
}

/// runs the given message and returns its result without any persisted changes.
pub fn call(
&self,
message: &mut UnsignedMessage,
tipset: Option<Tipset>,
) -> StateCallResult<UnsignedMessage>
pub fn call(&self, message: &mut UnsignedMessage, tipset: Option<Tipset>) -> StateCallResult
where
DB: BlockStore,
{
Expand Down Expand Up @@ -375,6 +373,53 @@ where
})
}

fn tipset_executed_message(
block_store: &DB,
tipset: &Tipset,
cid: &Cid,
(message_from_address, message_sequence): (&Address, &u64),
ec2 marked this conversation as resolved.
Show resolved Hide resolved
) -> Result<Option<MessageReceipt>, Error>
where
DB: BlockStore,
{
if tipset.epoch() == 0 {
return Ok(None);
}
let tipset = chain::tipset_from_keys(block_store, tipset.parents())
.map_err(|err| Error::Other(err.to_string()))?;
let messages = chain::messages_for_tipset(block_store, &tipset)
.map_err(|err| Error::Other(err.to_string()))?;
messages
.iter()
.enumerate()
.rev()
austinabell marked this conversation as resolved.
Show resolved Hide resolved
.filter(|(_, s)| s.from() == message_from_address)
.filter_map(|(index,s)| {
if s.sequence() == *message_sequence {
if s.cid().map(|s| &s == cid).unwrap_or_default() {
return Some(
chain::get_parent_reciept(
block_store,
tipset.blocks().first().unwrap(),
index as u64,
)
.map_err(|err| {
Error::Other(err.to_string())
}),
);
}
let error_msg = format!("found message with equal nonce as the one we are looking for (F:{:} n {:}, TS: `Error Converting message to Cid` n{:})", cid, message_sequence, s.sequence());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can break this up with adding a \ in the string to break the lines up (line width isn't that important, so very optional)

return Some(Err(Error::Other(error_msg)))
}
if s.sequence() < *message_sequence {
return Some(Ok(None));
}

None
})
.next()
.unwrap_or_else(|| Ok(None))
austinabell marked this conversation as resolved.
Show resolved Hide resolved
}
fn search_back_for_message(
block_store: Arc<DB>,
current: &Tipset,
Expand Down Expand Up @@ -445,54 +490,6 @@ where
Ok(message_receipt)
}

fn tipset_executed_message(
block_store: &DB,
tipset: &Tipset,
cid: &Cid,
(message_from_address, message_sequence): (&Address, &u64),
) -> Result<Option<MessageReceipt>, Error>
where
DB: BlockStore,
{
if tipset.epoch() == 0 {
return Ok(None);
}
let tipset = chain::tipset_from_keys(block_store, tipset.parents())
.map_err(|err| Error::Other(err.to_string()))?;
let messages = chain::messages_for_tipset(block_store, &tipset)
.map_err(|err| Error::Other(err.to_string()))?;
messages
.iter()
.enumerate()
.rev()
.filter(|(_, s)| s.from() == message_from_address)
.filter_map(|(index,s)| {
if s.sequence() == *message_sequence {
if s.cid().map(|s| &s == cid).unwrap_or_default() {
return Some(
chain::get_parent_reciept(
block_store,
tipset.blocks().first().unwrap(),
index as u64,
)
.map_err(|err| {
Error::Other(err.to_string())
}),
);
}
let error_msg = format!("found message with equal nonce as the one we are looking for (F:{:} n {:}, TS: `Error Converting message to Cid` n{:})", cid, message_sequence, s.sequence());
return Some(Err(Error::Other(error_msg)))
}
if s.sequence() < *message_sequence {
return Some(Ok(None));
}

None
})
.next()
.unwrap_or_else(|| Ok(None))
}

/// WaitForMessage blocks until a message appears on chain. It looks backwards in the chain to see if this has already
/// happened. It guarantees that the message has been on chain for at least confidence epochs without being reverted
/// before returning.
Expand Down Expand Up @@ -679,7 +676,7 @@ where
state_tree.lookup_id(addr).map_err(Error::State)
}

pub fn market_balance(&mut self, addr: &Address, ts: &Tipset) -> Result<MarketBalance, Error> {
pub fn market_balance(&self, addr: &Address, ts: &Tipset) -> Result<MarketBalance, Error> {
let market_state: market::State =
self.load_actor_state(&*STORAGE_MARKET_ACTOR_ADDR, ts.parent_state())?;

Expand Down
5 changes: 3 additions & 2 deletions forest/src/daemon.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ use libp2p::identity::{ed25519, Keypair};
use log::{debug, info, trace};
use message_pool::{MessagePool, MpoolRpcProvider};
use rpc::{start_rpc, RpcState};
use state_manager::StateManager;
use std::sync::Arc;
use utils::write_to_file;
use wallet::PersistentKeyStore;
Expand Down Expand Up @@ -96,14 +97,14 @@ pub(super) async fn start(config: Config) {
});

let rpc_task = if config.enable_rpc {
let db_rpc = Arc::clone(&db);
let db_rpc = StateManager::new(Arc::clone(&db));
let keystore_rpc = Arc::clone(&keystore);
let rpc_listen = format!("127.0.0.1:{}", &config.rpc_port);
Some(task::spawn(async move {
info!("JSON RPC Endpoint at {}", &rpc_listen);
start_rpc(
RpcState {
store: db_rpc,
state_manager: db_rpc,
keystore: keystore_rpc,
mpool,
bad_blocks,
Expand Down
8 changes: 6 additions & 2 deletions node/rpc/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ authors = ["ChainSafe Systems <info@chainsafe.io>"]
edition = "2018"

[dependencies]
actor = { path = "../../vm/actor/" }
async-std = { version = "1.6.0", features = ["attributes"] }
tide = "0.9.0"
serde = { version = "1.0.101", default-features = false, features = ["derive"] }
Expand All @@ -22,15 +23,18 @@ crypto = { package = "forest_crypto", path = "../../crypto", features = ["json"]
num-traits = "0.2.11"
wallet = {package = "key_management", path = "../../key_management", features = ["json"] }
state_manager = {package = "state_manager", path = "../../blockchain/state_manager" }
address = { package = "forest_address", path = "../../vm/address" }
address = { package = "forest_address", path = "../../vm/address", features = ["json"] }
encoding = { package = "forest_encoding", path = "../../encoding/"}
num-bigint = { path = "../../utils/bigint", package = "forest_bigint" }
thiserror = "1.0"
state_tree = { path = "../../vm/state_tree" }
forest_libp2p = { path = "../forest_libp2p" }
interpreter = { path = "../../vm/interpreter/" }
fil_types = { path = "../../types" }
bitfield = { path = "../../utils/bitfield",features = ["json"] }

[dev-dependencies]
db = { path = "../db" }
futures = "0.3.5"
test_utils = { version = "0.1.0", path = "../../utils/test_utils/", features = ["test_constructors"] }
hex = "0.4.2"
hex = "0.4.2"
Loading