Skip to content

Commit

Permalink
Update Miner actor (#691)
Browse files Browse the repository at this point in the history 
* WIP

* More changes

* Even more changes

* Amt docs

* Clippy & copyright

* Squashed commit of the following:

commit 8ef5ae5
Author: Austin Abell <austinabell8@gmail.com>
Date:   Wed Sep 16 12:30:01 2020 -0400

    Peer stats tracking and selection (#701)

    * Added peer manager logic and started to move blocksync handling into network context

    * Migrate to using handled blocksync requests and remove a duplicate code

    * Fix test use

    * Fix duration subtract logic

    * Update peer manager updates from hello requests

    * Docs and cleanup

    * more cleanup

commit dc0ff4c
Author: Eric Tu <6364934+ec2@users.noreply.github.com>
Date:   Tue Sep 15 14:43:18 2020 -0400

    Semantic Validation for Messages (#703)

    * semantic validation

    * suggestions

    * suggestions

commit 3411459
Author: Austin Abell <austinabell8@gmail.com>
Date:   Mon Sep 14 15:14:48 2020 -0400

    ChainSync refactor (#693)

    * Switch chain store to threadsafe

    * Update genesis to arc reference

    * wip refactoring chain sync to workers in async tasks

    * Update network event handling and remove NetworkHandler

    * Update tipset scheduling logic

    * Update peer retrieval to take a random sample of available peers

    * Cleanup and enabling all existing tests

    * fix worker task spawn

    * Add TODO for emit event ignoring and change to error log

    * oops

    * Update comment

    * Fix typo

commit 66ca99e
Author: Eric Tu <6364934+ec2@users.noreply.github.com>
Date:   Mon Sep 14 13:48:22 2020 -0400

    Fix StateManager use in different components (#694)

    * wrap sm in arc

    * lint

    * fix tests

    * suggestions

    * clippy

commit 96b64cb
Author: nannick <rajarupans@gmail.com>
Date:   Mon Sep 14 10:33:27 2020 -0400

    Drand ignore env variable (#697)

    * First commit

    * Using enviromental vairable instead of cli flag

    * FIxing dumb mistakes

    * Renaming drand flag

    * UPdate latest_beacon_entry

    * Adding drand check

    * Adding doc

commit 548a464
Author: Austin Abell <austinabell8@gmail.com>
Date:   Thu Sep 10 13:22:22 2020 -0400

    Print out conformance results and add log for skips (#695)

commit 0d7b16c
Author: Stepan <s.s.naumov@gmail.com>
Date:   Tue Sep 8 09:01:05 2020 -0400

    Add CLI command to add Genesis Miner to Genesis Template (#644)

    * AddMinerGenesis cmd command

    * fixing tests

    * addressing comments

    * removing jsons

* Delete old deadline tests

* Fix verify_block_signature

* Fix state_api

* Silent unused variable warnings

* clippy

* clippy

* clippy

* Fix miner worker addr function

* fix load_sectors_from_set

* Fix get_miner_info

* miner faults update

* Setup faults and recoveries logic

* Add todo for checking verification function

* Update to specs-actors v0.9.3

* Fix test

* Remove Cid.Undef check

* PR fixes

* Use the actor_error! macro in the miner actor

* Fix type mismatch

* PR changes

* Use return value

* Rename file

* More PR fixes

* Add assertions and error message

* Downcast errors originating from Deadlines::{load_deadline, update_deadline} and State::save_deadlines

* Rename Sectors::{new, load} to Sectors::{load, load_sector}

* Fix inverted logic

* Fix state manager

* Remove unused parameters

* clippy

* Replace function by const, remove duplication, remove comment

* Rename field, fix macro formatting, remove unnecessary vec

* Downcast errors

Co-authored-by: austinabell <austinabell8@gmail.com>
  • Loading branch information
@timvermeulen @austinabell
timvermeulen and austinabell authored Sep 26, 2020
1 parent 285b9c3 commit f007210
Show file tree
Hide file tree
Showing 34 changed files with 7,330 additions and 3,287 deletions.
1 change: 1 addition & 0 deletions Cargo.lock
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions blockchain/state_manager/src/lib.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -139,8 +139,10 @@ where

let state = StateTree::new_from_root(self.bs.as_ref(), state_cid)
.map_err(|e| Error::State(e.to_string()))?;
// Note: miner::State info likely to be changed to CID
let addr = resolve_to_key_addr(&state, self.bs.as_ref(), &ms.info.worker)

let info = ms.get_info(self.bs.as_ref()).map_err(|e| e.to_string())?;

let addr = resolve_to_key_addr(&state, self.bs.as_ref(), &info.worker)
.map_err(|e| Error::Other(format!("Failed to resolve key address; error: {}", e)))?;
Ok(addr)
}
Expand Down
215 changes: 124 additions & 91 deletions blockchain/state_manager/src/utils.rs
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
// Copyright 2020 ChainSafe Systems
// SPDX-License-Identifier: Apache-2.0, MIT

#![allow(unused)]

use crate::errors::*;
use crate::StateManager;
use actor::miner;
use actor::miner::{self, Partition};
use actor::{
miner::{ChainSectorInfo, Deadlines, MinerInfo, SectorOnChainInfo, SectorPreCommitOnChainInfo},
power,
Expand All @@ -19,6 +21,7 @@ use forest_blocks::Tipset;
use ipld_amt::Amt;
use ipld_hamt::Hamt;
use std::convert::TryInto;
use std::error::Error as StdError;

pub fn get_sectors_for_winning_post<DB>(
state_manager: &StateManager<DB>,
Expand All @@ -29,71 +32,72 @@ pub fn get_sectors_for_winning_post<DB>(
where
DB: BlockStore,
{
let miner_actor_state: miner::State =
state_manager
.load_actor_state(&address, &st)
.map_err(|err| {
Error::State(format!(
"(get sectors) failed to load miner actor state: %{:}",
err
))
})?;
let sector_set = get_proving_set_raw(state_manager, &miner_actor_state)?;
if sector_set.is_empty() {
return Ok(Vec::new());
}
let seal_proof_type = match miner_actor_state.info.sector_size {
SectorSize::_2KiB => RegisteredSealProof::StackedDRG2KiBV1,
SectorSize::_8MiB => RegisteredSealProof::StackedDRG8MiBV1,
SectorSize::_512MiB => RegisteredSealProof::StackedDRG512MiBV1,
SectorSize::_32GiB => RegisteredSealProof::StackedDRG32GiBV1,
SectorSize::_64GiB => RegisteredSealProof::StackedDRG64GiBV1,
};
let wpt = seal_proof_type.registered_winning_post_proof()?;
todo!()
// let miner_actor_state: miner::State =
// state_manager
// .load_actor_state(&address, &st)
// .map_err(|err| {
// Error::State(format!(
// "(get sectors) failed to load miner actor state: %{:}",
// err
// ))
// })?;
// let sector_set = get_proving_set_raw(state_manager, &miner_actor_state)?;
// if sector_set.is_empty() {
// return Ok(Vec::new());
// }
// let seal_proof_type = match miner_actor_state.info.sector_size {
// SectorSize::_2KiB => RegisteredSealProof::StackedDRG2KiBV1,
// SectorSize::_8MiB => RegisteredSealProof::StackedDRG8MiBV1,
// SectorSize::_512MiB => RegisteredSealProof::StackedDRG512MiBV1,
// SectorSize::_32GiB => RegisteredSealProof::StackedDRG32GiBV1,
// SectorSize::_64GiB => RegisteredSealProof::StackedDRG64GiBV1,
// };
// let wpt = seal_proof_type.registered_winning_post_proof()?;

if address.protocol() != Protocol::ID {
return Err(Error::Other(format!(
"failed to get ID from miner address {:}",
address
)));
};
let mut prover_id = ProverId::default();
let prover_bytes = address.to_bytes();
prover_id[..prover_bytes.len()].copy_from_slice(&prover_bytes);
let ids = generate_winning_post_sector_challenge(
wpt.try_into()?,
&rand,
sector_set.len() as u64,
prover_id,
)
.map_err(|err| Error::State(format!("generate winning posts challenge {:}", err)))?;
// if address.protocol() != Protocol::ID {
// return Err(Error::Other(format!(
// "failed to get ID from miner address {:}",
// address
// )));
// };
// let mut prover_id = ProverId::default();
// let prover_bytes = address.to_bytes();
// prover_id[..prover_bytes.len()].copy_from_slice(&prover_bytes);
// let ids = generate_winning_post_sector_challenge(
// wpt.try_into()?,
// &rand,
// sector_set.len() as u64,
// prover_id,
// )
// .map_err(|err| Error::State(format!("generate winning posts challenge {:}", err)))?;

Ok(ids
.iter()
.map::<Result<SectorInfo, Error>, _>(|i: &u64| {
let index = *i as usize;
let sector_number = sector_set
.get(index)
.ok_or_else(|| {
Error::Other(format!("Could not get sector_number at index {:}", index))
})?
.info
.sector_number;
let sealed_cid = sector_set
.get(index)
.ok_or_else(|| {
Error::Other(format!("Could not get sealed cid at index {:}", index))
})?
.info
.sealed_cid
.clone();
Ok(SectorInfo {
proof: seal_proof_type,
sector_number,
sealed_cid,
})
})
.collect::<Result<Vec<SectorInfo>, _>>()?)
// Ok(ids
// .iter()
// .map::<Result<SectorInfo, Error>, _>(|i: &u64| {
// let index = *i as usize;
// let sector_number = sector_set
// .get(index)
// .ok_or_else(|| {
// Error::Other(format!("Could not get sector_number at index {:}", index))
// })?
// .info
// .sector_number;
// let sealed_cid = sector_set
// .get(index)
// .ok_or_else(|| {
// Error::Other(format!("Could not get sealed cid at index {:}", index))
// })?
// .info
// .sealed_cid
// .clone();
// Ok(SectorInfo {
// proof: seal_proof_type,
// sector_number,
// sealed_cid,
// })
// })
// .collect::<Result<Vec<SectorInfo>, _>>()?)
}

pub fn get_proving_set_raw<DB>(
Expand All @@ -103,11 +107,12 @@ pub fn get_proving_set_raw<DB>(
where
DB: BlockStore,
{
let not_proving = &actor_state.faults | &actor_state.recoveries;
todo!()
// let not_proving = &actor_state.faults | &actor_state.recoveries;

actor_state
.load_sector_infos(&*state_manager.blockstore_cloned(), &not_proving)
.map_err(|err| Error::Other(format!("failed to get proving set :{:}", err)))
// actor_state
// .load_sector_infos(&*state_manager.blockstore_cloned(), &not_proving)
// .map_err(|err| Error::Other(format!("failed to get proving set :{:}", err)))
}

pub fn get_miner_sector_set<DB>(
Expand Down Expand Up @@ -158,7 +163,7 @@ where
}
}
sset.push(ChainSectorInfo {
info: sector_chain.info.to_owned(),
info: sector_chain.clone(),
id: i,
});
Ok(())
Expand Down Expand Up @@ -223,7 +228,7 @@ pub fn get_miner_info<DB>(
state_manager: &StateManager<DB>,
tipset: &Tipset,
address: &Address,
) -> Result<MinerInfo, Error>
) -> Result<MinerInfo, Box<dyn StdError>>
where
DB: BlockStore,
{
Expand All @@ -235,7 +240,7 @@ where
err
))
})?;
Ok(miner_actor_state.info)
Ok(miner_actor_state.get_info(state_manager.blockstore())?)
}

pub fn get_miner_deadlines<DB>(
Expand Down Expand Up @@ -264,42 +269,70 @@ where
})
}

fn for_each_deadline_partition<DB, F>(
state_manager: &StateManager<DB>,
tipset: &Tipset,
address: &Address,
mut cb: F,
) -> Result<(), Box<dyn StdError>>
where
F: FnMut(&Partition) -> Result<(), Box<dyn StdError>>,
DB: BlockStore,
{
let store = state_manager.blockstore();

// TODO clean this logic up
let miner_actor_state: miner::State =
state_manager.load_actor_state(&address, tipset.parent_state())?;
let deadlines = miner_actor_state.load_deadlines(store)?;
let pa = deadlines.for_each(store, |i, deadline| {
let partitions = deadline.partitions_amt(store).map_err(|e| e.to_string())?;
partitions
.for_each(|i, part| {
cb(part);
Ok(())
})
.map_err(|e| e.to_string())?;
Ok(())
});

Ok(())
}

pub fn get_miner_faults<DB>(
state_manager: &StateManager<DB>,
tipset: &Tipset,
address: &Address,
) -> Result<BitField, Error>
) -> Result<BitField, Box<dyn StdError>>
where
DB: BlockStore,
{
let miner_actor_state: miner::State = state_manager
.load_actor_state(&address, &tipset.parent_state())
.map_err(|err| {
Error::State(format!(
"(get miner faults) failed to load miner actor state: {:}",
err
))
})?;
Ok(miner_actor_state.faults)
let mut out = BitField::new();

for_each_deadline_partition(state_manager, tipset, address, |part| {
out |= &part.faults;
Ok(())
})?;

Ok(out)
}

pub fn get_miner_recoveries<DB>(
state_manager: &StateManager<DB>,
tipset: &Tipset,
address: &Address,
) -> Result<BitField, Error>
) -> Result<BitField, Box<dyn StdError>>
where
DB: BlockStore,
{
let miner_actor_state: miner::State = state_manager
.load_actor_state(&address, &tipset.parent_state())
.map_err(|err| {
Error::State(format!(
"(get miner recoveries) failed to load miner actor state: {:}",
err
))
})?;
Ok(miner_actor_state.recoveries)
let mut out = BitField::new();

for_each_deadline_partition(state_manager, tipset, address, |part| {
out |= &part.recoveries;
Ok(())
})?;

Ok(out)
}

pub fn list_miner_actors<'a, DB>(
Expand Down
2 changes: 1 addition & 1 deletion encoding/src/bytes.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ use serde_bytes::ByteBuf;
pub struct BytesSer<'a>(#[serde(with = "serde_bytes")] pub &'a [u8]);

/// Wrapper for deserializing dynamic sized Bytes.
#[derive(Deserialize, Serialize)]
#[derive(Deserialize, Serialize, Debug, PartialEq)]
#[serde(transparent)]
pub struct BytesDe(#[serde(with = "serde_bytes")] pub Vec<u8>);

Expand Down
51 changes: 50 additions & 1 deletion ipld/amt/src/amt.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,15 @@ where
Ok(true)
}

/// Deletes multiple items from AMT
pub fn batch_delete(&mut self, iter: impl IntoIterator<Item = u64>) -> Result<(), Error> {
// TODO: optimize this
for i in iter {
self.delete(i)?;
}
Ok(())
}

/// flush root and return Cid used as key in block store
pub fn flush(&mut self) -> Result<Cid, Error> {
self.root.node.flush(self.block_store)?;
Expand Down Expand Up @@ -228,9 +237,49 @@ where
where
V: DeserializeOwned,
F: FnMut(u64, &V) -> Result<(), Box<dyn StdError>>,
{
self.for_each_while(|i, x| {
f(i, x)?;
Ok(true)
})
}

/// Iterates over each value in the Amt and runs a function on the values, for as long as that
/// function keeps returning `true`.
pub fn for_each_while<F>(&self, mut f: F) -> Result<(), Box<dyn StdError>>
where
V: DeserializeOwned,
F: FnMut(u64, &V) -> Result<bool, Box<dyn StdError>>,
{
self.root
.node
.for_each_while(self.block_store, self.height(), 0, &mut f)
.map(|_| ())
}

/// Iterates over each value in the Amt and runs a function on the values that allows modifying
/// each value.
pub fn for_each_mut<F>(&mut self, mut f: F) -> Result<(), Box<dyn StdError>>
where
V: DeserializeOwned,
F: FnMut(u64, &mut V) -> Result<(), Box<dyn StdError>>,
{
self.for_each_while_mut(|i, x| {
f(i, x)?;
Ok(true)
})
}

/// Iterates over each value in the Amt and runs a function on the values that allows modifying
/// each value, for as long as that function keeps returning `true`.
pub fn for_each_while_mut<F>(&mut self, mut f: F) -> Result<(), Box<dyn StdError>>
where
V: DeserializeOwned,
F: FnMut(u64, &mut V) -> Result<bool, Box<dyn StdError>>,
{
self.root
.node
.for_each(self.block_store, self.height(), 0, &mut f)
.for_each_while_mut(self.block_store, self.height(), 0, &mut f)
.map(|_| ())
}
}
Loading

0 comments on commit f007210

Please sign in to comment.