Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Decouple Staking and Election #7319

Closed
wants to merge 89 commits into from
Closed
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
25bddc9
mockup
kianenigma Sep 11, 2020
7bef52f
Add files
kianenigma Sep 11, 2020
4f3dfa4
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Sep 23, 2020
bc30ce1
More mockup
kianenigma Sep 24, 2020
9e52fbc
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Oct 1, 2020
14ebc95
Lots more changes and tests. Initial design of signed is forseable now.
kianenigma Oct 2, 2020
0587fc9
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Oct 8, 2020
233d4e1
More changes for the signed phase, now going to the cluster-fu*k of g…
kianenigma Oct 9, 2020
9c880f3
Finally decoupled solution from the pallet.. such a PITA.
kianenigma Oct 9, 2020
0cb35eb
Much simplifications and cleanups.
kianenigma Oct 9, 2020
c1ff3ec
A skeleton for the unsigned version.
kianenigma Oct 9, 2020
85a346e
Master.into()
kianenigma Oct 13, 2020
6a2542d
Staking builds and tests compile, some tests fail
kianenigma Oct 13, 2020
c361911
All staking tests work
kianenigma Oct 14, 2020
218db10
Unsigned almost done, primitive crate added.
kianenigma Oct 15, 2020
5e21a79
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Oct 15, 2020
25c1bc3
Use decl_error
kianenigma Oct 15, 2020
d53785c
Add basic benchmark setup, needs more work
kianenigma Oct 16, 2020
525f6e0
Comment.
kianenigma Oct 16, 2020
3026316
Checkpoint.
kianenigma Oct 20, 2020
9841e3b
node-runtime builds.
kianenigma Oct 20, 2020
16c0550
Builds and all good?
kianenigma Dec 3, 2020
f197b6b
Undo some formatting mistakes
kianenigma Dec 3, 2020
5005281
Working on benchmarks'
kianenigma Dec 7, 2020
bd0d331
Fix bench
kianenigma Dec 8, 2020
67dd4d1
Strange merge -- everything builds.
kianenigma Dec 8, 2020
eac4571
Everything compiles and good -- need to fix benches.
kianenigma Dec 8, 2020
a7c4841
Some Staking stuff left to do.
kianenigma Dec 9, 2020
2ca3dcf
Master.into()
kianenigma Dec 14, 2020
4922897
Split the snapshot; type for CompactSolution and MaxNominations
kianenigma Dec 15, 2020
ef593a7
Fix a bunch more issues.
kianenigma Dec 15, 2020
8661814
check new design
kianenigma Dec 18, 2020
57f7294
Everything within the module works and compiles.
kianenigma Dec 21, 2020
af304d1
Stkaing also builds
kianenigma Dec 21, 2020
80e7450
Everything builds with the new scheme.
kianenigma Dec 25, 2020
4b4712a
Master.into()
kianenigma Dec 26, 2020
206933a
Simplify elect API.
kianenigma Dec 26, 2020
af9b97d
Fix all other tests.
kianenigma Dec 26, 2020
9663fea
make the staking api better and well tested.
kianenigma Jan 4, 2021
7f4f5b7
Almost done with conversion
kianenigma Jan 4, 2021
8fb11f1
Move to pallet V2 + construct_runtime
kianenigma Jan 5, 2021
bd6d4d9
Master.into()
kianenigma Jan 5, 2021
65f5ac1
Revert staking changes;
kianenigma Jan 5, 2021
3860885
Revert Staking fuzzer.
kianenigma Jan 5, 2021
392f35a
Fix fuzzers
kianenigma Jan 5, 2021
1337e1b
Update frame/election-providers/src/two_phase/mod.rs
kianenigma Jan 6, 2021
4f6ec0d
a round of self review
kianenigma Jan 6, 2021
b3113f7
A large number of bugfixes after local test and self-review
kianenigma Jan 7, 2021
abca039
Upstream.into()
kianenigma Jan 7, 2021
6bc578a
Master.into()
kianenigma Jan 7, 2021
876f666
Apply suggestions from code review
kianenigma Jan 7, 2021
81df1df
Update frame/election-providers/src/two_phase/benchmarking.rs
kianenigma Jan 7, 2021
e3862ce
Fix some review grumbles.
kianenigma Jan 8, 2021
ca06f8e
Fix some docs
kianenigma Jan 8, 2021
6206c32
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 8, 2021
95d7a3c
Master.into()
kianenigma Jan 8, 2021
63776ff
Maybe relocalte the code?
kianenigma Jan 8, 2021
4732854
Master.into()
kianenigma Jan 8, 2021
a922f0b
Witness -> SoltionSize
kianenigma Jan 8, 2021
6d8247a
cargo run --release --features=runtime-benchmarks --manifest-path=bin…
Jan 8, 2021
e9f4437
cargo run --release --features=runtime-benchmarks --manifest-path=bin…
Jan 8, 2021
02069e1
Update frame/session/src/lib.rs
kianenigma Jan 12, 2021
000ecdc
Revamp macros
kianenigma Jan 12, 2021
cb6f156
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 12, 2021
df04ef3
Merge remote-tracking branch 'origin/master' into kiz-election-provider
Jan 12, 2021
ee2a316
Master.into()
kianenigma Jan 12, 2021
0d92ed5
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 12, 2021
88a96a0
Fix build
kianenigma Jan 12, 2021
6480ce1
Merge remote-tracking branch 'origin/master' into kiz-election-provider
Jan 12, 2021
8a68915
cargo run --release --features=runtime-benchmarks --manifest-path=bin…
Jan 12, 2021
f4e2c03
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Jan 12, 2021
c41ecb0
fix all the tests.
kianenigma Jan 13, 2021
b42336a
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 13, 2021
f5e23f9
Fix the strange issue with cargo lock
kianenigma Jan 13, 2021
56bb93c
revamp closures.
kianenigma Jan 14, 2021
257795c
Update frame/two-phase-election-provider/src/benchmarking.rs
kianenigma Jan 14, 2021
9c114bc
Merge remote-tracking branch 'origin/master' into kiz-election-provider
Jan 14, 2021
1ac97f5
Some review grumbles.
kianenigma Jan 14, 2021
366b15b
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Jan 14, 2021
afa5507
Upstream.into()
kianenigma Jan 14, 2021
5d206ab
Fix build
kianenigma Jan 14, 2021
ea193c7
Bring back benches
kianenigma Jan 14, 2021
60e57d7
cargo run --release --features=runtime-benchmarks --manifest-path=bin…
Jan 14, 2021
77f53a3
Add a bunch of more fixes: signed weight check, tests etc.
kianenigma Jan 15, 2021
9ce9d6b
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 15, 2021
b30dfb3
Fix build
kianenigma Jan 15, 2021
6001d7d
Master.into()
kianenigma Jan 15, 2021
a066e4a
Last small tweaks
kianenigma Jan 15, 2021
10f6e7b
Fix all tests.
kianenigma Jan 15, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ members = [
"frame/staking",
"frame/staking/fuzzer",
"frame/staking/reward-curve",
"frame/election-providers",
"frame/sudo",
"frame/support",
"frame/support/procedural",
Expand Down Expand Up @@ -137,6 +138,7 @@ members = [
"primitives/core",
"primitives/database",
"primitives/debug-derive",
"primitives/election-providers",
"primitives/externalities",
"primitives/finality-grandpa",
"primitives/inherents",
Expand Down
5 changes: 5 additions & 0 deletions bin/node/runtime/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,14 @@ sp-keyring = { version = "2.0.0", optional = true, path = "../../../primitives/k
sp-session = { version = "2.0.0", default-features = false, path = "../../../primitives/session" }
sp-transaction-pool = { version = "2.0.0", default-features = false, path = "../../../primitives/transaction-pool" }
sp-version = { version = "2.0.0", default-features = false, path = "../../../primitives/version" }
sp-npos-elections = { version = "2.0.0", default-features = false, path = "../../../primitives/npos-elections" }

# frame dependencies
frame-executive = { version = "2.0.0", default-features = false, path = "../../../frame/executive" }
frame-benchmarking = { version = "2.0.0", default-features = false, path = "../../../frame/benchmarking", optional = true }
frame-support = { version = "2.0.0", default-features = false, path = "../../../frame/support" }
frame-system = { version = "2.0.0", default-features = false, path = "../../../frame/system" }
frame-election-providers = { version = "2.0.0", default-features = false, path = "../../../frame/election-providers/" }
frame-system-benchmarking = { version = "2.0.0", default-features = false, path = "../../../frame/system/benchmarking", optional = true }
frame-system-rpc-runtime-api = { version = "2.0.0", default-features = false, path = "../../../frame/system/rpc/runtime-api/" }
pallet-assets = { version = "2.0.0", default-features = false, path = "../../../frame/assets" }
Expand Down Expand Up @@ -113,6 +115,7 @@ std = [
"pallet-im-online/std",
"pallet-indices/std",
"sp-inherents/std",
"sp-npos-elections/std",
"pallet-membership/std",
"pallet-mmr/std",
"pallet-multisig/std",
Expand All @@ -138,6 +141,7 @@ std = [
"frame-benchmarking/std",
"frame-system-rpc-runtime-api/std",
"frame-system/std",
"frame-election-providers/std",
"pallet-timestamp/std",
"pallet-tips/std",
"pallet-transaction-payment-rpc-runtime-api/std",
Expand All @@ -154,6 +158,7 @@ runtime-benchmarks = [
"frame-benchmarking",
"frame-support/runtime-benchmarks",
"frame-system/runtime-benchmarks",
"frame-election-providers/runtime-benchmarks",
"sp-runtime/runtime-benchmarks",
"pallet-assets/runtime-benchmarks",
"pallet-babe/runtime-benchmarks",
Expand Down
64 changes: 57 additions & 7 deletions bin/node/runtime/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ use frame_support::{
construct_runtime, parameter_types, debug, RuntimeDebug,
weights::{
Weight, IdentityFee,
constants::{BlockExecutionWeight, ExtrinsicBaseWeight, RocksDbWeight, WEIGHT_PER_SECOND}, DispatchClass,
constants::{BlockExecutionWeight, ExtrinsicBaseWeight, RocksDbWeight, WEIGHT_PER_SECOND},
DispatchClass,
},
traits::{
Currency, Imbalance, KeyOwnerProofSystem, OnUnbalanced, Randomness, LockIdentifier,
Expand All @@ -37,7 +38,7 @@ use frame_support::{
};
use frame_system::{
EnsureRoot, EnsureOneOf,
limits::{BlockWeights, BlockLength}
limits::{BlockWeights, BlockLength},
};
use frame_support::traits::InstanceFilter;
use codec::{Encode, Decode};
Expand All @@ -50,14 +51,14 @@ pub use node_primitives::{AccountId, Signature};
use node_primitives::{AccountIndex, Balance, BlockNumber, Hash, Index, Moment};
use sp_api::impl_runtime_apis;
use sp_runtime::{
Permill, Perbill, Perquintill, Percent, ApplyExtrinsicResult,
impl_opaque_keys, generic, create_runtime_str, ModuleId, FixedPointNumber,
Permill, Perbill, Perquintill, Percent, ApplyExtrinsicResult, impl_opaque_keys, generic,
create_runtime_str, ModuleId, FixedPointNumber,
};
use sp_runtime::curve::PiecewiseLinear;
use sp_runtime::transaction_validity::{TransactionValidity, TransactionSource, TransactionPriority};
use sp_runtime::traits::{
self, BlakeTwo256, Block as BlockT, StaticLookup, SaturatedConversion,
ConvertInto, OpaqueKeys, NumberFor,
self, BlakeTwo256, Block as BlockT, StaticLookup, SaturatedConversion, ConvertInto, OpaqueKeys,
NumberFor,
};
use sp_version::RuntimeVersion;
#[cfg(any(feature = "std", test))]
Expand All @@ -72,6 +73,7 @@ use pallet_session::{historical as pallet_session_historical};
use sp_inherents::{InherentData, CheckInherentsResult};
use static_assertions::const_assert;
use pallet_contracts::WeightInfo;
use frame_election_providers::two_phase as pallet_two_phase_election_provider;

#[cfg(any(feature = "std", test))]
pub use sp_runtime::BuildStorage;
Expand Down Expand Up @@ -488,18 +490,64 @@ impl pallet_staking::Config for Runtime {
type SessionInterface = Self;
type RewardCurve = RewardCurve;
type NextNewSession = Session;
type MaxNominatorRewardedPerValidator = MaxNominatorRewardedPerValidator;
type ElectionLookahead = ElectionLookahead;
type Call = Call;
type MaxIterations = MaxIterations;
type MinSolutionScoreBump = MinSolutionScoreBump;
type MaxNominatorRewardedPerValidator = MaxNominatorRewardedPerValidator;
type UnsignedPriority = StakingUnsignedPriority;
// The unsigned solution weight targeted by the OCW. We set it to the maximum possible value of
// a single extrinsic.
type OffchainSolutionWeightLimit = OffchainSolutionWeightLimit;
type ElectionProvider = TwoPhaseElectionProvider;
type WeightInfo = pallet_staking::weights::SubstrateWeight<Runtime>;
}

parameter_types! {
// phase durations
pub const SignedPhase: u32 = 25;
pub const UnsignedPhase: u32 = 25;

// signed configs
pub const MaxSignedSubmissions: u32 = 10;
pub const SignedRewardBase: Balance = 1 * DOLLARS;
pub const SignedDepositBase: Balance = 1 * DOLLARS;
pub const SignedDepositByte: Balance = 1 * CENTS;

// unsigned configs
pub const TwoPhaseUnsignedPriority: TransactionPriority = StakingUnsignedPriority::get() - 1u64;
pub const MaxUnsignedIterations: u32 = 10;
pub SolutionImprovementThreshold: Perbill = Perbill::from_rational_approximation(5u32, 10_000);
pub MinerMaxWeight: Weight = RuntimeBlockWeights::get()
.get(DispatchClass::Normal)
.max_extrinsic.expect("Normal extrinsics have a weight limit configured; qed")
.saturating_sub(BlockExecutionWeight::get());
}

impl pallet_two_phase_election_provider::Config for Runtime {
type Event = Event;
type Currency = Balances;
type SignedPhase = SignedPhase;
type UnsignedPhase = UnsignedPhase;
type MaxSignedSubmissions = MaxSignedSubmissions;
type SignedRewardBase = SignedRewardBase;
type SignedRewardFactor = (); // no score-based reward
type SignedRewardMax = SignedRewardBase;
type SignedDepositBase = SignedDepositBase;
type SignedDepositByte = SignedDepositByte;
type SignedDepositWeight = (); // no weight-based deposit.
type SolutionImprovementThreshold = MinSolutionScoreBump;
type SlashHandler = (); // burn slashes
type RewardHandler = ();
type MinerMaxIterations = MaxUnsignedIterations;
type MinerMaxWeight = MinerMaxWeight;
type UnsignedPriority = TwoPhaseUnsignedPriority;
type DataProvider = Staking;
type OnChainAccuracy = Perbill;
type CompactSolution = pallet_staking::CompactAssignments;
type WeightInfo = pallet_two_phase_election_provider::weights::SubstrateWeight<Runtime>;
}

parameter_types! {
pub const LaunchPeriod: BlockNumber = 28 * 24 * 60 * MINUTES;
pub const VotingPeriod: BlockNumber = 28 * 24 * 60 * MINUTES;
Expand Down Expand Up @@ -982,6 +1030,7 @@ construct_runtime!(
Indices: pallet_indices::{Module, Call, Storage, Config<T>, Event<T>},
Balances: pallet_balances::{Module, Call, Storage, Config<T>, Event<T>},
TransactionPayment: pallet_transaction_payment::{Module, Storage},
TwoPhaseElectionProvider: pallet_two_phase_election_provider::{Module, Call, Storage, Event<T>, ValidateUnsigned},
Staking: pallet_staking::{Module, Call, Config<T>, Storage, Event<T>, ValidateUnsigned},
Session: pallet_session::{Module, Call, Storage, Event, Config<T>},
Democracy: pallet_democracy::{Module, Call, Storage, Config, Event<T>},
Expand Down Expand Up @@ -1304,6 +1353,7 @@ impl_runtime_apis! {
add_benchmark!(params, batches, pallet_treasury, Treasury);
add_benchmark!(params, batches, pallet_utility, Utility);
add_benchmark!(params, batches, pallet_vesting, Vesting);
add_benchmark!(params, batches, pallet_two_phase_election_provider, TwoPhaseElectionProvider);

if batches.is_empty() { return Err("Benchmark not found for this pallet.".into()) }
Ok(batches)
Expand Down
2 changes: 0 additions & 2 deletions client/executor/src/integration_tests/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -761,7 +761,6 @@ fn spawning_runtime_instance_nested_should_work(wasm_method: WasmExecutionMethod
#[test_case(WasmExecutionMethod::Interpreted)]
#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))]
fn panic_in_spawned_instance_panics_on_joining_its_result(wasm_method: WasmExecutionMethod) {

let mut ext = TestExternalities::default();
let mut ext = ext.ext();

Expand All @@ -772,6 +771,5 @@ fn panic_in_spawned_instance_panics_on_joining_its_result(wasm_method: WasmExecu
&mut ext,
).unwrap_err();

dbg!(&error_result);
assert!(format!("{}", error_result).contains("Spawned task"));
}
6 changes: 3 additions & 3 deletions client/service/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,13 @@ pub type Result<T> = std::result::Result<T, Error>;
pub enum Error {
#[error(transparent)]
Client(#[from] sp_blockchain::Error),

#[error(transparent)]
Io(#[from] std::io::Error),

#[error(transparent)]
Consensus(#[from] sp_consensus::Error),

#[error(transparent)]
Network(#[from] sc_network::error::Error),

Expand Down
12 changes: 9 additions & 3 deletions frame/babe/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -407,12 +407,14 @@ impl<T: Config> Module<T> {
/// In other word, this is only accurate if no slots are missed. Given missed slots, the slot
/// number will grow while the block number will not. Hence, the result can be interpreted as an
/// upper bound.
// -------------- IMPORTANT NOTE --------------
//
// ## IMPORTANT NOTE
//
// This implementation is linked to how [`should_epoch_change`] is working. This might need to
// be updated accordingly, if the underlying mechanics of slot and epochs change.
//
// WEIGHT NOTE: This function is tied to the weight of `EstimateNextSessionRotation`. If you update
// this function, you must also update the corresponding weight.
// WEIGHT NOTE: This function is tied to the weight of `EstimateNextSessionRotation`. If you
// update this function, you must also update the corresponding weight.
Comment on lines +424 to +425
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this note was before we did weights at the full implementation level. I bet if we look at this code, we may be able to remove some artifacts of passing around weights, and instead just measure the top level function for its weight.

pub fn next_expected_epoch_change(now: T::BlockNumber) -> Option<T::BlockNumber> {
let next_slot = Self::current_epoch_start().saturating_add(T::EpochDuration::get());
next_slot
Expand Down Expand Up @@ -706,6 +708,10 @@ impl<T: Config> OnTimestampSet<T::Moment> for Module<T> {
}

impl<T: Config> frame_support::traits::EstimateNextSessionRotation<T::BlockNumber> for Module<T> {
fn average_session_length() -> T::BlockNumber {
T::EpochDuration::get().saturated_into()
}

fn estimate_next_session_rotation(now: T::BlockNumber) -> Option<T::BlockNumber> {
Self::next_expected_epoch_change(now)
}
Expand Down
Loading