Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Disputes runtime #2947

Merged
74 commits merged into from
Jul 19, 2021
Merged
Show file tree
Hide file tree
Changes from 46 commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
8bd80d3
disputes module skeleton and storage
rphmeier Apr 28, 2021
101fde9
implement dispute module initialization logic
rphmeier Apr 28, 2021
4198797
implement disputes session change logic
rphmeier Apr 28, 2021
7dac193
provide dispute skeletons
rphmeier May 4, 2021
dd01c06
deduplication & ancient check
rphmeier May 4, 2021
ef153dc
fix a couple of warnings
rphmeier May 4, 2021
8dea7e6
begin provide_dispute_data impl
rphmeier May 5, 2021
f70f6e2
flesh out statement set import somewhat
rphmeier May 5, 2021
0451139
move ApprovalVote to shared primitives
rphmeier May 5, 2021
6748578
add a signing-payload API to explicit dispute statements
rphmeier May 5, 2021
95d6c19
implement statement signature checking
rphmeier May 5, 2021
ce5906b
Merge branch 'master' into rh-disputes-runtime
rphmeier May 5, 2021
01e948e
some bitflags glue for observing changes in disputes
rphmeier May 6, 2021
8e25b48
implement dispute vote import logic
rphmeier May 7, 2021
271434b
flesh out everything except slashing
rphmeier May 8, 2021
310ee52
guide: tweaks
rphmeier May 10, 2021
37c3c3a
declare and use punishment trait
rphmeier May 10, 2021
8180eae
punish validators for inconclusive disputes
rphmeier May 10, 2021
2ed7fe5
guide: tiny fix
rphmeier May 10, 2021
31c27c0
guide: update docs
rphmeier May 10, 2021
cfdbebd
add disputes getter fn
rphmeier May 10, 2021
dcd410c
guide: small change to spam slots handling
rphmeier May 10, 2021
ebd8b18
improve spam slots handling and fix some bugs
rphmeier May 10, 2021
da2d43a
finish API of disputes runtime
rphmeier May 10, 2021
d09c851
define and deposit `RevertTo` log
rphmeier May 11, 2021
cfe3241
begin integrating disputes into para_inherent
rphmeier May 11, 2021
d14e904
use precomputed slash_for/against
rphmeier May 11, 2021
7ffe889
return candidate hash from process_bitfields
rphmeier May 11, 2021
3638877
implement inclusion::collect_disputed
rphmeier May 11, 2021
82e86da
finish integration into rest of runtime
rphmeier May 11, 2021
955e0c4
add Disputes to initializer
rphmeier May 11, 2021
83e7b62
address suggestions
gui1117 May 21, 2021
fc3ab87
use pallet macro
gui1117 May 21, 2021
6fa374c
fix typo
gui1117 May 21, 2021
1b4838d
Update runtime/parachains/src/disputes.rs
gui1117 May 27, 2021
c0d15be
Merge branch 'rh-disputes-runtime' of https://github.com/paritytech/p…
rphmeier May 28, 2021
7ef44b5
add test: fix pruning
gui1117 May 28, 2021
7570236
document specific behavior
gui1117 May 28, 2021
6dd0ad6
deposit events on dispute changes
rphmeier May 31, 2021
040cc17
add an allow(unused) on fn disputes
rphmeier May 31, 2021
4a3eeba
add a dummy PunishValidators implementation
rphmeier May 31, 2021
6f5b27f
add disputes module to Rococo
rphmeier May 31, 2021
aacc3de
add disputes module to westend runtime
rphmeier May 31, 2021
993709b
add disputes module to test runtime
rphmeier May 31, 2021
5b7928a
add disputes module to kusama runtime
rphmeier May 31, 2021
14bd62d
Merge branch 'master' into rh-disputes-runtime
rphmeier May 31, 2021
776b3b6
guide: prepare for runtime API for checking frozenness
rphmeier Jun 2, 2021
a4ee80f
remove revert digests in favor of state variable
rphmeier Jun 2, 2021
b0d579c
merge reversions
rphmeier Jun 3, 2021
9430d3c
Update runtime/parachains/src/disputes.rs
rphmeier Jun 3, 2021
0680641
Update runtime/parachains/src/disputes.rs
rphmeier Jun 3, 2021
a715d61
Update runtime/parachains/src/disputes.rs
rphmeier Jun 3, 2021
3f6ce32
Merge branch 'master' into rh-disputes-runtime
rphmeier Jun 7, 2021
8ebc635
add byzantine_threshold and supermajority_threshold utilities to prim…
rphmeier Jun 10, 2021
f3e22c0
use primitive helpers
rphmeier Jun 10, 2021
53e2ccb
deposit revert event when freezing chain
andresilva Jun 23, 2021
4f4b84b
deposit revert log when freezing chain
andresilva Jun 23, 2021
a7ad191
test revert event and log are generated when freezing
andresilva Jun 23, 2021
7b04be6
add trait to decouple disputes handling from paras inherent handling
andresilva Jun 23, 2021
c6cf3f7
runtime: fix compilation and setup dispute handler
andresilva Jun 23, 2021
3acf2c1
Merge branch 'master' into rh-disputes-runtime
andresilva Jun 23, 2021
9aa95b4
disputes: add hook for filtering out dispute statements
andresilva Jun 23, 2021
ff06d0f
disputes: add initializer hooks to DisputesHandler
andresilva Jun 23, 2021
dd35e7f
runtime: remove disputes pallet from all runtimes
andresilva Jun 23, 2021
df6bfa8
Merge branch 'master' into rh-disputes-runtime
rphmeier Jul 15, 2021
9214a99
Merge branch 'rh-disputes-runtime' of https://github.com/paritytech/p…
rphmeier Jul 15, 2021
eb81811
tag TODOs
rphmeier Jul 15, 2021
c7afa0c
don't import any dispute statements just yet...
rphmeier Jul 15, 2021
4378b5c
address grumbles
rphmeier Jul 17, 2021
8a7abdb
fix spellcheck, hopefully
rphmeier Jul 17, 2021
663b5cc
maybe now?
rphmeier Jul 17, 2021
3be97bb
last spellcheck round
rphmeier Jul 17, 2021
bb47ed4
fix runtime tests
rphmeier Jul 19, 2021
5ecf237
fix test-runtime
rphmeier Jul 19, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

26 changes: 5 additions & 21 deletions node/core/approval-voting/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,15 +38,13 @@ use polkadot_primitives::v1::{
ValidatorIndex, Hash, SessionIndex, SessionInfo, CandidateHash,
CandidateReceipt, BlockNumber, PersistedValidationData,
ValidationCode, CandidateDescriptor, ValidatorPair, ValidatorSignature, ValidatorId,
CandidateIndex, GroupIndex,
CandidateIndex, GroupIndex, ApprovalVote,
};
use polkadot_node_primitives::{ValidationResult, PoV};
use polkadot_node_primitives::approval::{
IndirectAssignmentCert, IndirectSignedApprovalVote, ApprovalVote, DelayTranche,
BlockApprovalMeta,
IndirectAssignmentCert, IndirectSignedApprovalVote, DelayTranche, BlockApprovalMeta,
};
use polkadot_node_jaeger as jaeger;
use parity_scale_codec::Encode;
use sc_keystore::LocalKeystore;
use sp_consensus::SyncOracle;
use sp_consensus_slots::Slot;
Expand Down Expand Up @@ -1226,15 +1224,6 @@ async fn handle_approved_ancestor(
Ok(all_approved_max)
}

fn approval_signing_payload(
approval_vote: ApprovalVote,
session_index: SessionIndex,
) -> Vec<u8> {
const MAGIC: [u8; 4] = *b"APPR";

(MAGIC, approval_vote, session_index).encode()
}

// `Option::cmp` treats `None` as less than `Some`.
fn min_prefer_some<T: std::cmp::Ord>(
a: Option<T>,
Expand Down Expand Up @@ -1466,10 +1455,8 @@ fn check_and_import_approval<T>(
None => respond_early!(ApprovalCheckResult::Bad)
};

let approval_payload = approval_signing_payload(
ApprovalVote(approved_candidate_hash),
block_entry.session(),
);
let approval_payload = ApprovalVote(approved_candidate_hash)
.signing_payload(block_entry.session());

let pubkey = match session_info.validators.get(approval.validator.0 as usize) {
Some(k) => k,
Expand Down Expand Up @@ -2157,10 +2144,7 @@ fn sign_approval(
) -> Option<ValidatorSignature> {
let key = keystore.key_pair::<ValidatorPair>(public).ok().flatten()?;

let payload = approval_signing_payload(
ApprovalVote(candidate_hash),
session_index,
);
let payload = ApprovalVote(candidate_hash).signing_payload(session_index);

Some(key.sign(&payload[..]))
}
2 changes: 1 addition & 1 deletion node/core/approval-voting/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@ fn sign_approval(
candidate_hash: CandidateHash,
session_index: SessionIndex,
) -> ValidatorSignature {
key.sign(&super::approval_signing_payload(ApprovalVote(candidate_hash), session_index)).into()
key.sign(&ApprovalVote(candidate_hash).signing_payload(session_index)).into()
}

#[derive(Clone)]
Expand Down
4 changes: 0 additions & 4 deletions node/primitives/src/approval.rs
Original file line number Diff line number Diff line change
Expand Up @@ -98,10 +98,6 @@ pub struct IndirectAssignmentCert {
pub cert: AssignmentCert,
}

/// A vote of approval on a candidate.
#[derive(Debug, Clone, Encode, Decode)]
pub struct ApprovalVote(pub CandidateHash);

/// A signed approval vote which references the candidate indirectly via the block.
///
/// In practice, we have a look-up from block hash and candidate index to candidate hash,
Expand Down
8 changes: 8 additions & 0 deletions primitives/src/v0.rs
Original file line number Diff line number Diff line change
Expand Up @@ -672,6 +672,14 @@ pub enum CompactStatement {
Valid(CandidateHash),
}

impl CompactStatement {
/// Yields the payload used for validator signatures on this kind
/// of statement.
pub fn signing_payload(&self, context: &SigningContext) -> Vec<u8> {
(self, context).encode()
}
}

// Inner helper for codec on `CompactStatement`.
#[derive(Encode, Decode)]
enum CompactStatementInner {
Expand Down
37 changes: 33 additions & 4 deletions primitives/src/v1/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -842,6 +842,22 @@ pub struct SessionInfo {
pub needed_approvals: u32,
}

/// A vote of approval on a candidate.
#[derive(Clone, RuntimeDebug)]
pub struct ApprovalVote(pub CandidateHash);

impl ApprovalVote {
/// Yields the signing payload for this approval vote.
pub fn signing_payload(
&self,
session_index: SessionIndex,
) -> Vec<u8> {
const MAGIC: [u8; 4] = *b"APPR";

(MAGIC, &self.0, session_index).encode()
}
}

sp_api::decl_runtime_apis! {
/// The API for querying the state of parachains on-chain.
pub trait ParachainHost<H: Decode = Hash, N: Encode + Decode = BlockNumber> {
Expand Down Expand Up @@ -1039,6 +1055,10 @@ pub enum ConsensusLog {
/// number in the current chain, inclusive.
#[codec(index = 3)]
ForceApprove(BlockNumber),
/// The runtime is informing the node to revert the chain back to the given block number
/// in the same chain.
#[codec(index = 4)]
RevertTo(BlockNumber),
}

impl ConsensusLog {
Expand Down Expand Up @@ -1081,10 +1101,10 @@ pub enum ValidDisputeStatementKind {
Explicit,
/// A seconded statement on a candidate from the backing phase.
#[codec(index = 1)]
BackingSeconded,
BackingSeconded(Hash),
/// A valid statement on a candidate from the backing phase.
#[codec(index = 2)]
BackingValid,
BackingValid(Hash),
/// An approval vote from the approval checking phase.
#[codec(index = 3)]
ApprovalChecking,
Expand All @@ -1099,7 +1119,7 @@ pub enum InvalidDisputeStatementKind {
}

/// An explicit statement on a candidate issued as part of a dispute.
#[derive(Encode, Decode, Clone, PartialEq, RuntimeDebug)]
#[derive(Clone, PartialEq, RuntimeDebug)]
pub struct ExplicitDisputeStatement {
/// Whether the candidate is valid
pub valid: bool,
Expand All @@ -1109,6 +1129,15 @@ pub struct ExplicitDisputeStatement {
pub session: SessionIndex,
}

impl ExplicitDisputeStatement {
/// Produce the payload used for signing this type of statement.
pub fn signing_payload(&self) -> Vec<u8> {
const MAGIC: [u8; 4] = *b"DISP";

(MAGIC, self.valid, self.candidate_hash, self.session).encode()
}
}

/// A set of statements about a specific candidate.
#[derive(Encode, Decode, Clone, PartialEq, RuntimeDebug)]
pub struct DisputeStatementSet {
Expand All @@ -1124,7 +1153,7 @@ pub struct DisputeStatementSet {
pub type MultiDisputeStatementSet = Vec<DisputeStatementSet>;

/// The entire state of a dispute.
#[derive(Encode, Decode, Clone, RuntimeDebug)]
#[derive(Encode, Decode, Clone, RuntimeDebug, PartialEq)]
pub struct DisputeState<N = BlockNumber> {
/// A bitfield indicating all validators for the candidate.
pub validators_for: BitVec<bitvec::order::Lsb0, u8>, // one bit per validator.
Expand Down
17 changes: 8 additions & 9 deletions roadmap/implementers-guide/src/runtime/disputes.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Included: double_map (SessionIndex, CandidateHash) -> Option<BlockNumber>,
// fewer than `byzantine_threshold + 1` validators.
//
// The i'th entry of the vector corresponds to the i'th validator in the session.
drahnr marked this conversation as resolved.
Show resolved Hide resolved
SpamSlots: map SessionIndex -> Vec<u32>,
SpamSlots: map SessionIndex -> Option<Vec<u32>>,
// Whether the chain is frozen or not. Starts as `false`. When this is `true`,
// the chain will not accept any new parachain blocks for backing or inclusion.
// It can only be set back to `false` by governance intervention.
Expand All @@ -65,7 +65,6 @@ Frozen: bool,
## Routines

* `provide_multi_dispute_data(MultiDisputeStatementSet) -> Vec<(SessionIndex, Hash)>`:
1. Fail if any disputes in the set are duplicate or concluded before the `config.dispute_post_conclusion_acceptance_period` window relative to now.
gui1117 marked this conversation as resolved.
Show resolved Hide resolved
1. Pass on each dispute statement set to `provide_dispute_data`, propagating failure.
1. Return a list of all candidates who just had disputes initiated.

Expand All @@ -75,18 +74,18 @@ Frozen: bool,
1. If there is no dispute under `Disputes`, create a new `DisputeState` with blank bitfields.
1. If `concluded_at` is `Some`, and is `concluded_at + config.post_conclusion_acceptance_period < now`, return false.
1. If the overlap of the validators in the `DisputeStatementSet` and those already present in the `DisputeState` is fewer in number than `byzantine_threshold + 1` and the candidate is not present in the `Included` map
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does overlap mean intersection or union here? Either way I don't think either works perfectly.

Assuming the first message is less than the threshold, in the intersection case, the second could be a disjoint set and hence be empty and marked as spam.

Assuming the union, and q-size small subsets (total / q) it's marking only the first q-1 spammers as spammers.

Could elaborate a bit on the intended spam mechanics?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, yes, it should be a union. a dispute is unconfirmed until it has f + 1 participants. The amount of participants can be determined by the union of the 'for' participants and the 'against' participants.

1. increment `SpamSlots` for each validator in the `DisputeStatementSet` which is not already in the `DisputeState`. Initialize the `SpamSlots` to a zeroed vector first, if necessary.
1. If the value for any spam slot exceeds `config.dispute_max_spam_slots`, return false.
1. increment `SpamSlots` for each validator in the `DisputeStatementSet` which is not already in the `DisputeState`. Initialize the `SpamSlots` to a zeroed vector first, if necessary. do not increment `SpamSlots` if the candidate is local.
1. If the value for any spam slot exceeds `config.dispute_max_spam_slots`, return false.
1. If the overlap of the validators in the `DisputeStatementSet` and those already present in the `DisputeState` is at least `byzantine_threshold + 1`, the `DisputeState` has fewer than `byzantine_threshold + 1` validators, and the candidate is not present in the `Included` map, decrement `SpamSlots` for each validator in the `DisputeState`.
1. Import all statements into the dispute. This should fail if any statements are duplicate; if the corresponding bit for the corresponding validator is set in the dispute already.
1. If `concluded_at` is `None`, reward all statements slightly less.
1. Import all statements into the dispute. This should fail if any statements are duplicate or if the corresponding bit for the corresponding validator is set in the dispute already.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Iirc, voting twice for a validator in opposing ways is not an issue, since that would impose slashing itself for sure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Voting twice in opposing ways doesn't necessarily need to lead to a slash as long as it's -EV. The way that disputes are designed now ensures that it is -EV as it raises the likelihood of ending up against a supermajority.

1. If `concluded_at` is `None`, reward all statements.
1. If `concluded_at` is `Some`, reward all statements slightly less.
1. If either side now has supermajority, slash the other side. This may be both sides, and we support this possibility in code, but note that this requires validators to participate on both sides which has negative expected value. Set `concluded_at` to `Some(now)`.
1. If either side now has supermajority and did not previously, slash the other side. This may be both sides, and we support this possibility in code, but note that this requires validators to participate on both sides which has negative expected value. Set `concluded_at` to `Some(now)` if it was `None`.
1. If just concluded against the candidate and the `Included` map contains `(session, candidate)`: invoke `revert_and_freeze` with the stored block number.
1. Return true if just initiated, false otherwise.

* `disputes() -> Vec<(SessionIndex, CandidateHash, DisputeState)>`: Get a list of all disputes and info about dispute state.
1. Iterate over all disputes in `Disputes`. Set the flag according to `concluded`.
1. Iterate over all disputes in `Disputes` and collect into a vector.

* `note_included(SessionIndex, CandidateHash, included_in: BlockNumber)`:
1. Add `(SessionIndex, CandidateHash)` to the `Included` map with `included_in - 1` as the value.
Expand All @@ -97,7 +96,7 @@ Frozen: bool,

* `is_frozen()`: Load the value of `Frozen` from storage.

* `revert_and_freeze(BlockNumber):
* `revert_and_freeze(BlockNumber)`:
1. If `is_frozen()` return.
1. issue a digest in the block header which indicates the chain is to be abandoned back to the stored block number.
1. Set `Frozen` to true.
4 changes: 2 additions & 2 deletions roadmap/implementers-guide/src/types/disputes.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ Kinds of dispute statements. Each of these can be combined with a candidate hash
```rust
enum ValidDisputeStatementKind {
Explicit,
BackingSeconded,
BackingValid,
BackingSeconded(Hash),
drahnr marked this conversation as resolved.
Show resolved Hide resolved
BackingValid(Hash),
ApprovalChecking,
}

Expand Down
8 changes: 8 additions & 0 deletions runtime/kusama/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ use runtime_parachains::hrmp as parachains_hrmp;
use runtime_parachains::scheduler as parachains_scheduler;
use runtime_parachains::reward_points as parachains_reward_points;
use runtime_parachains::runtime_api_impl::v1 as parachains_runtime_api_impl;
use runtime_parachains::disputes as parachains_disputes;

use xcm::v0::{MultiLocation::{self, Null, X1}, NetworkId, BodyId, Xcm, Junction::Parachain};
use xcm::v0::MultiAsset::{self, AllConcreteFungible};
Expand Down Expand Up @@ -1081,6 +1082,12 @@ impl parachains_hrmp::Config for Runtime {
type Currency = Balances;
}

impl parachains_disputes::Config for Runtime {
type Event = Event;
type RewardValidators = ();
type PunishValidators = ();
}

impl parachains_paras_inherent::Config for Runtime {}

impl parachains_scheduler::Config for Runtime {}
Expand Down Expand Up @@ -1458,6 +1465,7 @@ construct_runtime! {
ParasUmp: parachains_ump::{Pallet, Call, Storage} = 59,
ParasHrmp: parachains_hrmp::{Pallet, Call, Storage, Event} = 60,
ParasSessionInfo: parachains_session_info::{Pallet, Call, Storage} = 61,
ParasDisputes: parachains_disputes::{Pallet, Call, Storage, Event} = 62,

// Parachain Onboarding Pallets. Start indices at 70 to leave room.
Registrar: paras_registrar::{Pallet, Call, Storage, Event<T>} = 70,
Expand Down
1 change: 1 addition & 0 deletions runtime/parachains/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ log = { version = "0.4.14", default-features = false }
rustc-hex = { version = "2.1.0", default-features = false }
serde = { version = "1.0.123", features = [ "derive" ], optional = true }
derive_more = "0.99.11"
bitflags = "1"

sp-api = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
inherents = { package = "sp-inherents", git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
Expand Down
Loading