Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent block bloat when purging witness data #2103

Closed
dandanlen opened this issue Aug 30, 2022 · 0 comments · Fixed by #2117
Closed

Prevent block bloat when purging witness data #2103

dandanlen opened this issue Aug 30, 2022 · 0 comments · Fixed by #2117
Assignees
Labels
effort-3 Will need to plan this before implementing State Chain

Comments

@dandanlen
Copy link
Collaborator

Description

See the discission here and this substrate PR.

We currently have the following in the witnesser pallet (soon-to-merged substrate update branch):

impl<T: pallet::Config> cf_traits::EpochTransitionHandler for Pallet<T> {
	type ValidatorId = T::ValidatorId;

	/// Purge the pallet storage of stale entries. This is prevent the storage from growing
	/// indefinitely.
	fn on_expired_epoch(expired: EpochIndex) {
		let _empty = Votes::<T>::clear_prefix(expired, u32::MAX, None);
		let _empty = ExtraCallData::<T>::clear_prefix(expired, u32::MAX, None);
		let _empty = CallHashExecuted::<T>::clear_prefix(expired, u32::MAX, None);
	}
}

This will likely be fine initially but we should be more defensive: each of these items is potentially unbounded and will be larger as our chain is more active. Deleting them all at once is risky.

Either we use the strange new substrate api using MultiRemovalResults, or we use a similar 'garbage collection' approach to the one used for reaping dust accounts in the Flip pallet. Or some hybrid of the two approaches. I don't fully understand the reason behind / need for / proper usage of the cursor in MultiRemovalResults - will require some reading to make the right decision.

Relevant people

@syan095

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort-3 Will need to plan this before implementing State Chain
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants