Allow mixed replay descendant + gossip tracking for optimistic confirmation

[AshwinSekar]

Problem

Optimistic confirmation for slot S is aggregated in 2 separate pathways:

• Replay rollup: count votes for slot S and its descendants, compare against threshold
• Cluster rollup: count replay votes and gossip votes for exactly slot S, compare against threshold

There is a potential speedup case that we are missing:

• Count replay votes for slot S + descendants and gossip votes for exactly slot S

This is a potential timesave when there is a competing fork. Imagine a potential scenario:

1. Some validators are in a threshold check, 1/3 of the cluster votes for slot S but the tx sends to a minor fork leader
2. 1/3 of the cluster cannot vote on slot S because of threshold check but votes on slot S + 1 a descendant of S which lands in the major fork leader

Although S has supermajority stake vote, we have to wait until the validators from (1) vote on a descendant of S + 1 which lands in the major fork leader in order to optimistically confirm a block on the fork.
The potential timesave is allowing a mixture of descendants and gossip, as (1) would land in gossip and (2) would land in descendant rollup.

[AshwinSekar]

@carllin following up from your comment #34014 (comment)

The proof https://docs.solana.com/proposals/optimistic_confirmation works with Range(v):

Optimistic Confirmation - A block B is then said to have achieved "optimistic confirmation" if >2/3 of stake have voted with votes v where Range(v) for each such v includes B.slot.

Why is it the case that OC can't be safely rolled up?

[carllin]

We don't have the back pointers implemented for each vote, so the Range(v) is just the vote itself. Thus there's no range of ancestors that you can roll up to.

[AshwinSekar]

I understand that's the case for gossip votes, but for replay votes you have the ancestors available right?

[jstarry]

For the "Replay rollup" pathway are you talking about the vote threshold for lockout depth 8?

[AshwinSekar]

For the "Replay rollup" pathway are you talking about the vote threshold for lockout depth 8?

I'm referring to confirm_forks in replay

solana/core/src/replay_stage.rs

Lines 757 to 779 in 71c1782

	for slot in newly_computed_slot_stats {
	let fork_stats = progress.get_fork_stats(slot).unwrap();
	let confirmed_forks = Self::confirm_forks(
	&tower,
	&fork_stats.voted_stakes,
	fork_stats.total_stake,
	&progress,
	&bank_forks,
	);
	Self::mark_slots_confirmed(
	&confirmed_forks,
	&blockstore,
	&bank_forks,
	&mut progress,
	&mut duplicate_slots_tracker,
	&mut heaviest_subtree_fork_choice,
	&mut epoch_slots_frozen_slots,
	&mut duplicate_slots_to_repair,
	&ancestor_hashes_replay_update_sender,
	&mut purge_repair_slot_counter,
	);
	}

which uses the voted stakes populated by compute bank stats to "rollup" voted stake from descendants to ancestors per replayed fork:

solana/core/src/consensus.rs

Lines 423 to 429 in 71c1782

	// Update all the parents of this last vote with the stake of this vote account
	Self::update_ancestor_voted_stakes(
	&mut voted_stakes,
	vote.slot(),
	voted_stake,
	ancestors,
	);

This allows us to detect optimistically confirmed slots that don't have an individual vote of 67%. Consider the example:

0 - 1 (33%) - 3 (10%) - 4 (30%) - 5
 \- 2

After replaying we can conclude that 1 is optimistically confirmed because the fork descending from it has a total of 33 + 10 + 30 = 73%. However this is not possible through the gossip vote listener which can only aggregate votes for individual slots.

What i'm proposing here is a speedup for the scenario where 33% voted on slot 1 however these vote txs did not land in the fork 3 - 4 - 5 because it was sent to the minority leader who built 2.

0 - 1 (0%) - 3 (10%) - 4 (30%) - 5
 \- 2 (tries to pack votes for 1, but they fail)

The gossip vote listener will tell us that 33% has voted on slot 1. However it is unable to aggregate this with the information that 40% has voted through replay on the fork descending from 1.

I don't think this is that big of a deal as eventually a leader on the majority fork will include these vote txs in a later block or the 33% will vote again/refresh, but just wanted to document it here.

[jstarry]

Ah I see, you're talking about "duplicate confirmed"

[AshwinSekar]

just OC, duplicate confirmed does use the same pathway so maybe a benefit there as well, but was mostly focusing on OC here.