expands number of erasure coding shreds in the last batch in slots (backport #16484) #16707

mergify · 2021-04-21T12:51:40Z

This is an automatic backport of pull request #16484 done by Mergify.

Cherry-pick of 37b8587 has failed:

On branch mergify/bp/v1.6/pr-16484
Your branch is up to date with 'origin/v1.6'.

You are currently cherry-picking commit 37b8587d4.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   core/benches/retransmit_stage.rs
	modified:   core/src/broadcast_stage.rs
	modified:   core/src/broadcast_stage/broadcast_fake_shreds_run.rs
	modified:   core/src/broadcast_stage/fail_entry_verification_broadcast_run.rs
	modified:   core/src/broadcast_stage/standard_broadcast_run.rs
	modified:   core/src/cluster_info.rs
	modified:   core/src/duplicate_shred.rs
	modified:   core/src/shred_fetch_stage.rs
	modified:   core/src/window_service.rs
	modified:   ledger/src/blockstore.rs
	modified:   ledger/src/erasure.rs
	modified:   ledger/tests/shred.rs

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   core/benches/shredder.rs
	both modified:   ledger/src/shred.rs

To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

@Mergifyio refresh will re-evaluate the rules
@Mergifyio rebase will rebase this PR on its base branch
@Mergifyio update will merge the base branch into this PR
@Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

look at your merge queues
generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.io/

…16484) Number of parity coding shreds is always less than the number of data shreds in FEC blocks: https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L719 Data shreds are batched in chunks of 32 shreds each: https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L714 However the very last batch of data shreds in a slot can be small, in which case the loss rate can be exacerbated. This commit expands the number of coding shreds in the last FEC block in slots to: 64 - number of data shreds; so that FEC blocks are always 64 data and parity coding shreds each. As a consequence of this, the last FEC block has more parity coding shreds than data shreds. So for some shred indices we will have a coding shred but no data shreds. This should not cause any kind of overlapping FEC blocks as in: #10095 since this is done only for the very last batch in a slot, and the next slot will reset the shred index. (cherry picked from commit 37b8587) # Conflicts: # core/benches/shredder.rs # ledger/src/shred.rs

mergify · 2021-04-21T12:52:24Z