Gossipsub: ignore duplicate publish errors

[twoeths]

Motivation

Got PublishError.Duplicate errors at validator side even through it's a success. There are 2 reasons for it:

• when publishing a block using buider flow, builder may already published that block and gossip block reached our node (see Validator: wrong publish error log #5140)
• when vc works with more than 1 beacon nodes, an item may reach one node first and spread across the network through gossipsub, when the other node calls gossipsub.publish() it throws PublishError.Duplicate error because it sees that item already through gossipsub (see v1.4.0: submitPoolAttestations errors #5098)

Description

• ignoreDuplicatePublishError option was added to gossipsub through https://github.com/ChainSafe/js-libp2p-gossipsub/pull/404/files and included in gossipsub v6.2.0
• Apply that to all publish*() APIs that come from vc

Closes #5140 #5098

[github-actions]

Performance Report

✔️ no performance regression detected

Full benchmark results

Benchmark suite	Current: 990c749	Previous: f1b8c5f	Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc	573.52 us/op	848.45 us/op	0.68
getPubkeys - validatorsArr - req 1000 vs - 250000 vc	48.528 us/op	53.231 us/op	0.91
BLS verify - blst-native	1.2413 ms/op	1.2328 ms/op	1.01
BLS verifyMultipleSignatures 3 - blst-native	2.5339 ms/op	2.5074 ms/op	1.01
BLS verifyMultipleSignatures 8 - blst-native	5.4393 ms/op	5.4075 ms/op	1.01
BLS verifyMultipleSignatures 32 - blst-native	19.668 ms/op	19.580 ms/op	1.00
BLS aggregatePubkeys 32 - blst-native	26.127 us/op	26.469 us/op	0.99
BLS aggregatePubkeys 128 - blst-native	101.69 us/op	101.82 us/op	1.00
getAttestationsForBlock	59.909 ms/op	60.315 ms/op	0.99
isKnown best case - 1 super set check	294.00 ns/op	278.00 ns/op	1.06
isKnown normal case - 2 super set checks	279.00 ns/op	277.00 ns/op	1.01
isKnown worse case - 16 super set checks	284.00 ns/op	274.00 ns/op	1.04
CheckpointStateCache - add get delete	6.0280 us/op	5.3390 us/op	1.13
validate gossip signedAggregateAndProof - struct	2.8503 ms/op	2.8420 ms/op	1.00
validate gossip attestation - struct	1.3721 ms/op	1.3454 ms/op	1.02
pickEth1Vote - no votes	1.3597 ms/op	1.3653 ms/op	1.00
pickEth1Vote - max votes	9.1357 ms/op	11.995 ms/op	0.76
pickEth1Vote - Eth1Data hashTreeRoot value x2048	8.8413 ms/op	9.5803 ms/op	0.92
pickEth1Vote - Eth1Data hashTreeRoot tree x2048	14.865 ms/op	15.806 ms/op	0.94
pickEth1Vote - Eth1Data fastSerialize value x2048	834.13 us/op	744.46 us/op	1.12
pickEth1Vote - Eth1Data fastSerialize tree x2048	5.3179 ms/op	7.2013 ms/op	0.74
bytes32 toHexString	629.00 ns/op	551.00 ns/op	1.14
bytes32 Buffer.toString(hex)	417.00 ns/op	404.00 ns/op	1.03
bytes32 Buffer.toString(hex) from Uint8Array	621.00 ns/op	626.00 ns/op	0.99
bytes32 Buffer.toString(hex) + 0x	400.00 ns/op	432.00 ns/op	0.93
Object access 1 prop	0.20300 ns/op	0.19600 ns/op	1.04
Map access 1 prop	0.16800 ns/op	0.16100 ns/op	1.04
Object get x1000	6.8470 ns/op	6.9290 ns/op	0.99
Map get x1000	0.68300 ns/op	0.67400 ns/op	1.01
Object set x1000	64.640 ns/op	59.324 ns/op	1.09
Map set x1000	56.961 ns/op	51.997 ns/op	1.10
Return object 10000 times	0.26070 ns/op	0.24740 ns/op	1.05
Throw Error 10000 times	4.7929 us/op	4.3189 us/op	1.11
fastMsgIdFn sha256 / 200 bytes	3.8600 us/op	3.6050 us/op	1.07
fastMsgIdFn h32 xxhash / 200 bytes	331.00 ns/op	307.00 ns/op	1.08
fastMsgIdFn h64 xxhash / 200 bytes	492.00 ns/op	454.00 ns/op	1.08
fastMsgIdFn sha256 / 1000 bytes	12.254 us/op	11.838 us/op	1.04
fastMsgIdFn h32 xxhash / 1000 bytes	461.00 ns/op	426.00 ns/op	1.08
fastMsgIdFn h64 xxhash / 1000 bytes	546.00 ns/op	506.00 ns/op	1.08
fastMsgIdFn sha256 / 10000 bytes	106.15 us/op	104.69 us/op	1.01
fastMsgIdFn h32 xxhash / 10000 bytes	2.0920 us/op	1.9880 us/op	1.05
fastMsgIdFn h64 xxhash / 10000 bytes	1.5120 us/op	1.4630 us/op	1.03
enrSubnets - fastDeserialize 64 bits	1.8420 us/op	1.4040 us/op	1.31
enrSubnets - ssz BitVector 64 bits	657.00 ns/op	541.00 ns/op	1.21
enrSubnets - fastDeserialize 4 bits	219.00 ns/op	211.00 ns/op	1.04
enrSubnets - ssz BitVector 4 bits	699.00 ns/op	570.00 ns/op	1.23
prioritizePeers score -10:0 att 32-0.1 sync 2-0	110.90 us/op	107.93 us/op	1.03
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25	134.89 us/op	131.37 us/op	1.03
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5	214.57 us/op	186.52 us/op	1.15
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75	384.80 us/op	339.09 us/op	1.13
prioritizePeers score 0:0 att 64-1 sync 4-1	440.89 us/op	400.52 us/op	1.10
array of 16000 items push then shift	1.7259 us/op	1.7134 us/op	1.01
LinkedList of 16000 items push then shift	9.5210 ns/op	9.2640 ns/op	1.03
array of 16000 items push then pop	121.45 ns/op	108.06 ns/op	1.12
LinkedList of 16000 items push then pop	9.1190 ns/op	8.9980 ns/op	1.01
array of 24000 items push then shift	2.4504 us/op	2.4382 us/op	1.00
LinkedList of 24000 items push then shift	9.9860 ns/op	9.0700 ns/op	1.10
array of 24000 items push then pop	88.949 ns/op	83.356 ns/op	1.07
LinkedList of 24000 items push then pop	9.0940 ns/op	8.9380 ns/op	1.02
intersect bitArray bitLen 8	13.826 ns/op	13.630 ns/op	1.01
intersect array and set length 8	104.35 ns/op	88.599 ns/op	1.18
intersect bitArray bitLen 128	46.209 ns/op	44.592 ns/op	1.04
intersect array and set length 128	1.3269 us/op	1.1766 us/op	1.13
Buffer.concat 32 items	2.8910 us/op	3.0150 us/op	0.96
Uint8Array.set 32 items	2.9760 us/op	2.5160 us/op	1.18
pass gossip attestations to forkchoice per slot	2.4708 ms/op	2.3785 ms/op	1.04
computeDeltas	3.1053 ms/op	2.9615 ms/op	1.05
computeProposerBoostScoreFromBalances	1.8272 ms/op	1.8030 ms/op	1.01
altair processAttestation - 250000 vs - 7PWei normalcase	2.6570 ms/op	2.1806 ms/op	1.22
altair processAttestation - 250000 vs - 7PWei worstcase	3.6682 ms/op	4.4699 ms/op	0.82
altair processAttestation - setStatus - 1/6 committees join	145.75 us/op	147.92 us/op	0.99
altair processAttestation - setStatus - 1/3 committees join	296.00 us/op	280.99 us/op	1.05
altair processAttestation - setStatus - 1/2 committees join	394.31 us/op	386.25 us/op	1.02
altair processAttestation - setStatus - 2/3 committees join	495.57 us/op	484.06 us/op	1.02
altair processAttestation - setStatus - 4/5 committees join	726.77 us/op	674.81 us/op	1.08
altair processAttestation - setStatus - 100% committees join	803.51 us/op	787.12 us/op	1.02
altair processBlock - 250000 vs - 7PWei normalcase	19.943 ms/op	19.310 ms/op	1.03
altair processBlock - 250000 vs - 7PWei normalcase hashState	31.079 ms/op	26.925 ms/op	1.15
altair processBlock - 250000 vs - 7PWei worstcase	57.640 ms/op	51.591 ms/op	1.12
altair processBlock - 250000 vs - 7PWei worstcase hashState	73.592 ms/op	73.810 ms/op	1.00
phase0 processBlock - 250000 vs - 7PWei normalcase	2.5103 ms/op	2.1923 ms/op	1.15
phase0 processBlock - 250000 vs - 7PWei worstcase	31.655 ms/op	29.166 ms/op	1.09
altair processEth1Data - 250000 vs - 7PWei normalcase	574.97 us/op	518.56 us/op	1.11
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15	10.303 us/op	11.245 us/op	0.92
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219	30.707 us/op	30.520 us/op	1.01
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42	11.782 us/op	10.526 us/op	1.12
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18	9.8760 us/op	8.0980 us/op	1.22
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020	119.79 us/op	105.19 us/op	1.14
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777	678.24 us/op	726.55 us/op	0.93
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384	925.35 us/op	915.06 us/op	1.01
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384	930.30 us/op	894.38 us/op	1.04
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384	2.4204 ms/op	2.3781 ms/op	1.02
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384	1.7381 ms/op	1.7138 ms/op	1.01
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384	4.4503 ms/op	3.9885 ms/op	1.12
Tree 40 250000 create	399.27 ms/op	316.17 ms/op	1.26
Tree 40 250000 get(125000)	215.52 ns/op	198.20 ns/op	1.09
Tree 40 250000 set(125000)	1.8916 us/op	1.0324 us/op	1.83
Tree 40 250000 toArray()	28.713 ms/op	19.877 ms/op	1.44
Tree 40 250000 iterate all - toArray() + loop	24.095 ms/op	20.137 ms/op	1.20
Tree 40 250000 iterate all - get(i)	84.983 ms/op	70.043 ms/op	1.21
MutableVector 250000 create	12.530 ms/op	10.092 ms/op	1.24
MutableVector 250000 get(125000)	7.0870 ns/op	6.3760 ns/op	1.11
MutableVector 250000 set(125000)	602.36 ns/op	254.27 ns/op	2.37
MutableVector 250000 toArray()	4.7843 ms/op	2.9913 ms/op	1.60
MutableVector 250000 iterate all - toArray() + loop	5.2370 ms/op	3.0209 ms/op	1.73
MutableVector 250000 iterate all - get(i)	1.6571 ms/op	1.5840 ms/op	1.05
Array 250000 create	4.2770 ms/op	2.5910 ms/op	1.65
Array 250000 clone - spread	1.2930 ms/op	1.1358 ms/op	1.14
Array 250000 get(125000)	0.60100 ns/op	0.58400 ns/op	1.03
Array 250000 set(125000)	0.68100 ns/op	0.65700 ns/op	1.04
Array 250000 iterate all - loop	90.685 us/op	84.291 us/op	1.08
effectiveBalanceIncrements clone Uint8Array 300000	55.317 us/op	29.511 us/op	1.87
effectiveBalanceIncrements clone MutableVector 300000	373.00 ns/op	371.00 ns/op	1.01
effectiveBalanceIncrements rw all Uint8Array 300000	179.68 us/op	170.03 us/op	1.06
effectiveBalanceIncrements rw all MutableVector 300000	99.043 ms/op	84.710 ms/op	1.17
phase0 afterProcessEpoch - 250000 vs - 7PWei	119.87 ms/op	116.74 ms/op	1.03
phase0 beforeProcessEpoch - 250000 vs - 7PWei	47.351 ms/op	39.227 ms/op	1.21
altair processEpoch - mainnet_e81889	311.05 ms/op	333.96 ms/op	0.93
mainnet_e81889 - altair beforeProcessEpoch	53.836 ms/op	68.056 ms/op	0.79
mainnet_e81889 - altair processJustificationAndFinalization	19.146 us/op	18.533 us/op	1.03
mainnet_e81889 - altair processInactivityUpdates	6.3952 ms/op	5.7018 ms/op	1.12
mainnet_e81889 - altair processRewardsAndPenalties	72.482 ms/op	71.585 ms/op	1.01
mainnet_e81889 - altair processRegistryUpdates	3.9330 us/op	2.5780 us/op	1.53
mainnet_e81889 - altair processSlashings	775.00 ns/op	577.00 ns/op	1.34
mainnet_e81889 - altair processEth1DataReset	770.00 ns/op	1.0110 us/op	0.76
mainnet_e81889 - altair processEffectiveBalanceUpdates	1.3810 ms/op	1.2751 ms/op	1.08
mainnet_e81889 - altair processSlashingsReset	4.7300 us/op	7.1350 us/op	0.66
mainnet_e81889 - altair processRandaoMixesReset	5.3710 us/op	8.3560 us/op	0.64
mainnet_e81889 - altair processHistoricalRootsUpdate	723.00 ns/op	963.00 ns/op	0.75
mainnet_e81889 - altair processParticipationFlagUpdates	3.6790 us/op	3.0820 us/op	1.19
mainnet_e81889 - altair processSyncCommitteeUpdates	638.00 ns/op	1.1130 us/op	0.57
mainnet_e81889 - altair afterProcessEpoch	132.59 ms/op	132.07 ms/op	1.00
phase0 processEpoch - mainnet_e58758	397.02 ms/op	393.56 ms/op	1.01
mainnet_e58758 - phase0 beforeProcessEpoch	151.26 ms/op	163.27 ms/op	0.93
mainnet_e58758 - phase0 processJustificationAndFinalization	20.805 us/op	18.298 us/op	1.14
mainnet_e58758 - phase0 processRewardsAndPenalties	69.586 ms/op	68.510 ms/op	1.02
mainnet_e58758 - phase0 processRegistryUpdates	10.577 us/op	8.4360 us/op	1.25
mainnet_e58758 - phase0 processSlashings	623.00 ns/op	502.00 ns/op	1.24
mainnet_e58758 - phase0 processEth1DataReset	536.00 ns/op	498.00 ns/op	1.08
mainnet_e58758 - phase0 processEffectiveBalanceUpdates	1.0551 ms/op	1.0261 ms/op	1.03
mainnet_e58758 - phase0 processSlashingsReset	4.3430 us/op	4.0700 us/op	1.07
mainnet_e58758 - phase0 processRandaoMixesReset	4.9830 us/op	5.0540 us/op	0.99
mainnet_e58758 - phase0 processHistoricalRootsUpdate	633.00 ns/op	655.00 ns/op	0.97
mainnet_e58758 - phase0 processParticipationRecordUpdates	4.5010 us/op	4.4450 us/op	1.01
mainnet_e58758 - phase0 afterProcessEpoch	102.19 ms/op	99.381 ms/op	1.03
phase0 processEffectiveBalanceUpdates - 250000 normalcase	1.2937 ms/op	1.2566 ms/op	1.03
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5	1.5527 ms/op	1.6494 ms/op	0.94
altair processInactivityUpdates - 250000 normalcase	28.769 ms/op	26.821 ms/op	1.07
altair processInactivityUpdates - 250000 worstcase	28.282 ms/op	29.907 ms/op	0.95
phase0 processRegistryUpdates - 250000 normalcase	7.5880 us/op	6.4110 us/op	1.18
phase0 processRegistryUpdates - 250000 badcase_full_deposits	278.43 us/op	271.61 us/op	1.03
phase0 processRegistryUpdates - 250000 worstcase 0.5	147.79 ms/op	127.59 ms/op	1.16
altair processRewardsAndPenalties - 250000 normalcase	76.467 ms/op	70.922 ms/op	1.08
altair processRewardsAndPenalties - 250000 worstcase	80.962 ms/op	70.203 ms/op	1.15
phase0 getAttestationDeltas - 250000 normalcase	9.5924 ms/op	6.8454 ms/op	1.40
phase0 getAttestationDeltas - 250000 worstcase	9.2816 ms/op	6.7463 ms/op	1.38
phase0 processSlashings - 250000 worstcase	3.8238 ms/op	3.5313 ms/op	1.08
altair processSyncCommitteeUpdates - 250000	190.19 ms/op	180.01 ms/op	1.06
BeaconState.hashTreeRoot - No change	305.00 ns/op	272.00 ns/op	1.12
BeaconState.hashTreeRoot - 1 full validator	55.571 us/op	55.574 us/op	1.00
BeaconState.hashTreeRoot - 32 full validator	500.61 us/op	532.16 us/op	0.94
BeaconState.hashTreeRoot - 512 full validator	6.1352 ms/op	5.4378 ms/op	1.13
BeaconState.hashTreeRoot - 1 validator.effectiveBalance	67.728 us/op	64.288 us/op	1.05
BeaconState.hashTreeRoot - 32 validator.effectiveBalance	1.0196 ms/op	920.66 us/op	1.11
BeaconState.hashTreeRoot - 512 validator.effectiveBalance	12.922 ms/op	12.156 ms/op	1.06
BeaconState.hashTreeRoot - 1 balances	55.240 us/op	53.909 us/op	1.02
BeaconState.hashTreeRoot - 32 balances	479.96 us/op	489.94 us/op	0.98
BeaconState.hashTreeRoot - 512 balances	4.9914 ms/op	4.5711 ms/op	1.09
BeaconState.hashTreeRoot - 250000 balances	81.192 ms/op	74.140 ms/op	1.10
aggregationBits - 2048 els - zipIndexesInBitList	20.406 us/op	16.131 us/op	1.27
regular array get 100000 times	37.240 us/op	33.763 us/op	1.10
wrappedArray get 100000 times	35.024 us/op	33.373 us/op	1.05
arrayWithProxy get 100000 times	16.513 ms/op	16.150 ms/op	1.02
ssz.Root.equals	590.00 ns/op	544.00 ns/op	1.08
byteArrayEquals	607.00 ns/op	540.00 ns/op	1.12
shuffle list - 16384 els	7.1592 ms/op	6.9586 ms/op	1.03
shuffle list - 250000 els	105.13 ms/op	102.52 ms/op	1.03
processSlot - 1 slots	9.6150 us/op	8.4830 us/op	1.13
processSlot - 32 slots	1.5108 ms/op	1.3217 ms/op	1.14
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei	227.46 us/op	192.30 us/op	1.18
getCommitteeAssignments - req 1 vs - 250000 vc	3.0271 ms/op	2.9487 ms/op	1.03
getCommitteeAssignments - req 100 vs - 250000 vc	4.2747 ms/op	4.1904 ms/op	1.02
getCommitteeAssignments - req 1000 vs - 250000 vc	4.6042 ms/op	4.4937 ms/op	1.02
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei	5.3000 ns/op	4.7600 ns/op	1.11
state getBlockRootAtSlot - 250000 vs - 7PWei	946.84 ns/op	549.02 ns/op	1.72
computeProposers - vc 250000	11.507 ms/op	10.433 ms/op	1.10
computeEpochShuffling - vc 250000	105.93 ms/op	102.27 ms/op	1.04
getNextSyncCommittee - vc 250000	195.44 ms/op	173.16 ms/op	1.13

by benchmarkbot/action

[nflaig]

is it intended that c-kzg is added in yarn.lock? I noticed it was showing up as a diff when running yarn install locally, this is fixed but this change seems unrelated to the PR

[wemeetagain]

yeah, its unrelated to this PR

[wemeetagain]

🎉 This PR is included in v1.6.0 🎉