7311 add get headers from peer task #7781

Matilda-Clerke · 2024-10-18T00:16:52Z

PR description

Add the GetHeadersFromPeerTask PeerTask and implement usages

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

…tem' into 7311-add-GetReceiptsFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

…tem' into 7311-add-GetReceiptsFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

…tem' into 7311-add-GetReceiptsFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

…tem' into 7311-add-GetReceiptsFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

…tem' into 7311-add-GetReceiptsFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

…peertask-system' into 7311-add-cli-feature-toggle-for-peertask-system

…tem' into 7311-add-GetReceiptsFromPeerTask

…' into 7311-add-GetReceiptsFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

…tem' into 7311-add-GetReceiptsFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

…Test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

…Results in test classes Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Signed-off-by: Matilda-Clerke <matilda.clerke@consensys.net>

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

…ementFilter Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

...ain/java/org/hyperledger/besu/ethereum/eth/manager/peertask/task/GetHeadersFromPeerTask.java

jframe · 2024-11-28T05:13:56Z

ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/DownloadHeadersStep.java

+                  final AtomicReference<BlockHeader> highestBlockHeader =
+                      new AtomicReference<>(taskResult.result().get().getFirst());
+                  for (BlockHeader blockHeader : taskResult.result().get()) {
+                    if (highestBlockHeader.get().getNumber() < blockHeader.getNumber()) {
+                      highestBlockHeader.set(blockHeader);
+                    }
+                  }
+                  taskResult
+                      .ethPeer()
+                      .ifPresent((peer) -> peer.chainState().update(highestBlockHeader.get()));


Updating the chainstate should be in the peer task so it is updated when anything uses that like in the peer system.

jframe · 2024-11-28T05:29:46Z

.../eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/backwardsync/BackwardSyncStep.java

+                            batchSize,
+                            0,
+                            Direction.REVERSE,
+                            Integer.MAX_VALUE,


Had another look at this I think this is safe after all. Forgot the we are filtering out tried peers in the PeerTaskExecutor.

But from a readability point of view and parity with existing code, I would prefer changing this to use the peer count.

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

…SyncStep, FastSyncActions, and PivotSelectorFromSafeBlock Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

…esult Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

pinges · 2024-12-02T03:13:42Z

...ain/java/org/hyperledger/besu/ethereum/eth/manager/peertask/task/GetHeadersFromPeerTask.java

+  @Override
+  public Predicate<EthPeer> getPeerRequirementFilter() {
+    return (ethPeer) ->
+        protocolSchedule.anyMatch((ps) -> ps.spec().isPoS())


still think that we should calculate that in the constructor, as this does not change, instead of calculating that every time we look for a new peer.

The current code uses a supplier like this, which led me to believe maybe the outcome changes for same reason? Is it safe to just calculate it on start up?

It's safe to calculate on start up. The protocol schedule can only be updated by restarting Besu after modifying the genesis file.

Great, will do

pinges · 2024-12-02T03:25:12Z

...ain/java/org/hyperledger/besu/ethereum/eth/manager/peertask/task/GetHeadersFromPeerTask.java

+  public void postProcessResult(final PeerTaskExecutorResult<List<BlockHeader>> result) {
+    final AtomicReference<BlockHeader> highestBlockHeader =
+        new AtomicReference<>(result.result().get().getFirst());
+    for (BlockHeader blockHeader : result.result().get()) {


we should only have to check the first and the last header, or better, we know which one to check because we know the direction ...

Definitely an optimisation we can make in the future, but for now the aim is to match existing code

pinges · 2024-12-02T03:44:52Z

...m/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/peertask/PeerTaskExecutor.java

@@ -138,43 +141,59 @@ public <T> PeerTaskExecutorResult<T> executeAgainstPeer(
          inflightRequestCountForThisTaskClass.decrementAndGet();
        }

-        if (peerTask.isSuccess(result)) {
+        PeerTaskValidationResponse validationResponse = peerTask.validateResult(result);


up in line 128 we are getting the retries against the same peer, which is always 5(?) If we do have retries against other peers I think we should (maybe) just try the next peer if the result was empty? If we got a partial response it might make sense to ask the same peer?

Individual instances can override the number of retries against different peers. In practice, the old validation code ended up producing partial responses, so I'm reluctant to change the retry scheme without a concrete reason.

pinges · 2024-12-02T03:51:43Z

...in/java/org/hyperledger/besu/ethereum/eth/manager/peertask/task/GetReceiptsFromPeerTask.java

@@ -132,4 +132,8 @@ public Predicate<EthPeer> getPeerRequirementFilter() {
  public boolean isSuccess(final Map<BlockHeader, List<TransactionReceipt>> result) {
    return !result.isEmpty();
  }
+
+  public Collection<BlockHeader> getBlockHeaders() {


probably not ...

# Conflicts: # ethereum/eth/src/jmh/java/org/hyperledger/besu/ethereum/eth/sync/worldstate/WorldStateDownloaderBenchmark.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/EthPeersTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/EthProtocolManagerTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/EthProtocolManagerTestUtil.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/task/WaitForPeerTaskTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/task/WaitForPeersTaskTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/peervalidation/AbstractPeerBlockValidatorTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/peervalidation/DaoForkPeerValidatorTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/peervalidation/PeerValidatorRunnerTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/peervalidation/RequiredBlocksPeerValidatorTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/AbstractBlockPropagationManagerTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/ChainHeadTrackerTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/DownloadHeadersStepTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/RangeHeadersFetcherTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/backwardsync/BackwardSyncContextTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/backwardsync/BackwardSyncStepTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/backwardsync/ForwardSyncStepTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/checkpointsync/CheckPointSyncChainDownloaderTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/fastsync/DownloadReceiptsStepTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/fastsync/FastSyncChainDownloaderTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/fastsync/PivotBlockConfirmerTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/fastsync/worldstate/FastWorldStateDownloaderTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/fullsync/FullSyncChainDownloaderForkTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/fullsync/FullSyncChainDownloaderTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/fullsync/FullSyncChainDownloaderTotalTerminalDifficultyTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/fullsync/FullSyncDownloaderTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/fullsync/FullSyncTargetManagerTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncStateTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/tasks/DetermineCommonAncestorTaskParameterizedTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/tasks/DetermineCommonAncestorTaskTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/tasks/PersistBlockTaskTest.java # ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/AbstractTransactionPoolTestBase.java

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

...m/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/peertask/PeerTaskExecutor.java

jframe · 2024-12-09T05:56:00Z

...ain/java/org/hyperledger/besu/ethereum/eth/manager/peertask/task/GetHeadersFromPeerTask.java

+  @Override
+  public Predicate<EthPeer> getPeerRequirementFilter() {
+    return (ethPeer) ->
+        protocolSchedule.anyMatch((ps) -> ps.spec().isPoS())


It's safe to calculate on start up. The protocol schedule can only be updated by restarting Besu after modifying the genesis file.

...m/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/peertask/PeerTaskExecutor.java

* 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken BesuCommandTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: add class Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move PeerTaskFeatureToggle to more appropriate location Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: add X prefix to peertask-system-enabled Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move --Xpeertask-system-enabled out of BesuCommand and make hidden Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add GetReceiptsFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move isPeerTaskSystemEnabled to SynchronizerOptions Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix javadoc issue Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix javadoc issue Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move PeerTaskFeatureToggleTestHelper to TestUtil and fix RunnerTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove PeerTaskFeatureToggle in favor of including isPeerTaskSystemEnabled in SynchronizerConfiguration Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Adjust to the removal of PeerTaskFeatureToggle and use SynchronizerConfiguration to get the toggle instead Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Reduce timeout in PeerTaskRequestSender to 5s Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Refactor PeerManager to be an interface Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up compile errors after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix MetricsAcceptanceTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix MetricsAcceptanceTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix DownloadReceiptsStep when using peer task system Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename PeerManager to PeerSelector Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Reword PeerSelector javadoc to avoid implementation details Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use ConcurrentHashMap in DefaultPeerSelector Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Reword trace log in DefaultPeerSelector Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove unused imports Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use a 1 second delay between retries in PeerTaskExecutor to match old implementation Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix MetricsAcceptanceTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix MetricsAcceptanceTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Modify PeerTaskExecutor metric to include response time from peer Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use SubProtocol instead of subprotocol name string in PeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: rename timing context to ignored to prevent intellij warnings Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use constants for number of retries Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Convert PeerTaskExecutorResult to a record Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename PeerTaskBehavior to PeerTaskRetryBehavior Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move peer selection logic to PeerSelector Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up everything broken after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Attempt to improve performance of peer task system in pipeline Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: fix compile check Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken workflow Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Reduce logging in JsonRpcExecutor to trace level Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: More changes in DownloadReceiptsStep Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rework DownloadReceiptsStep Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Make changes as discussed in walkthrough meeting Remove DefaultPeerSelector, make EthPeers implement PeerSelector interface, and add PeerTask.getPeerRequirementFilter Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update after merge and make discussed changes from walkthrough discussion Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Change to regular HashMap Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove runtime exception again Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rework PeerTaskExecutor retry system to be 0-based Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up compile errors after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken DownloadReceiptsStepTest test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move GetReceipts to services worker for parallelism Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Refactor peer task system usage in DownloadReceiptsStep to better match old system Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove unused async methods in PeerTaskExecutor Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Return Optional<EthPeer> in PeerSelector.getPeer and utilise existing peer selection behavior in EthPeers Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Redo getPeer again to include hasAvailableRequestCapacity check Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add protocol spec supplier to GetReceiptsFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rework getPeer again to use LEAST_TO_MOST_BUSY comparator Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Import PeerNotConnected class instead of using fully qualified class name Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Change to specifying retry counts in PeerTask instead of behavior enums Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: clean up after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: clean up after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up javadoc Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add additional metrics to PeerTaskExecutor Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add Predicate to PeerTask to check for partial success Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix incorrect name on isPartialSuccessTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement isPartialSuccess and add unit tests Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add partialSuccessCounter and inflightRequestGauge in PeerTaskExecutor Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Also filter by whether a peer is fully validated Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove unneeded throws in RunnerTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up inflight requests gauge in PeerTaskExecutor Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update plugin api hash Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update plugin api hash Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add javadoc to LabelledGauge.isLabelsObserved Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update plugin-api hash Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update changelog Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement GetHeadersFromPeerTask and use in DetermineCommonAncestorTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Handle headers with no receipts as a special case in DownloadReceiptsStep Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Complete merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Get DetermineCommonAncestorTask working with peer task system Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use taskName instead of className for labelNames Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use snake_case for metric names Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use _total metric name suffix Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: rework partial success handling Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update GetReceiptsFromPeerTask with partialSuccess changes Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update GetHeadersFromPeerTask with partialSuccess changes Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add default implementation to LabelledGauge.isLabelsObserved Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use Peer task systems GetHeadersFromPeerTask in GetBlockFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken unit test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove unused constructor from AbstractPeerBlockValidator Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use GetHeadersFromPeerTask in AbstractPeerBlockValidator Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use peer task executor in SyncTargetManager Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix javadoc on BesuControllerBuilder Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove logs used to confirm operation Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement GetHeadersFromPeerTask in FastSyncActions and PivotBlockConfirmer Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename parseResponse to processResponse Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Wrap peer task system usage in ethScheduler call to match other usages Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: apply spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move check for empty trie hash into GetReceiptsFromPeerTask and update unit test to test for this functionality Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix compile issue after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix compile issue after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove BodyValidator and update code and test to match Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement GetHeadersForPeerTask usage in DownloadHeadersStep Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: remove unneeded logs Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up pre-fill and add test to test failure scenario Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use ProtocolSchedule.anyMatch to find if any ProtocolSpecs are PoS, remove new usages of currentProtocolSpecSupplier Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Only attempt to remove headers on successful requests Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: clean up after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: clean up after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use peer task system in RangeHeadersFetcher Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use peer task system in DownloadHeaderSequenceTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix GetHeadersFromPeerTask mocking in CheckPointSyncChainDownloaderTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Extract peer task executor answer for getHeaders to separate class for reuse in tests Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement peer task system usage in BackwardSyncStep Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement peer task system usage in ChainHeadTracker Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement peer task system usage in PivotSelectorFromSafeBlock and improve logging Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement unit test for GetHeadersFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up merge compile error Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Ensure FastSyncActions and PivotSelectorFromSafeBlock retry getting headers for all peers, matching RetryingGetHeaderFromPeerByHashTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Change PeerTaskExecutorResult.ethPeer to an Optional Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use CancellationException instead of InterruptedException in PivotBlockConfirmer Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use PivotBlockRetriever.MAX_QUERY_RETRIES_PER_PEER to set retries for GetHeadersFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add PeerTask.shouldDisconnectPeer and ensure functionality matches old code Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove old info logs Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken test by correctly including peer in PeerTaskExecutorResults in test classes Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix incorrect equality tests Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move PeerTaskExecutor into EthContext to reduce plumbing changes Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove protocol check from GetHeadersFromPeerTask.getPeerRequirementFilter Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken integration test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Refactor peer task validation Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Refactor peer task validation Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use peer count for retry count when getting headers in BackwardSyncStep, FastSyncActions, and PivotSelectorFromSafeBlock Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move chainstate update into GetHeadersFromPeerTask.postProcessResult Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix compile errors Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> --------- Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> Signed-off-by: Matilda-Clerke <matilda.clerke@consensys.net> Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com> Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

Matilda-Clerke and others added 30 commits September 18, 2024 09:22

7311: spotless

2fb2690

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: Fix broken BesuCommandTest

f14aaeb

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…

06553be

…tem' into 7311-add-GetReceiptsFromPeerTask

7311: add class

1ba5184

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: Move PeerTaskFeatureToggle to more appropriate location

f2500dd

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…

0992c35

…tem' into 7311-add-GetReceiptsFromPeerTask

7311: add X prefix to peertask-system-enabled

33b810b

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…

a378d22

…tem' into 7311-add-GetReceiptsFromPeerTask

7311: Move --Xpeertask-system-enabled out of BesuCommand and make hidden

15b6bdf

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…

a9f6714

…tem' into 7311-add-GetReceiptsFromPeerTask

7311: spotless

e3fbc6c

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…

c03bffb

…tem' into 7311-add-GetReceiptsFromPeerTask

Merge branch 'main' into 7311-add-cli-feature-toggle-for-peertask-system

5e8b750

Merge branch 'main' into 7311-add-peertask-foundation-code

e4be5c0

Merge branch 'main' into 7311-add-cli-feature-toggle-for-peertask-system

bc11e0c

7311: Add GetReceiptsFromPeerTask

b2a45c2

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: Move isPeerTaskSystemEnabled to SynchronizerOptions

513b74f

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Merge remote-tracking branch 'origin/7311-add-cli-feature-toggle-for-…

6b86919

…peertask-system' into 7311-add-cli-feature-toggle-for-peertask-system

Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…

645e0e3

…tem' into 7311-add-GetReceiptsFromPeerTask

Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask

4522414

Merge remote-tracking branch 'origin/7311-add-GetReceiptsFromPeerTask…

28fc2cd

…' into 7311-add-GetReceiptsFromPeerTask

7311: Fix javadoc issue

2364ed5

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…

ced19cd

…tem' into 7311-add-GetReceiptsFromPeerTask

7311: Fix javadoc issue

03f6495

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Merge branch 'main' into 7311-add-peertask-foundation-code

5859444

Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask

9a4f3dd

7311: Move PeerTaskFeatureToggleTestHelper to TestUtil and fix Runner…

97e5918

…Test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: spotless

e0f736d

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask

df7f62d

Merge branch 'main' into 7311-add-peertask-foundation-code

c335cbe

Matilda-Clerke added 13 commits November 26, 2024 10:45

7311: Remove old info logs

ddceb7c

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: Fix broken test by correctly including peer in PeerTaskExecutor…

f002ba3

…Results in test classes Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Merge branch 'main' into 7311-add-GetHeadersFromPeerTask

75b091a

7311: Fix incorrect equality tests

371c192

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Merge branch 'main' into 7311-add-GetHeadersFromPeerTask

d320d61

7311: Fix broken test

4618689

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: spotless

dd15253

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: Move PeerTaskExecutor into EthContext to reduce plumbing changes

128d300

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Merge branch 'main' into 7311-add-GetHeadersFromPeerTask

59d4912

Signed-off-by: Matilda-Clerke <matilda.clerke@consensys.net>

7311: spotless

48b224e

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: Remove protocol check from GetHeadersFromPeerTask.getPeerRequir…

bc4d963

…ementFilter Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: Fix broken test

a30f2c4

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: Fix broken integration test

979b761

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

jframe reviewed Nov 28, 2024

View reviewed changes

Matilda-Clerke added 7 commits November 29, 2024 15:42

7311: Refactor peer task validation

e90c003

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: Refactor peer task validation

497e22b

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: Use peer count for retry count when getting headers in Backward…

c683304

…SyncStep, FastSyncActions, and PivotSelectorFromSafeBlock Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: spotless

8efad37

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

Merge branch 'main' into 7311-add-GetHeadersFromPeerTask

639ef87

7311: Move chainstate update into GetHeadersFromPeerTask.postProcessR…

a0d16b0

…esult Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

7311: Fix compile errors

4dd3875

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

pinges reviewed Dec 2, 2024

View reviewed changes

Matilda-Clerke added 2 commits December 3, 2024 09:10

7311: Update after merge

7df7688

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

jframe reviewed Dec 9, 2024

View reviewed changes

jframe approved these changes Dec 11, 2024

View reviewed changes

...m/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/peertask/PeerTaskExecutor.java Show resolved Hide resolved

Merge branch 'main' into 7311-add-GetHeadersFromPeerTask

2fde29f

Matilda-Clerke enabled auto-merge (squash) December 11, 2024 04:26

Matilda-Clerke merged commit 657efff into hyperledger:main Dec 11, 2024
43 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

7311 add get headers from peer task #7781

7311 add get headers from peer task #7781

Matilda-Clerke commented Oct 18, 2024

jframe Nov 28, 2024

jframe Nov 28, 2024

pinges Dec 2, 2024

Matilda-Clerke Dec 2, 2024

jframe Dec 9, 2024

Matilda-Clerke Dec 9, 2024

pinges Dec 2, 2024

Matilda-Clerke Dec 2, 2024

pinges Dec 2, 2024

Matilda-Clerke Dec 2, 2024

pinges Dec 2, 2024

jframe Dec 9, 2024

7311 add get headers from peer task #7781

7311 add get headers from peer task #7781

Conversation

Matilda-Clerke commented Oct 18, 2024

PR description

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment