Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
7311 add get headers from peer task (#7781)
* 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken BesuCommandTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: add class Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move PeerTaskFeatureToggle to more appropriate location Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: add X prefix to peertask-system-enabled Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move --Xpeertask-system-enabled out of BesuCommand and make hidden Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add GetReceiptsFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move isPeerTaskSystemEnabled to SynchronizerOptions Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix javadoc issue Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix javadoc issue Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move PeerTaskFeatureToggleTestHelper to TestUtil and fix RunnerTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove PeerTaskFeatureToggle in favor of including isPeerTaskSystemEnabled in SynchronizerConfiguration Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Adjust to the removal of PeerTaskFeatureToggle and use SynchronizerConfiguration to get the toggle instead Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Reduce timeout in PeerTaskRequestSender to 5s Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Refactor PeerManager to be an interface Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up compile errors after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix MetricsAcceptanceTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix MetricsAcceptanceTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix DownloadReceiptsStep when using peer task system Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename PeerManager to PeerSelector Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Reword PeerSelector javadoc to avoid implementation details Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use ConcurrentHashMap in DefaultPeerSelector Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Reword trace log in DefaultPeerSelector Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove unused imports Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use a 1 second delay between retries in PeerTaskExecutor to match old implementation Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix MetricsAcceptanceTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix MetricsAcceptanceTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Modify PeerTaskExecutor metric to include response time from peer Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use SubProtocol instead of subprotocol name string in PeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: rename timing context to ignored to prevent intellij warnings Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use constants for number of retries Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Convert PeerTaskExecutorResult to a record Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename PeerTaskBehavior to PeerTaskRetryBehavior Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move peer selection logic to PeerSelector Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up everything broken after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Attempt to improve performance of peer task system in pipeline Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: fix compile check Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken workflow Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Reduce logging in JsonRpcExecutor to trace level Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: More changes in DownloadReceiptsStep Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rework DownloadReceiptsStep Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Make changes as discussed in walkthrough meeting Remove DefaultPeerSelector, make EthPeers implement PeerSelector interface, and add PeerTask.getPeerRequirementFilter Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update after merge and make discussed changes from walkthrough discussion Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Change to regular HashMap Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove runtime exception again Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rework PeerTaskExecutor retry system to be 0-based Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up compile errors after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken DownloadReceiptsStepTest test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move GetReceipts to services worker for parallelism Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Refactor peer task system usage in DownloadReceiptsStep to better match old system Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove unused async methods in PeerTaskExecutor Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Return Optional<EthPeer> in PeerSelector.getPeer and utilise existing peer selection behavior in EthPeers Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Redo getPeer again to include hasAvailableRequestCapacity check Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add protocol spec supplier to GetReceiptsFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rework getPeer again to use LEAST_TO_MOST_BUSY comparator Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Import PeerNotConnected class instead of using fully qualified class name Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Change to specifying retry counts in PeerTask instead of behavior enums Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: clean up after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: clean up after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up javadoc Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add additional metrics to PeerTaskExecutor Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add Predicate to PeerTask to check for partial success Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix incorrect name on isPartialSuccessTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement isPartialSuccess and add unit tests Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add partialSuccessCounter and inflightRequestGauge in PeerTaskExecutor Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Also filter by whether a peer is fully validated Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove unneeded throws in RunnerTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up inflight requests gauge in PeerTaskExecutor Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update plugin api hash Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update plugin api hash Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add javadoc to LabelledGauge.isLabelsObserved Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update plugin-api hash Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update changelog Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement GetHeadersFromPeerTask and use in DetermineCommonAncestorTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Handle headers with no receipts as a special case in DownloadReceiptsStep Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Complete merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Get DetermineCommonAncestorTask working with peer task system Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use taskName instead of className for labelNames Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use snake_case for metric names Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use _total metric name suffix Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: rework partial success handling Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update GetReceiptsFromPeerTask with partialSuccess changes Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update GetHeadersFromPeerTask with partialSuccess changes Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add default implementation to LabelledGauge.isLabelsObserved Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use Peer task systems GetHeadersFromPeerTask in GetBlockFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken unit test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove unused constructor from AbstractPeerBlockValidator Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use GetHeadersFromPeerTask in AbstractPeerBlockValidator Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use peer task executor in SyncTargetManager Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix javadoc on BesuControllerBuilder Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove logs used to confirm operation Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement GetHeadersFromPeerTask in FastSyncActions and PivotBlockConfirmer Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename parseResponse to processResponse Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Wrap peer task system usage in ethScheduler call to match other usages Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: apply spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move check for empty trie hash into GetReceiptsFromPeerTask and update unit test to test for this functionality Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix compile issue after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix compile issue after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove BodyValidator and update code and test to match Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement GetHeadersForPeerTask usage in DownloadHeadersStep Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: remove unneeded logs Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up pre-fill and add test to test failure scenario Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use ProtocolSchedule.anyMatch to find if any ProtocolSpecs are PoS, remove new usages of currentProtocolSpecSupplier Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Only attempt to remove headers on successful requests Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: clean up after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: clean up after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use peer task system in RangeHeadersFetcher Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use peer task system in DownloadHeaderSequenceTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix GetHeadersFromPeerTask mocking in CheckPointSyncChainDownloaderTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Extract peer task executor answer for getHeaders to separate class for reuse in tests Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement peer task system usage in BackwardSyncStep Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement peer task system usage in ChainHeadTracker Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement peer task system usage in PivotSelectorFromSafeBlock and improve logging Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement unit test for GetHeadersFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up merge compile error Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Ensure FastSyncActions and PivotSelectorFromSafeBlock retry getting headers for all peers, matching RetryingGetHeaderFromPeerByHashTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Change PeerTaskExecutorResult.ethPeer to an Optional Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use CancellationException instead of InterruptedException in PivotBlockConfirmer Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use PivotBlockRetriever.MAX_QUERY_RETRIES_PER_PEER to set retries for GetHeadersFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add PeerTask.shouldDisconnectPeer and ensure functionality matches old code Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove old info logs Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken test by correctly including peer in PeerTaskExecutorResults in test classes Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix incorrect equality tests Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move PeerTaskExecutor into EthContext to reduce plumbing changes Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove protocol check from GetHeadersFromPeerTask.getPeerRequirementFilter Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken integration test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Refactor peer task validation Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Refactor peer task validation Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use peer count for retry count when getting headers in BackwardSyncStep, FastSyncActions, and PivotSelectorFromSafeBlock Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move chainstate update into GetHeadersFromPeerTask.postProcessResult Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix compile errors Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> --------- Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> Signed-off-by: Matilda-Clerke <matilda.clerke@consensys.net> Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
- Loading branch information