This repository has been archived by the owner on Aug 23, 2020. It is now read-only.
Update Broadcast Stage logic #1827
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Comment on lines
+61
to
+68
| // Check the transaction solidifier to see if there are solid transactions that need to be broadcast. | ||
| // If so, forward them to the BroadcastStageQueue to be processed. | ||
| TransactionViewModel transactionToBroadcast; | ||
| if((transactionToBroadcast = transactionSolidifier.getNextTxInBroadcastQueue()) != null){ | ||
| ctx.setNextStage(TransactionProcessingPipeline.Stage.BROADCAST); | ||
| ctx.setPayload(new BroadcastPayload(payload.getOriginNeighbor(), transactionToBroadcast)); | ||
| return ctx; | ||
| } |
There was a problem hiding this comment.
hmm I thought you were going to do a while loop
I guess what you did here works as well
Not sure about performance what is better?
Did you test?
There was a problem hiding this comment.
I did test it in ICC and there were no issues with this approach. Originally I was going to do a while, but I was worried about holding up the forwarding of received transactions. With this approach it will mesh the received and the solidified transactions in the queue for the broadcasts.
GalRogozinski
approved these changes
Apr 7, 2020
acha-bill
pushed a commit
to acha-bill/iri
that referenced
this pull request
Apr 14, 2020
* Update broadcast fetch logic * Comment
GalRogozinski
added a commit
that referenced
this pull request
Apr 21, 2020
* Move validation to validation service package * Add Transaction Solidifier * Update Transaction Validator * Update Unit Tests * Remove tip field from solidify stage * Move broadcast queue retreival to solidify stage * Undo auto-formatting * More autoformatting * Re-remove refillBroadcast * Move propagation logic to inner class * Remove unused imports * Add comment to propagator * Remove unused txSolidifier private field * Remove separate thread logic, check solidity from milestone solidifier * Add solidification queue to transaction solidifier * LinkedHashSet -> ArrayDeque in checkSolidity * Update maxProcessedTransactions value determination * Make Transaction Propagator private * Add milestone stage * add heartbeat message (#1825) * add heartbeat message * added javadocs * print exception message * Regression: Update pruning test (#1775) * Update pruning test for before and after pruning status * m6 -> m1 Co-authored-by: Gal Rogozinski <gal.rogozinski@iota.org> * Fix: Update Broadcast Stage logic (#1827) * Update broadcast fetch logic * Comment * Feature: Add transaction solidifier queue (#1821) * Move validation to validation service package * Add Transaction Solidifier * Update Transaction Validator * Update Unit Tests * Remove tip field from solidify stage * Move broadcast queue retreival to solidify stage * Undo auto-formatting * More autoformatting * Re-remove refillBroadcast * Move propagation logic to inner class * Remove unused imports * Add comment to propagator * Remove unused txSolidifier private field * Remove separate thread logic, check solidity from milestone solidifier * Add solidification queue to transaction solidifier * LinkedHashSet -> ArrayDeque in checkSolidity * Update maxProcessedTransactions value determination * Make Transaction Propagator private * Fix test conflicts * Update src/main/java/com/iota/iri/service/validation/impl/TransactionSolidifierImpl.java Co-Authored-By: Gal Rogozinski <galrogogit@gmail.com> * Update src/main/java/com/iota/iri/service/validation/impl/TransactionSolidifierImpl.java Co-Authored-By: Gal Rogozinski <galrogogit@gmail.com> * Rename Co-authored-by: Gal Rogozinski <gal.rogozinski@iota.org> Co-authored-by: Gal Rogozinski <galrogogit@gmail.com> * Unit test for milestone stage Co-authored-by: Gal Rogozinski <gal.rogozinski@iota.org> Co-authored-by: Acha Bill <57879913+acha-bill@users.noreply.github.com> Co-authored-by: Gal Rogozinski <galrogogit@gmail.com>
GalRogozinski
added a commit
that referenced
this pull request
Apr 23, 2020
* Move validation to validation service package * Add Transaction Solidifier * Update Transaction Validator * Update Unit Tests * Remove tip field from solidify stage * Move broadcast queue retreival to solidify stage * Undo auto-formatting * More autoformatting * Re-remove refillBroadcast * Move propagation logic to inner class * Remove unused imports * Add comment to propagator * Remove unused txSolidifier private field * Remove separate thread logic, check solidity from milestone solidifier * Add solidification queue to transaction solidifier * LinkedHashSet -> ArrayDeque in checkSolidity * Update maxProcessedTransactions value determination * Make Transaction Propagator private * Add milestone stage * add heartbeat message (#1825) * add heartbeat message * added javadocs * print exception message * Regression: Update pruning test (#1775) * Update pruning test for before and after pruning status * m6 -> m1 Co-authored-by: Gal Rogozinski <gal.rogozinski@iota.org> * Fix: Update Broadcast Stage logic (#1827) * Update broadcast fetch logic * Comment * Feature: Add transaction solidifier queue (#1821) * Move validation to validation service package * Add Transaction Solidifier * Update Transaction Validator * Update Unit Tests * Remove tip field from solidify stage * Move broadcast queue retreival to solidify stage * Undo auto-formatting * More autoformatting * Re-remove refillBroadcast * Move propagation logic to inner class * Remove unused imports * Add comment to propagator * Remove unused txSolidifier private field * Remove separate thread logic, check solidity from milestone solidifier * Add solidification queue to transaction solidifier * LinkedHashSet -> ArrayDeque in checkSolidity * Update maxProcessedTransactions value determination * Make Transaction Propagator private * Fix test conflicts * Update src/main/java/com/iota/iri/service/validation/impl/TransactionSolidifierImpl.java Co-Authored-By: Gal Rogozinski <galrogogit@gmail.com> * Update src/main/java/com/iota/iri/service/validation/impl/TransactionSolidifierImpl.java Co-Authored-By: Gal Rogozinski <galrogogit@gmail.com> * Rename Co-authored-by: Gal Rogozinski <gal.rogozinski@iota.org> Co-authored-by: Gal Rogozinski <galrogogit@gmail.com> * Unit test for milestone stage * Consolidate milestone solidification logic * Update milestone stage tests * update bootstrap * Minor refactor * Log correct milestones, add switch for processing solidification queue * Bootstrap milestone hash and index properly * set index and hash directly instead of through method * Place all milestone tails into solidifier from milestone stage * setLatestMilestone -> logNewMilestone * remove second LSM update in update milestones * Stop repeats and solid entry points in broadcast queue * Don't abort milestone stage for second transaction in milestone bundle * Shorten regression wait times * Clean up, syntax updates, comment clarification * Remove unecessary method, syntax * Atomic variables and syntax changes * Update scanMilestones in queue * Extract repair logic * Update src/main/java/com/iota/iri/service/milestone/MilestoneSolidifier.java Co-Authored-By: Gal Rogozinski <galrogogit@gmail.com> * Typo, check solidity in milestone stage * syntax and minor logic change * Rename logNewMilestone * log -> register in milestone stage * Update milestone payload to pass forward transaction, fix tests Co-authored-by: Gal Rogozinski <gal.rogozinski@iota.org> Co-authored-by: Acha Bill <57879913+acha-bill@users.noreply.github.com> Co-authored-by: Gal Rogozinski <galrogogit@gmail.com>
Open
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
When the transaction solidifier was introduced, we updated how transactions are passed through to broadcast, and under initial review we opted to fetch broadcast transactions at the end of the solidification stage, to pass them forward to the broadcast stage in the event the original solidifying transaction is not solid. This sequential loading of transactions from the broadcast queue does not forward transactions fast enough. To correct this, the logic has been moved to the broadcast stage instead. The broadcast stage will complete the transaction forwarding as it was previously designed to do, and the broadcast stage will check if a transaction still exists to broadcast, and if there is, it will pass it to the broadcast stage queue until there is nothing left to pull out of the set.
Fixes an error that occurred in high tps environments that would cause a delay in the forwarding of transactions via broadcast
Type of change
How Has This Been Tested?
Using the same topology that the problem was first observed in (within the ICC network) transactions, specifically milestone transactions, are being forwarded in a timely fashion.
Checklist: