Network filter for publish and confirm_ack messages #2539
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
guilhermelawless
added
documentation
cryptocode
reviewed
Feb 6, 2020
cryptocode
reviewed
Feb 6, 2020
clemahieu
reviewed
Feb 6, 2020
guilhermelawless
force-pushed
the
stream-filter
branch
from
b83d40d to
70a9fd7
Compare
cryptocode
previously approved these changes
Feb 6, 2020
guilhermelawless
changed the title
clemahieu
previously approved these changes
Feb 6, 2020
cryptocode
previously approved these changes
Feb 6, 2020
|
If a publish message is dropped when the block processor is full, should it be removed from the filter so that it can try to process it the next time it's seen? How would this impact bootstrapping the block if the publish message is dropped but the filter still contains the block? See this line. |
|
That's absolutely missing, thank you @Srayman . |
guilhermelawless
force-pushed
the
stream-filter
branch
3 times, most recently
from
4c0e7bb to
84af7de
Compare
guilhermelawless
changed the title
guilhermelawless
force-pushed
the
stream-filter
branch
2 times, most recently
from
d163eaf to
8ba2d4e
Compare
guilhermelawless
changed the title
guilhermelawless
removed
the
documentation
…hey are deserialized. When they are unique, the digest is saved and passed around to network processing which may drop the messages if the block or vote processor is full. After processing, there are instances where digests have to be cleared from the filter to ensure the original message can be received again: - Cleaning up a long unchecked block erases that element from the publish filter. - Reaching max capacity of the inactive votes cache clears the oldest item from the cache and the confirm_ack filter. For this reason, inactive_votes_cache now stores the digests, and votes pulled from this cache for new elections are now erased. - Interactions between bootstrap frontier confirmation and inactive_votes_cache - An active election is dropped. The vote digest is now saved here as well. Special case if a vote came from inactive_votes_cache, must ensure the digest is passed along. Many tests were added to ensure proper clearing of the filter when blocks and votes are dropped. The blocks_filter has been removed due to redundancy. 16 extra bytes are used for each vote in an (inactive) vote stored in memory.
guilhermelawless
force-pushed
the
stream-filter
branch
from
435d221 to
07328be
Compare
guilhermelawless
changed the title
cryptocode
previously approved these changes
Mar 3, 2020
SergiySW
previously approved these changes
Mar 4, 2020
guilhermelawless
dismissed stale reviews from SergiySW and cryptocode
via
2ee598b
cryptocode
previously approved these changes
Mar 4, 2020
cryptocode
previously approved these changes
Mar 5, 2020
… happy for Rel builds
cryptocode
previously approved these changes
Mar 5, 2020
This reverts commit c56e0ce. Not necessary as the vote goes into inactive votes cache and can later be removed if necessary.
|
Moving with #2643 in V21, filtering votes is left for a future version. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To-do test cases:
Adds duplicate detection of publish and confirm_ack messages before they are deserialized. When they are unique, the digest is saved and passed around to
networkwhich may drop the messages if the block or vote processor is full.After processing, there are instances where digests have to be cleared from the filter to ensure the original message can be received again:
The blocks_filter has been removed due to redundancy. 16 extra bytes are used for each hash in an (inactive) vote stored in memory.