This repository has been archived by the owner on Feb 3, 2023. It is now read-only.
Cache validation packages #2202
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zippy
changed the title
neonphog
reviewed
Jul 8, 2020
neonphog
reviewed
Jul 8, 2020
| ValidationCacheData { | ||
| entries: validation.source_chain_entries.clone(), | ||
| headers: validation.source_chain_headers.as_ref().unwrap().clone(), | ||
| cached_at: SystemTime::now(), |
There was a problem hiding this comment.
Are we cool keeping these things cached forever? cached_at doesn't seem to ever be read.
There was a problem hiding this comment.
I'm not persisting this data, so it will be cleared on reboot, which is fine I think because in that case the chain in the cache is probably pretty stale and will need to be fetched anyways. I think he only real reason to dump the cache is because memory gets tight...
neonphog
approved these changes
Jul 8, 2020
Co-authored-by: David Braden <neonphog@gmail.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR summary
This PR adds caching of FullChain ValidationPackages received from agents so that on subsequent attempts get data from the same agent we can skip the network roud-trip. This is especially important when nodes first come on any may have to hold lots of data from a given node. This can drastically reduce the number of network round-trips.
testing/benchmarking notes
N.B. App-spec failing
This PR includes changing the sim2h RETRY_FETCH_MISSING_ASPECTS_INTERVAL_MS from .5s to 30 seconds. This is necessary for systemic health (especially as new nodes come online), but causes a number of app-spec tests to fail due to timing issues.
followups
( any new tickets/concerns that were discovered or created during this work but aren't in scope for review here )
changelog
documentation