FastSync with skipHistory fixes #1060
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
zilm13
commented
Apr 25, 2018
zilm13
commented
- modified structure of headers store used in FastSync
- if skipHistory is used, headers store is used for getBlockHeaders message answer, so fixing Lack of commonly validating blocks if sync.fast.skipHistory = true #1058
- total difficulty recalculated if skipHistory used, fixing Incorrect totalDifficulty after FastSync in skipHistory mode #1041
- migration added to modify DB for those who already did sync with skipHistory
- if skipHistory was not used, headers store is removed, because it's not used at all (1.6GB currently on MainNet)
… skipped during fast sync
…kipHistory is used
mkalinin
suggested changes
Apr 26, 2018
| public HeaderStore headerStore() { | ||
| DbSource<byte[]> dataSource = keyValueDataSource("headers"); | ||
|
|
||
| WriteCache.BytesKey<byte[]> cache = new WriteCache.BytesKey<>( |
There was a problem hiding this comment.
I'd suggest to simplify source pipeline in following way:
xorSource("header")
\
cachedSource -> kvSource("headers")
/
xorSource("index")
| /** | ||
| * Dummy serializer (doesn't change anything) | ||
| */ | ||
| public final static Serializer<byte[], byte[]> DummySerializer = new Serializer<byte[], byte[]>() { |
There was a problem hiding this comment.
Maybe IdentitySerializer is better? dummy pretty equals to stub
| @@ -874,6 +898,10 @@ private BlockHeader getPivotHeaderByHash(byte[] pivotBlockHash) throws Exception | |||
| return null; | |||
| } | |||
|
|
|||
| public boolean isEndedOrNotStarted() { | |||
There was a problem hiding this comment.
Isn't isInProgress() a better name?
| for (int i = 1; i < headerSource.size(); ++i) { | ||
| BlockHeader curHeader = headerSource.get(i); | ||
| headerStore.saveHeader(curHeader); | ||
| headerSource.set(i, null); |
There was a problem hiding this comment.
Won't two sources with same physical source collide here?
There was a problem hiding this comment.
The same chance that two xor sources in one db will collide. Very small.
mkalinin
reviewed
Apr 26, 2018
| * @deprecated | ||
| * Remove alone with migration from {@link org.ethereum.manager.WorldManager} | ||
| */ | ||
| @Deprecated | ||
| @Bean | ||
| @Lazy | ||
| public DataSourceArray<BlockHeader> headerSource() { |
There was a problem hiding this comment.
Let's get rid of this. It can be created manually to run a migration
mkalinin
reviewed
Apr 26, 2018
| * adds additional headerSource to Blockchain | ||
| * as Blockstore is incomplete in this mode | ||
| */ | ||
| private void fastSyncDbJobs() { |
There was a problem hiding this comment.
Seems WorldManager is not the best place for that code. Let's move it to separate class.
mkalinin
reviewed
Apr 27, 2018
| /** | ||
| * No change serializer (doesn't change anything) | ||
| */ | ||
| public final static Serializer<byte[], byte[]> NoChangeSerializer = new Serializer<byte[], byte[]>() { |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.