[IBD] IBD in StratisD on testnet is slow

[Aprogiena]

Compared to StratisX, StratisD is maybe slow (with debug/trace logging disabled) during IBD. Compared to BitcoinD C# node on mainnet, StratisD on testnet is very slow.

The reason seams to be the very low total download speed, which was about 5 kb/s on my machine. Syncing just 5k of last blocks took ages.

In this issue I'd be looking for some specific comparisons between StratisX on testnet and StratisD. First we need to check whether StratisD is actually slower than StratisX and if so, we should try to find out why.

[Aprogiena]

Full StratisX sync from 0 to 99850 blocks took about 23 minutes.

[dangershony]

I also noticed when connecting to both stratisx and stratis-charp the sharp node was favored in terms of download score.

Very quickly:
StriasX got a score of 1
Statis-sharp got a score of 150

It might be that stratisx upload is not so fast and that slows down the downloading node.
Additional options to explore is the speed of dbreeze and how often we cash its entries.

[Aprogiena]

That's a good point, we should measure separately connections to SX and SD nodes

[Aprogiena]

I've measured StratisX on mainnet. First, up to block 390k it has checkpoints, so it just downloads, no validation is done, no CPU used. And blocks between 409400-419400 were processed in 6 minutes 10 seconds. This gives us 37 ms per block for StratisX.

[dangershony]

That's great news, I think when we add checkpoints it sill speed up the node very much.
Additionally we will implement AssumValid flag #22 so that nodes can sync faster without hard coding checkpoints.

[Aprogiena]

I'm not a big fan of checkpoints. Yes, we can use them for past blocks, but I'm mostly interested in speeding up the validation process. I need to measure how StratisD will perform on those exact blocks.

[dangershony]

Checkpoints are essential in the POS early days to avoid building a chain with the early high value outputs.
When distribution of coins is still not spread an attacker might buy a spent private keys with lots of coins and rewrite history.

Blockstream have a very gokdgood paper about this issue.

[Aprogiena]

Yes I agree with that, I'm just not big fan of checkpoints once the distribution is somewhat plausible.

I haven't checked, but does SD implement any such checkpoints?
FYI https://download.wpsoftware.net/bitcoin/pos.pdf

[Aprogiena]

We have improved the performance significantly with the recent builds and we have also designed a roadmap for further improvements, all this effort basically closes this issue.