Implement Fast Launch #142

cdonnachie · 2023-08-27T12:38:59Z

Code as supplied by ZULUPooL

Keep track of last block per algo
In GetNextWorkRequiredV4 drop out of retarget loop sooner if bnNew > than params.powLimit

#130

Code as supplied by ZULUPooL

gto90

cACK

saltedlolly · 2023-08-27T21:40:55Z

Amazing work @cdonnachie! Thanks for all your contributions! A fix for this problem has been a long time coming.

cdonnachie · 2023-08-29T20:09:28Z

Amazing work @cdonnachie! Thanks for all your contributions! A fix for this problem has been a long time coming.

Credit here goes to @ZULUPooL, he did the coding and I just created the PR

barrystyle · 2023-09-03T09:27:10Z

with this PR applied:
2023-09-03T09:18:31Z block index 142040ms
2023-09-03T09:18:31Z block tree size = 17806039
2023-09-03T09:18:31Z nBestHeight = 17806033

without (develop at 5e756e2):
2023-09-03T09:25:01Z block index 137991ms
2023-09-03T09:25:01Z block tree size = 17806073
2023-09-03T09:25:01Z nBestHeight = 17806067

am i missing something here?

cdonnachie · 2023-09-03T14:33:32Z

with this PR applied: 2023-09-03T09:18:31Z block index 142040ms 2023-09-03T09:18:31Z block tree size = 17806039 2023-09-03T09:18:31Z nBestHeight = 17806033

without (develop at 5e756e2): 2023-09-03T09:25:01Z block index 137991ms 2023-09-03T09:25:01Z block tree size = 17806073 2023-09-03T09:25:01Z nBestHeight = 17806067

am i missing something here?

This corrects testnet load time more than mainnet.

With this PR applied on Testnet:
2023-09-03T12:41:20Z block index 130360ms
2023-09-03T12:41:20Z block tree size = 2438555
2023-09-03T12:41:20Z nBestHeight = 2438554

without (develop at af42429):
2023-09-03T12:49:54Z init message: Loading block index…
2023-09-05T19:00:03Z block index 195008733ms
2023-09-05T19:00:04Z block tree size = 2438596
2023-09-05T19:00:04Z nBestHeight = 2438595

Current code base took 54 hours to load testnet, with this PR ~2 minutes.

saltedlolly · 2023-09-10T14:27:15Z

Can we get a few more eyeballs on this? It would be great to get it reviewed.

gto90

I agree with the concept of this PR, but need one of @digicontributer , @JaredTate or @SmartArray to also have a look.

cACK

ycagel · 2023-10-03T02:53:51Z

Thanks for the review @gto90! Any thoughts @JaredTate, @digicontributer or @SmartArray?

ghost · 2023-10-09T11:42:26Z

Testnet loads very slowly. hope someone can take a look at this pr.

ycagel · 2023-10-10T16:34:40Z

Testnet loads very slowly. hope someone can take a look at this pr.

One solution would be to check out the source branch for this PR (142) and test it. If you can reply in this PR with your findings that would be great.

saltedlolly · 2023-10-10T18:36:23Z

I have been running this for over a month on a Raspberry Pi 4 8GB. Before this fix, running a testnet node on the Pi was impossible. It never finished syncing because it took more than 15 seconds to verify each historic block meaning it was forever playing catch up. This has completely solved the issue. Starting up a testnet node now takes a matter of minutes, and it syncs to 100% within a few hours. The difference is night and day. And it helps makes the testnet network considerably more usable again for developers.

@SmartArray @digicontributer @JaredTate Could you please help by reviewing this? It would be great to get this merged so we can release RC3.

ycagel · 2023-10-10T18:49:27Z

We would still need testing based on Windows and Mac. Can anyone help with that?

jchappelow · 2023-10-11T02:58:01Z

src/pow.cpp

@@ -239,6 +239,10 @@ unsigned int GetNextWorkRequiredV4(const CBlockIndex* pindexLast, const Consensu
        {
            bnNew *= (100 + params.nLocalTargetAdjustment);
            bnNew /= 100;
+            if (i % 16 == 0 && bnNew > UintToArith256(params.powLimit)) {


Should add some comments regarding the this logic and mod 16. This is consensus code.

For testnet this loop could be in the thousands of iterations, since bnNew continues to be incremented by params.nLocalTargetAdjustment it's value may become greater than params.powLimit which the code after the loop already checks for and will set it to params.powLimit on line 249. So may as well break out the loop at the point bnNew is greater than params.powLimit. It will not break consensus as it's not altering what will be determined as the bnNew value.

if (bnNew > UintToArith256(params.powLimit)) { bnNew = UintToArith256(params.powLimit); }

For the the i % 16 it's really limiting this check to every 16 iterations to reduce CPU usage of checking each time if bnNew > powLimit which will incur CPU resources

Modulo Check (i % 16 == 0): This part of the condition checks if the loop's iterator i is a multiple of 16. It essentially acts as a periodic trigger within the loop, activating the subsequent code block every 16 iterations or blocks.

Difficulty Check (bnNew > UintToArith256(params.powLimit)): This condition checks if the current calculated difficulty (bnNew) is greater than maximum difficulty (UintToArith256(params.powLimit)).

When both conditions are met (every 16 iterations and if the calculated difficulty exceeds the maximum limit), the specific code block following this line will execute.

So is this happening for every 16 blocks total? Or just every 16 blocks per algo? It's per algo correct? Are we potentially opening up the vulnerability of an unchecked 15 block reorg? Just my threat brain asking what if questions.

ghost · 2023-10-11T15:18:27Z

We would still need testing based on Windows and Mac. Can anyone help with that?

Build for windows. With pr 140 AND 142. Super fast.

ghost · 2023-10-11T16:04:09Z

26 seconds to load. normally more than 24 hours. On testnet.

IMG_0074.mov

ycagel · 2023-10-11T16:18:26Z

@Jongjan88 Thanks for doing this. Can you attach this video on PR 140 too so we know it applies there as well?

ghost · 2023-10-11T16:40:32Z

@Jongjan88 Thanks for doing this. Can you attach this video on PR 140 too so we know it applies there as well?

OFC

ycagel · 2023-10-11T18:10:43Z

Looks like we need testing on Mac.

j50ng

cACK (tested but did not review)

When I tested RC2 when it initially came out, it would take 30+ hours to compile and synch, and worse another 30+ hours when shutting down and restarting. When testing the current develop branch without the changes in PR #140 and PR #142, it would take at least 14 hours to completely synch and same if shutdown and restarted. When adding the files from PR #140 and PR #142 it took 25 minutes for the initial synch to fully complete. Although I am not getting the 2-5 minute synch times that others are seeing, the dramatic improvement to full launch is almost unbelievable. Shutting down and restarting would only take 1 minute to resynch as well. This seems like a significant improvement, especially for testnet. Hoping this gets thoroughly reviewed as it would be a major milestone for testnet if implemented.

Wrong PR. Will review this in a bit.

ycagel · 2023-11-01T04:48:01Z

Was anyone able to test this on the Mac?

otherdeniz

i am not shure about my comments - but i guess i am right - i might be wrong - i just saw the code the first time and do not understand a lot of it yet - its just what i saw in half an hour looking at the changes

otherdeniz · 2023-11-07T20:59:00Z

src/pow.cpp

@@ -343,6 +347,25 @@ const CBlockIndex* GetLastBlockIndexForAlgo(const CBlockIndex* pindex, const Con
    return nullptr;
 }

+const CBlockIndex* GetLastBlockIndexForAlgoFast(const CBlockIndex* pindex, const Consensus::Params& params, int algo)
+{
+    for (; pindex; pindex = pindex->lastAlgoBlocks[algo])


if the algo is not the same as in pindex it returns a null pointer - row 354 raises then a null ptr exception i guess

otherdeniz · 2023-11-07T21:05:45Z

src/validation.cpp

+
+    if (pindexNew->pprev) {
+        for (unsigned i = 0; i < NUM_ALGOS_IMPL; i++)
+            pindexNew->lastAlgoBlocks[i] = pindexNew->pprev->lastAlgoBlocks[i];


this loop is useless - if "i" is not the GetAlgo() there are only null ptr assignments

and by the way - its not best practice to populate a array field of another instance on a foreign method

otherdeniz · 2023-11-07T21:06:14Z

src/validation.cpp

+
+        if (pindex->pprev) {
+            for (unsigned i = 0; i < NUM_ALGOS_IMPL; i++)
+              pindex->lastAlgoBlocks[i] = pindex->pprev->lastAlgoBlocks[i];


also useless - only null ptr in the array for the other algos

@otherdeniz We don't claim to have perfect code for tweaking other people's software. We came up with this quick solution ourselves, we used it for our own needs, and it works for us. We shared it with developers. Developers can, if they wish, understand the idea and rewrite this code, taking into account their understanding of program structure and acceptable methods.

JaredTate · 2023-11-22T18:51:37Z

First off thank you for submitting this PR & taking the time to make this improvement! And also thank you to everyone taking the time to review it. I am all for speeding things up, especially the core wallet loading time. My fear is that we don't want to reduce security somehow & increase attack vectors, so thanks again for all the eyes on this.

So if I understand this correctly, the new "GetLastBlockIndexForAlgoFast" function replaces "GetLastBlockIndexForAlgo."

If I see this correctly the main difference between GetLastBlockIndexForAlgoFast and the original function is the use of the lastAlgoBlocks array, which stores the last block index for each algorithm, thus allowing the new function to jump directly to the previous block of the same algorithm without checking every block from every other algo. In contrast, the old function would check the block of all algos, not just the individual algo, wasting a lot of CPU power & slowing down startup.

My main question with all this is this line of code:
if (i % 16 == 0 && bnNew > UintToArith256(params.powLimit)) {

If I understand this line correctly:

Modulo Check (i % 16 == 0): This part of the condition checks if the loop's iterator i is a multiple of 16. It essentially acts as a periodic trigger within the loop, activating the subsequent code block every 16 iterations or 16 blocks? Does this mean we only get a startup diff check every 16 blocks as it loops through for each algo? I need to back trace this logic a bit more if I understand it fully.

Difficulty Check (bnNew > UintToArith256(params.powLimit)): This condition checks if the current calculated difficulty (bnNew) is greater than the blockchain's predefined maximum difficulty (UintToArith256(params.powLimit)).

My fear here is are we opening ourselves up to a short term 51% attack somehow by allowing a shallow 15 block reorg that is never checked on a client start up. Granted this is a launch speed improvement, but I would love to verify this is not going to be an issue.

Has anyone test mined all 5 algos with these changes?

Also @otherdeniz brought up great points for the code below about "only null ptr in the array for the other algos".

if (pindex->pprev) { for (unsigned i = 0; i < NUM_ALGOS_IMPL; i++) pindex->lastAlgoBlocks[i] = pindex->pprev->lastAlgoBlocks[i];

ycagel · 2023-11-22T19:10:21Z

Has anyone test mined this? @cdonnachie @Jongjan88 @ZULUPooL @gto90

gto90 · 2023-11-22T20:25:00Z

First off thank you for submitting this PR & taking the time to make this improvement! And also thank you to everyone taking the time to review it. I am all for speeding things up, especially the core wallet loading time. My fear is that we don't want to reduce security somehow & increase attack vectors, so thanks again for all the eyes on this.

So if I understand this correctly, the new "GetLastBlockIndexForAlgoFast" function replaces "GetLastBlockIndexForAlgo."

If I see this correctly the main difference between GetLastBlockIndexForAlgoFast and the original function is the use of the lastAlgoBlocks array, which stores the last block index for each algorithm, thus allowing the new function to jump directly to the previous block of the same algorithm without checking every block from every other algo. In contrast, the old function would check the block of all algos, not just the individual algo, wasting a lot of CPU power & slowing down startup.

My main question with all this is this line of code: if (i % 16 == 0 && bnNew > UintToArith256(params.powLimit)) {

If I understand this line correctly:

Modulo Check (i % 16 == 0): This part of the condition checks if the loop's iterator i is a multiple of 16. It essentially acts as a periodic trigger within the loop, activating the subsequent code block every 16 iterations or 16 blocks? Does this mean we only get a startup diff check every 16 blocks as it loops through for each algo? I need to back trace this logic a bit more if I understand it fully.

Difficulty Check (bnNew > UintToArith256(params.powLimit)): This condition checks if the current calculated difficulty (bnNew) is greater than the blockchain's predefined maximum difficulty (UintToArith256(params.powLimit)).

My fear here is are we opening ourselves up to a short term 51% attack somehow by allowing a shallow 15 block reorg that is never checked on a client start up. Granted this is a launch speed improvement, but I would love to verify this is not going to be an issue.

Has anyone test mined all 5 algos with these changes?

Also @otherdeniz brought up great points for the code below about "only null ptr in the array for the other algos".

if (pindex->pprev) { for (unsigned i = 0; i < NUM_ALGOS_IMPL; i++) pindex->lastAlgoBlocks[i] = pindex->pprev->lastAlgoBlocks[i];

Thank you for this @JaredTate , I am also going to dive back in and take a deeper look over the weekend.

@JaredTate

Upon working with @JaredTate I believe this needs a deeper review.

saltedlolly · 2023-11-24T21:26:50Z

Correct me if I am wrong, but doesn't this fast start code only run when using testnet? It doesn't really benefit mainnet as far as I understand, and was never meant to. What it does do is make it possible to actually start up a testnet node in a reasonable time.

JaredTate · 2023-11-24T21:39:40Z

Correct me if I am wrong, but doesn't this fast start code only run when using testnet? It doesn't really benefit mainnet as far as I understand, and was never meant to. What it does do is make it possible to actually start up a testnet node in a reasonable time.

Great question, the changes as made, especially the change to GetLastBlockIndexForAlgoFast in pow.cpp inside GetNextWorkRequiredV4 will most definitely apply to main net as I am reading the code. If the intent is for this only to be applicable to testnet, then this needs to be written differently.

saltedlolly · 2023-12-16T17:57:04Z

Does anybody here have the ability to mine on testnet? If so would you mine doing it, even if only temporaily? cc @cdonnachie @jchappelow

I have tried to do it but am receiving errors, though I am probably doing something wrong. @Jongjan88 and I are seeing some strange behaviour on the testnet. We think it might because noone has mined for some time.

ZULUPooL · 2023-12-19T12:30:22Z

Does anybody here have the ability to mine on testnet? If so would you mine doing it, even if only temporaily? cc @cdonnachie @jchappelow

I have tried to do it but am receiving errors, though I am probably doing something wrong. @Jongjan88 and I are seeing some strange behaviour on the testnet. We think it might because noone has mined for some time.

I've started testnet mining, everything works so far.
https://dgb-t.zulupool.com/

JaredTate · 2023-12-20T22:39:30Z

After compiling, and giving a few test runs this code cuts the start up load time basically in half for my DGB main net client. I am on an 8-core intel i9 with 32gb ram. The loading estimate percentage is nice to see as well. Seems accurate. The load time was cut for me from 7 min 40 seconds to 3 min and 30 seconds on average.

Doing more of a deep dive and back tracing GetLastBlockIndexForAlgoFast fuly I am no longer worried about a problem as I was in my previous post. GetLastBlockIndexForAlgoFast <- GetNextWorkRequiredv4 <- GetNextWorkRequired <- GetBlockProof <-LoadBlockIndex

The changes in GetNextWorkRequiredv4 had me concerned, more specifically the per algo- retarget section " if (i % 16 == 0 && bnNew":

unsigned int GetNextWorkRequiredV4(const CBlockIndex* pindexLast, const Consensus::Params& params, int algo)
{
    // find first block in averaging interval
    // Go back by what we want to be nAveragingInterval blocks per algo
    const CBlockIndex* pindexFirst = pindexLast;
    for (int i = 0; pindexFirst && i < NUM_ALGOS*params.nAveragingInterval; i++)
    {
        pindexFirst = pindexFirst->pprev;
    }

    const CBlockIndex* pindexPrevAlgo = GetLastBlockIndexForAlgoFast(pindexLast, params, algo);
    if (pindexPrevAlgo == nullptr || pindexFirst == nullptr)
    {
        return InitialDifficulty(params, algo);
    }

    // Limit adjustment step
    // Use medians to prevent time-warp attacks
    int64_t nActualTimespan = pindexLast-> GetMedianTimePast() - pindexFirst->GetMedianTimePast();
    nActualTimespan = params.nAveragingTargetTimespanV4 + (nActualTimespan - params.nAveragingTargetTimespanV4)/4;

    if (nActualTimespan < params.nMinActualTimespanV4)
        nActualTimespan = params.nMinActualTimespanV4;
    if (nActualTimespan > params.nMaxActualTimespanV4)
        nActualTimespan = params.nMaxActualTimespanV4;

    //Global retarget
    arith_uint256 bnNew;
    bnNew.SetCompact(pindexPrevAlgo->nBits);

    bnNew *= nActualTimespan;
    bnNew /= params.nAveragingTargetTimespanV4;

    //Per-algo retarget
    int nAdjustments = pindexPrevAlgo->nHeight + NUM_ALGOS - 1 - pindexLast->nHeight;
    if (nAdjustments > 0)
    {
        for (int i = 0; i < nAdjustments; i++)
        {
            bnNew *= 100;
            bnNew /= (100 + params.nLocalTargetAdjustment);
        }
    }
    else if (nAdjustments < 0)//make it easier
    {
        for (int i = 0; i < -nAdjustments; i++)
        {
            bnNew *= (100 + params.nLocalTargetAdjustment);
            bnNew /= 100;
            if (i % 16 == 0 && bnNew > UintToArith256(params.powLimit)) {
              bnNew = UintToArith256(params.powLimit);
              break;
            }            
        }
    }

    if (bnNew > UintToArith256(params.powLimit))
    {
        bnNew = UintToArith256(params.powLimit);
    }

    return bnNew.GetCompact();
}

Since GetNextWorkRequiredV4 is such a critical part of DGB its important to break down exactly what its doing, Chat GPT provides a great summary:

GetNextWorkRequiredV4 Function Analysis

This C++ function, GetNextWorkRequiredV4, is part of the DigiByte blockchain's protocol, specifically designed to calculate the difficulty for the next block to be mined. Let's break down the function step by step:

Function Signature

GetNextWorkRequiredV4: This is the function name.
const CBlockIndex* pindexLast: A pointer to the last block in the blockchain.
const Consensus::Params& params: A reference to the blockchain's consensus parameters.
int algo: An integer representing the specific mining algorithm for which the difficulty is being calculated.

Initial Steps

Finding the First Block in Averaging Interval:

The function first identifies the starting point of an averaging interval by moving backward from the last block (pindexLast) in the blockchain.
It traverses NUM_ALGOS * params.nAveragingInterval blocks backward, where NUM_ALGOS is the total number of mining algorithms and nAveragingInterval is a parameter defining the interval length.

Finding the Previous Block for the Algorithm:

GetLastBlockIndexForAlgoFast finds the last block that was mined using the same algorithm (algo). If either this block or the first block of the averaging interval is not found (nullptr), the function returns the initial difficulty for the algorithm.

Difficulty Adjustment

Limit Adjustment Step:

The actual time span between the first and last blocks in the averaging interval is calculated using median times to prevent time-warp attacks.
This timespan is then adjusted to be within a minimum and maximum range specified by params.

Global Retarget:

The new difficulty (bnNew) is initially set to the difficulty of the last block mined using the same algorithm.
It's adjusted according to the actual timespan and the target timespan from params.

Per-Algorithm Retarget

Calculating `nAdjustments`:

nAdjustments calculates the difference in the height (number of blocks) between the last block of the current algorithm and the last block overall, adjusted by NUM_ALGOS - 1.

Adjusting Difficulty Based on `nAdjustments`:

If nAdjustments is Positive:
- This implies underuse of the current algorithm. The difficulty is decreased to encourage mining with this algorithm.
- The difficulty (bnNew) is adjusted in a loop, reducing it by multiplying by 100 and dividing by 100 + params.nLocalTargetAdjustment.
If nAdjustments is Negative:
- This suggests overuse of the current algorithm. The difficulty is increased to balance its usage.
- The difficulty is increased in a loop, by multiplying with (100 + params.nLocalTargetAdjustment) and dividing by 100.
- A check ensures the difficulty does not exceed the maximum (params.powLimit). If it does, it's set to this limit.

Final Difficulty Check:

The function ensures that the final difficulty (bnNew) does not exceed the maximum allowed (params.powLimit). If it does, it's set to the maximum limit.

Return Value:

The new difficulty is returned in a compact form, which is a compressed version of the difficulty value.

Summary

In essence, GetNextWorkRequiredV4 calculates the mining difficulty for the next block, ensuring that blocks are mined at a consistent rate and that no single mining algorithm becomes too dominant or underused. The difficulty adjustment accounts for the overall blockchain's mining rate and the specific rate for each algorithm, promoting a balanced and secure mining environment.

Condition for Harder Mining/ Difficulty Increase In Per Algo Retarget. (Part I was concerned with)

else if (nAdjustments < 0)
{
    // This code block is executed when nAdjustments is negative.
    // Implication: A negative nAdjustments indicates that the current mining algorithm has been used more frequently than others.
    // Objective: The goal is to "make it harder" – that is, to increase the mining difficulty for this algorithm, thereby balancing the mining activity across all algorithms.
    
    // The Loop for Difficulty Adjustment
    for (int i = 0; i < -nAdjustments; i++)
    {
        // Inside the Loop: Difficulty Increase
        bnNew *= (100 + params.nLocalTargetAdjustment); // Process: `bnNew`, representing the current difficulty, is multiplied by `(100 + params.nLocalTargetAdjustment)`.
        bnNew /= 100; // Operation: After the multiplication, the new difficulty is divided by 100. Effect: This division moderates the increase made in the previous step, scaling down the difficulty adjustment.

        // Checking Difficulty Limit: Conditional Check within the Loop
        if (i % 16 == 0 && bnNew > UintToArith256(params.powLimit))
        {
            // Periodic Check: `i % 16 == 0` ensures this condition is checked every 16 iterations.
            // Difficulty Check: `bnNew > UintToArith256(params.powLimit)` verifies if the adjusted difficulty exceeds a set limit.
            // Limit Conversion: `UintToArith256(params.powLimit)` converts the power limit to a comparable format.
            // Adjusting to Maximum Limit
            bnNew = UintToArith256(params.powLimit); // Limit Exceeding: If the difficulty surpasses the maximum allowed (`params.powLimit`), it is reset to this maximum limit.
            break; // Loop Termination: The `break;` statement exits the loop if the maximum difficulty is reached, preventing further adjustments.
        }
    }
}

After stepping through everything I am much more confident this code wont cause issues. As @cdonnachie mentioned the "i % 16 " simply breaks out of the difficulty increase loop quicker and the same check is still happening right after this.

if (bnNew > UintToArith256(params.powLimit)) { bnNew = UintToArith256(params.powLimit); }

My big question is are there any further tweaks we can make to improve load times or are people satisfied with this improvement? Also, these changes most definitely will affect main-net right away and should be fully test mined on main net too.

saltedlolly · 2023-12-20T22:57:25Z

@JaredTate Thank you for taking another look at this. You should also test it on testnet as this is where the larger benefit lies, and is the reason the PR was made in the first place. I suggest you try syncing 7.17.3 from scratch and then this. Big diference.

JaredTate · 2023-12-20T23:24:59Z

The reason testnet sees such speedier load times can be explained largely to the changes in the function GetLastBlockIndexForAlgoFast derived from GetLastBlockIndexForAlgo. Both of which are still in the code and used in various places. Main net also gets a bit of a speed increase on load times, but not nearly as much.

The DigiByte blockchain now has two distinct C++ functions, GetLastBlockIndexForAlgo and GetLastBlockIndexForAlgoFast, for finding the last block index of a specific mining algorithm. Here's an analysis of their primary differences:

1. Traversal Method

GetLastBlockIndexForAlgo:
- It traverses the blockchain linearly, block by block (pindex = pindex->pprev), checking each block's previous block.
GetLastBlockIndexForAlgoFast:
- Utilizes a more efficient method, jumping directly to the last block of the specific algorithm (pindex->lastAlgoBlocks[algo]). This approach skips many irrelevant blocks, leading to faster execution & less CPU drain.

2. Handling Minimum Difficulty Blocks

Both functions manage special minimum difficulty blocks on testnet, but with slightly different approaches:

GetLastBlockIndexForAlgo:
- Continues the loop to skip min-difficulty testnet blocks upon encountering them. This loop is where the main testnet slowdown was coming from.
GetLastBlockIndexForAlgoFast:
- Backtracks one block (pindex = pindex->pprev) when encountering a minimum difficulty block, then continues the search.

3. Performance

GetLastBlockIndexForAlgoFast is more efficient and optimized for performance, especially in a blockchain with a large number of blocks. It avoids linear search, reducing time complexity.

Summary

While both functions aim to find the last block index for a given mining algorithm in the DigiByte blockchain, GetLastBlockIndexForAlgoFast stands out for its efficiency and speed, leveraging an array (lastAlgoBlocks) that points directly to the relevant blocks. In contrast, GetLastBlockIndexForAlgo provides a more traditional and straightforward linear search method.

Definetly a great improvement, but we should mine some real-world blocks with this on main net.

ycagel · 2023-12-21T00:00:48Z

Thanks for all the analysis @JaredTate. We definitely need folks to do mining on main net and test net.

ycagel · 2023-12-21T02:23:56Z

JaredTate

ACK. Hearing reports this has been mined on main-net. Would be great to get this out in an RC3 for more testing.

JaredTate · 2023-11-22T19:07:22Z

src/pow.cpp

@@ -239,6 +239,10 @@ unsigned int GetNextWorkRequiredV4(const CBlockIndex* pindexLast, const Consensu
        {
            bnNew *= (100 + params.nLocalTargetAdjustment);
            bnNew /= 100;
+            if (i % 16 == 0 && bnNew > UintToArith256(params.powLimit)) {


Modulo Check (i % 16 == 0): This part of the condition checks if the loop's iterator i is a multiple of 16. It essentially acts as a periodic trigger within the loop, activating the subsequent code block every 16 iterations or blocks.

Difficulty Check (bnNew > UintToArith256(params.powLimit)): This condition checks if the current calculated difficulty (bnNew) is greater than maximum difficulty (UintToArith256(params.powLimit)).

When both conditions are met (every 16 iterations and if the calculated difficulty exceeds the maximum limit), the specific code block following this line will execute.

So is this happening for every 16 blocks total? Or just every 16 blocks per algo? It's per algo correct? Are we potentially opening up the vulnerability of an unchecked 15 block reorg? Just my threat brain asking what if questions.

gto90

Thank you, @JaredTate, for doing a live pair review of this code. After stepping through it with you in real-time, I am comfortable that this code is sound and my original concerns have been addressed.

ACK

Implement Fast Launch

4d8db4d

Code as supplied by ZULUPooL

ycagel requested review from SmartArray, JaredTate, ycagel, digicontributer, gto90 and j50ng August 27, 2023 15:25

gto90 assigned cdonnachie Aug 27, 2023

gto90 added the bug Something isn't working label Aug 27, 2023

gto90 previously approved these changes Aug 27, 2023

View reviewed changes

Merge branch 'develop' into feature/fastlaunch

a0cfadf

cdonnachie dismissed gto90’s stale review via a0cfadf September 5, 2023 17:46

gto90 previously approved these changes Sep 10, 2023

View reviewed changes

jchappelow reviewed Oct 11, 2023

View reviewed changes

ghost mentioned this pull request Oct 11, 2023

Update chainTxData for 8.22.0 #140

Merged

j50ng approved these changes Oct 12, 2023

View reviewed changes

This comment was marked as off-topic.

Sign in to view

JaredTate self-requested a review October 26, 2023 19:52

otherdeniz reviewed Nov 7, 2023

View reviewed changes

gto90 self-requested a review November 22, 2023 20:24

JaredTate approved these changes Dec 21, 2023

View reviewed changes

gto90 approved these changes Dec 21, 2023

View reviewed changes

ycagel merged commit b75fea6 into DigiByte-Core:develop Dec 21, 2023

This was referenced Dec 22, 2023

Testnet 8.22 RC2 Performance Issues #130

Closed

digibyte-cli freezing during header sync #149

Closed

ghost mentioned this pull request Mar 14, 2024

fix: Adds support for Apple silicon compilation #194

Merged

Implement Fast Launch #142

Implement Fast Launch #142

Conversation

cdonnachie commented Aug 27, 2023

gto90 left a comment

Choose a reason for hiding this comment

saltedlolly commented Aug 27, 2023

cdonnachie commented Aug 29, 2023

barrystyle commented Sep 3, 2023

cdonnachie commented Sep 3, 2023 • edited Loading

saltedlolly commented Sep 10, 2023

gto90 left a comment

Choose a reason for hiding this comment

ycagel commented Oct 3, 2023

ghost commented Oct 9, 2023

ycagel commented Oct 10, 2023

saltedlolly commented Oct 10, 2023

ycagel commented Oct 10, 2023

Choose a reason for hiding this comment

cdonnachie Oct 11, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghost commented Oct 11, 2023

ghost commented Oct 11, 2023 • edited by ghost Loading

ycagel commented Oct 11, 2023

ghost commented Oct 11, 2023

ycagel commented Oct 11, 2023

j50ng left a comment

Choose a reason for hiding this comment

This comment was marked as off-topic.

ycagel commented Nov 1, 2023

otherdeniz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JaredTate commented Nov 22, 2023 • edited Loading

ycagel commented Nov 22, 2023

gto90 commented Nov 22, 2023

saltedlolly commented Nov 24, 2023

JaredTate commented Nov 24, 2023

saltedlolly commented Dec 16, 2023

ZULUPooL commented Dec 19, 2023

JaredTate commented Dec 20, 2023 • edited Loading

GetNextWorkRequiredV4 Function Analysis

Function Signature

Initial Steps

Finding the First Block in Averaging Interval:

Finding the Previous Block for the Algorithm:

Difficulty Adjustment

Limit Adjustment Step:

Global Retarget:

Per-Algorithm Retarget

Calculating nAdjustments:

Adjusting Difficulty Based on nAdjustments:

Final Difficulty Check:

Return Value:

Summary

Condition for Harder Mining/ Difficulty Increase In Per Algo Retarget. (Part I was concerned with)

saltedlolly commented Dec 20, 2023

JaredTate commented Dec 20, 2023 • edited Loading

1. Traversal Method

2. Handling Minimum Difficulty Blocks

3. Performance

Summary

ycagel commented Dec 21, 2023

ycagel commented Dec 21, 2023

JaredTate left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gto90 left a comment • edited Loading

Choose a reason for hiding this comment

cdonnachie commented Sep 3, 2023 •

edited

Loading

cdonnachie Oct 11, 2023 •

edited

Loading

ghost commented Oct 11, 2023 •

edited by ghost

Loading

JaredTate commented Nov 22, 2023 •

edited

Loading

JaredTate commented Dec 20, 2023 •

edited

Loading

Calculating `nAdjustments`:

Adjusting Difficulty Based on `nAdjustments`:

JaredTate commented Dec 20, 2023 •

edited

Loading

gto90 left a comment •

edited

Loading