Use custom-diff shares for better proxy and worker stats

[CR4DL]

Objective

In daemon mode, a share is only accepted when a block is successfully mined. Therefore, the statistics show zero hashrate most of the time (for the proxy and all of the workers), making them almost useless. This PR provides a solution to this problem. It can also be used to improve the statistics for high difficulty upstream connections.

Proposal

When the custom-diff feature is enabled, more shares are submitted to the proxy, carrying valuable information about the actual hashrate of the workers. Currently, this information is discarded.

Using the existing AcceptEvent, an update of the statistics can be triggered for incoming shares at the custom-diff level. Flagging the event via its new member isCustomDiff() ensures correct handling by the associated subscribers.

Result

With custom-diff enabled, the proxy is able to calculate the actual hashrate although no shares are accepted by the daemon.

Setup:

• Proxy connected to Monero daemon for high difficulty
• one active worker with 2 kH/s hashrate (reported by XMRig)
• custom-diff set to 5000 for meaningful stats within 10 minutes

Side effect (may be desired): The worker stats now show the same accepted shares count as the miner software of the worker and not only the count of shares sucessfully passed upstream.

[20:17:10.492] #000 use daemon localhost:18081  127.0.0.1 
[20:17:10.492] #000 new job from localhost:18081 diff 113801973557 algo rx/0 height 1998253
[20:18:10.613] 2.00 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +1/-0
[20:18:38.743] #000 new job from localhost:18081 diff 113647291429 algo rx/0 height 1998254
[20:19:10.730] 1.83 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[20:20:10.867] 2.08 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[20:21:10.987] 2.25 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[20:21:40.172] #000 new job from localhost:18081 diff 113805628935 algo rx/0 height 1998255
[20:22:05.089] #000 new job from localhost:18081 diff 113747545689 algo rx/0 height 1998256
[20:22:11.070] 2.08 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[20:23:11.205] 1.83 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[20:24:11.296] 1.92 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[20:25:02.408] #000 new job from localhost:18081 diff 113656280473 algo rx/0 height 1998257
[20:25:11.403] 2.00 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[20:26:10.591] #000 new job from localhost:18081 diff 113791206739 algo rx/0 height 1998258
[20:26:11.530] 2.33 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[20:26:29.630] #000 new job from localhost:18081 diff 114082948766 algo rx/0 height 1998259
[20:27:11.664] 1.67 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[20:27:25.512] * speed (1m) 1.58, (10m) 2.02, (1h) 0.34, (12h) 0.03, (24h) 0.01 kH/s
WORKER NAME     | LAST IP         | COUNT | ACCEPTED | REJ |  10 MINUTES |    24 HOURS |
test            | 127.0.0.1       |     1 |      247 |   0 |   2.02 kH/s |   0.01 kH/s |

[xmrig]

Looks good, however I will review/merge it tomorrow.
Thank you.

[xmrig]

It not works as expected if proxy connected to pool, hashrate calculated twice (custom diff + accepted by pool), I think best solution is add new option to determine how hashrate should calculated, biggest issue: I have no idea how name this option.

custom-diff option is not good for it, becuase diff can be specified per miner by +diff.
Thank you.

[CR4DL]

You are right, I did not test this case. I'll try to make the necessary adjustments.

[CR4DL]

Forgive my foolish oversight. Now it should work as intended.
I will add the new option (and find a name) if you like the feature so far.

Changes

With the new method, every share has to be considered with the diff of the worker. In the first commit, shares accepted by the pool were considered with the pool diff instead. That's a bit like calculating the hashrate based on the actual diff of the shares ... it's always too high.

Method AcceptEvent::statsDiff() will now return the correct diff for the stats calculation.
Individual custom diffs via +diff work as well, as everything is based on Miner::customDiff().

Tests

This time, I tested with a pool (fixed diff at 400015).
First, with custom-diff set to 5000: We see the actual Hashrate of ~2 kH/s, no spikes for shares accepted by the pool.

Show log

[22:52:29.433] #000 use pool host:1234
[22:53:29.409] 2.25 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +1/-0
[22:54:29.524] 1.92 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[22:55:29.625] 1.83 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[22:56:29.725] 1.58 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[22:57:29.809] 2.67 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[22:58:29.911] 1.50 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[22:59:17.371] #000 accepted (1/0+0) diff 400015 ip 127.0.0.1 (45 ms)
[22:59:30.011] 3.00 kH/s, shares: 1/0 +1, upstreams: 1, miners: 1 (max 1) +0/-0
[22:59:40.129] #000 accepted (2/0+0) diff 400015 ip 127.0.0.1 (35 ms)
[23:00:30.086] 2.08 kH/s, shares: 2/0 +1, upstreams: 1, miners: 1 (max 1) +0/-0
[23:01:30.188] 2.50 kH/s, shares: 2/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:02:24.808] #000 accepted (3/0+0) diff 400015 ip 127.0.0.1 (38 ms)
[23:02:30.324] 2.33 kH/s, shares: 3/0 +1, upstreams: 1, miners: 1 (max 1) +0/-0

[23:02:37.670] * speed (1m) 2.50, (10m) 2.18, (1h) 0.36, (12h) 0.03, (24h) 0.02 kH/s
WORKER NAME  | LAST IP         | COUNT | ACCEPTED | REJ |  10 MINUTES |    24 HOURS |
test         | 127.0.0.1       |     1 |      262 |   0 |   2.18 kH/s |   0.02 kH/s |

Second, with custom-diff disabled: We only see multiples of 6.67 kH/s.

Show log

[23:03:14.851] #000 use pool host:1234
[23:04:14.059] 0.00 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +1/-0
[23:05:14.163] 0.00 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:05:14.614] #000 accepted (1/0+0) diff 400015 ip 127.0.0.1 (31 ms)
[23:06:14.283] 6.67 kH/s, shares: 1/0 +1, upstreams: 1, miners: 1 (max 1) +0/-0
[23:07:14.366] 0.00 kH/s, shares: 1/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:07:25.867] #000 accepted (2/0+0) diff 400015 ip 127.0.0.1 (35 ms)
[23:07:59.720] #000 accepted (3/0+0) diff 400015 ip 127.0.0.1 (36 ms)
[23:08:14.497] 13.33 kH/s, shares: 3/0 +2, upstreams: 1, miners: 1 (max 1) +0/-0
[23:09:14.618] 0.00 kH/s, shares: 3/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:10:14.696] 0.00 kH/s, shares: 3/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:11:14.779] 0.00 kH/s, shares: 3/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:11:34.677] #000 accepted (4/0+0) diff 400015 ip 127.0.0.1 (36 ms)
[23:12:14.858] 6.67 kH/s, shares: 4/0 +1, upstreams: 1, miners: 1 (max 1) +0/-0
[23:13:14.992] 0.00 kH/s, shares: 4/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0

[23:13:16.184] * speed (1m) 0.00, (10m) 2.67, (1h) 0.44, (12h) 0.04, (24h) 0.02 kH/s
WORKER NAME  | LAST IP         | COUNT | ACCEPTED | REJ |  10 MINUTES |    24 HOURS |
test         | 127.0.0.1       |     1 |        4 |   0 |   2.67 kH/s |   0.02 kH/s |

Third, with custom-diff set to 5000, overridden by miner with +2000 in user: Also ~2 kH/s. Same result with custom-diff disabled in config.

Show log

[23:43:37.284] #000 use pool host:1234
[23:44:34.211] 1.80 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +1/-0
[23:45:34.287] 2.13 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:46:34.391] 2.10 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:47:34.500] 2.13 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:48:34.562] 1.80 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:49:34.662] 1.83 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:50:34.757] 1.73 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:51:34.876] 2.27 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:52:34.996] 1.77 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:53:35.091] 1.97 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:54:35.178] 2.13 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[23:54:55.511] #000 accepted (1/0+0) diff 400015 ip 127.0.0.1 (39 ms)
[23:55:35.282] 1.87 kH/s, shares: 1/0 +1, upstreams: 1, miners: 1 (max 1) +0/-0

[23:56:13.324] * speed (1m) 1.50, (10m) 1.93, (1h) 0.41, (12h) 0.03, (24h) 0.02 kH/s
WORKER NAME  | LAST IP         | COUNT | ACCEPTED | REJ |  10 MINUTES |    24 HOURS |
test         | 127.0.0.1       |     1 |      732 |   0 |   1.93 kH/s |   0.02 kH/s |

[CR4DL]

Option added. As statistics (hashrates and worker share count) are calculated based on custom diff shares (via global or per-worker setting), I named it custom-diff-stats. That should be kind of self-explaining.

As the configuration needs to be accessed within Stats now, this class also needs a pointer to the controller. I guessed that within Proxy, Stats should then be treated similarly to the other classes (e.g. Workers) in this regard, so I made Proxy::m_stats an allocatable pointer as well. Of course this triggered some syntax changes down the road ...

I also did a test with a pool (fixed diff at 400015), first with custom-diff-stats disabled, then enabled it in the config file while the proxy was running. The transition is quite smooth with some averaging happening for the hashrate directly after the switch.

Show log

[12:52:09.175] #000 use pool host:1234
[12:53:06.192] 0.00 kH/s, shares: 0/0 +0, upstreams: 1, miners: 1 (max 1) +1/-0
[12:53:09.486] #000 accepted (1/0+0) diff 400015 ip 127.0.0.1 (35 ms)
[12:54:06.308] 6.67 kH/s, shares: 1/0 +1, upstreams: 1, miners: 1 (max 1) +0/-0
[12:55:06.410] 0.00 kH/s, shares: 1/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[12:56:06.511] 0.00 kH/s, shares: 1/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[12:57:06.626] 0.00 kH/s, shares: 1/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[12:58:06.730] 0.00 kH/s, shares: 1/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[12:59:06.883] 0.00 kH/s, shares: 1/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[13:00:07.016] 0.00 kH/s, shares: 1/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[13:01:07.120] 0.00 kH/s, shares: 1/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[13:01:54.422] #000 accepted (2/0+0) diff 400015 ip 127.0.0.1 (39 ms)
[13:02:07.221] 6.67 kH/s, shares: 2/0 +1, upstreams: 1, miners: 1 (max 1) +0/-0

[13:02:07.888] * speed (1m) 6.67, (10m) 1.33, (1h) 0.22, (12h) 0.02, (24h) 0.01 kH/s
WORKER NAME  | LAST IP         | COUNT | ACCEPTED | REJ |  10 MINUTES |    24 HOURS |
test         | 127.0.0.1       |     1 |        2 |   0 |   1.33 kH/s |   0.01 kH/s |

[13:02:12.108] "config.json" was changed, reloading configuration

[13:03:07.302] 1.37 kH/s, shares: 2/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[13:04:07.370] 2.20 kH/s, shares: 2/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[13:05:07.488] 1.80 kH/s, shares: 2/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[13:06:06.673] #000 accepted (3/0+0) diff 400015 ip 127.0.0.1 (35 ms)
[13:06:07.572] 2.40 kH/s, shares: 3/0 +1, upstreams: 1, miners: 1 (max 1) +0/-0
[13:07:07.681] 2.43 kH/s, shares: 3/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[13:08:07.867] 2.10 kH/s, shares: 3/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[13:09:07.980] 1.67 kH/s, shares: 3/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[13:10:08.091] 1.53 kH/s, shares: 3/0 +0, upstreams: 1, miners: 1 (max 1) +0/-0
[13:10:13.940] #000 accepted (4/0+0) diff 400015 ip 127.0.0.1 (38 ms)
[13:11:08.183] 2.10 kH/s, shares: 4/0 +1, upstreams: 1, miners: 1 (max 1) +0/-0
[13:12:04.055] #000 accepted (5/0+0) diff 400015 ip 127.0.0.1 (38 ms)
[13:12:04.511] #000 accepted (6/0+0) diff 400015 ip 127.0.0.1 (34 ms)
[13:12:08.274] 1.80 kH/s, shares: 6/0 +2, upstreams: 1, miners: 1 (max 1) +0/-0

[13:12:09.801] * speed (1m) 1.80, (10m) 1.94, (1h) 0.55, (12h) 0.05, (24h) 0.02 kH/s
WORKER NAME  | LAST IP         | COUNT | ACCEPTED | REJ |  10 MINUTES |    24 HOURS |
test         | 127.0.0.1       |     1 |      586 |   0 |   1.94 kH/s |   0.02 kH/s |

[CR4DL]

Thank you for the quick review!