Improve emai-1d's handling of huge/0/negative block times, and add "ema2-1d"

[jacob-eliosoff]

This implements an approach to handling huge/0/negative block times that I mailed @zawy12 about in early Dec: define block_time as max(0, timestamp - highest_previous_timestamp). So timestamps in the past ("negative block times") just result in a single 0 block time, and timestamps in the future result in one large block time which may be followed by some 0 block times.

I left ema-1d unchanged and implemented a more-resilient version, ema2-1d, as well as incorporating the resilience into emai-1d. Of the three (ema, ema2, emai) I think emai is probably the one worth focusing on.

[jacob-eliosoff]

(Apologies if I'm screwing up any of the PR process here... Just let me know...)

[jacob-eliosoff]

The weakness of emai is its approximation may not increase decrease (oops thanks @zawy12) difficulty fast enough after a very slow block, and the weakness of ema2 (apart from floating-point math) is it may not handle ~0s blocks cleanly... There should be an integer-math-only version that handles both ~0 and huge block times, I'll take a poke when I get a chance.

[jacob-eliosoff]

Ahh OK I think I see how to handle both long & short blocks cleanly, but gotta run, please hold off on this PR for now

[jacob-eliosoff]

OK, I think I was able to get ema2-1d to handle both very short and very long blocks sensibly. ema2 is now my recommended algo, if not for production use at least as a benchmark for other algos. Specifically, I believe:

1. On a <=0s block, ema2 increases difficulty by the maximum possible that can't lead to overshooting (and thus oscillation); and
2. Similarly for very slow blocks: ema2 cuts difficulty by the maximum possible that can't lead to oscillation.

I was not able to restrict ema2 to integer math - I think achieving the two properties above requires exponentiation (or great trickery). Oh well, integer math, whatev...

[zawy12]

You can't use the e^x =~ 1+x+x^2/2 to get integer math?

[jacob-eliosoff]

I think that's better when x is an integer? What I have is stuff like [large integer] * e^{[fraction < 1]}. I can see ways to approximate it but, gets pretty messy. How beneficial is integer math anyway? Perhaps the real goal should be to avoid numerically unstable math (division by numbers near 0, etc)?

[zawy12]

No, e^x = sum( x^i/i!) for i=0,1,2.... is exact for all x, maybe even complex x.
I know I write too much for anyone to read, but this is the basis of a lot of what I've said in comparing the two. It appears exact as far as we're concerned to use e^x => 1+x+x^2/2

However, I really like being able to use negative solvetimes.

A dev told me some compilers on some systems may cause the e^x to give something different, so those nodes would reject a block. Tom and Neil's comments seem to concur there's possible problem. I guess division is handled better. Another dev was concerned about overflow from WT (my WHM), which they mentioned here to. (64 bit seems plenty but if they they don't have enough leading zeros in the coin's max_target (which scales difficulty down by 2^x) I think there could be a problem).

I guess integer division is reliable enough on systems.

But primarily devs want to change existing code as little as possible. Making variable declaration changes might have hard to find errors appear out of no where after the fact compared to other work they're doing when they make clones.

[jacob-eliosoff]

Sure, my point was just that your formula doesn't help me calculate e^x using integer math, when x itself is a fraction.

[zawy12]

You can't just scaled everything up by something like (NT)^2 before starting the calculations then divide out at the end?

[jacob-eliosoff]

I could! It just seemed a little elaborate ("pretty messy")... Maybe I'll give it a shot

[zawy12]

Then we'd have the exact equation, and as I showed in the other thread the exact equation for EMA-TH is the same as EMA-JE so now we can have EMA-Z to replace them :)

[zawy12]

This is what I get:

 k= t*T*N
next_target = prev_target * ((T*N)^2-T^2*N+k*(k/2+(T*N)^2-T^2*N/2) / (T*N)^2

[jacob-eliosoff]

I added emai2-1d, a version of ema2 which matches its behavior very closely (even for very long block times) using only integer math. But, gak. Hairy.

[kyuupichan]

Can we avoid adding another member?

[jacob-eliosoff]

Well, I could probably do something hackier like max(state.timestamp for state in states[-100:-1]). Anyway I think @zawy12 may be convincing me that the max timestamp approach isn't needed... So I'm OK making that change if you prefer?

[kyuupichan]

I'd prefer that. If every algo started putting stuff in the state it would become problematic.

[jacob-eliosoff]

Done!