Simulate logic adds extra case in which UCB is not used

[gautams3]

I am not sure of the logic behind adding ltn<= 0.0 in the simulate() function. It implies that for two first two tree queries (ltn = -Inf and 0), it uses the node value as criterion. I understand that we use node value when ltn = -Inf, but don't understand why it is the case for ltn = 0

Ref this line in simulate()

    for node in t.children[h]
        n = t.n[node]
        if n == 0 && ltn <= 0.0
            criterion_value = t.v[node]
       elseif 
        .
        .
        .
        end

[zsunberg]

if n=0 and ltn = 0, the sqrt(ltn/n) calculation will produce a NaN. Can you think of any other way to handle this?

[gautams3]

It seems like instead of producing a NaN, we are assigning 0/0 a value of 0. Assigning 0 to undefined expression 0/0 does help in disabling UCB for all children nodes (even those with n != 0).

One failing edge case is pure exploration (completely random policy) where exploration coefficient coeff_c = Inf. Consider this scenario:

total_n = 1 and ltn = 0. So there is some child node ch with nch = 1. Its criterion value is

criterion_value_ch = value_ch + coeff_c * sqrt(0/1)

coeff_c * sqrt(0/1) = Inf * 0 = NaN. All other nodes might have finite critertion values, and comparing NaNs to these might result in an error.

The easy solution to this is to disable UCB for all children nodes if ltn=0. This way we avoid the Inf*0 expression. The code can read

if ltn <= 0.0 # ltn<0 implies all n = 0, ltn = 0 imples 1 n = 0. Both cases covered here.
            criterion_value = t.v[node]
       elseif 
        .
        .
        .
        end

If this edge case is important enough to address, I can send over a PR.