A better heuristic tolerance for @test_approx_eq?

[chenych11]

The @test_approx_eq macro does not work as expected with arrays. For example

using Base.Test

a = rand(Float32, 1000)
sigmoid(x) = 1.0 ./ (1.0 + exp(-x))
expa = exp(a)
siga = sigmoid(a)

@test_approx_eq(expa, siga)            # PASS
@test_approx_eq_eps(expa, siga, 1e-4)  # FAIL

Both tests (indicated by the last two lines) should fail, because exp(x) and sigmoid(x) are different.
Currently, this macro defined as follows.

macro test_approx_eq_eps(a, b, c)
    :(test_approx_eq($(esc(a)), $(esc(b)), $(esc(c)), $(string(a)), $(string(b))))
end

The function called by this macro is defined like this

test_approx_eq(va, vb, astr, bstr) = test_approx_eq(va, vb, 1E4*length(linearindices(va))*max(array_eps(va), array_eps(vb)), astr, bstr)

I think the primary reason for this issue is the length dependent tolerance heuristic: Eps = 1E4*length(linearindices(va))*max(array_eps(va), array_eps(vb))
Currently, the overload function test_approx_eq(va, vb, Eps, astr, bstr) computes the difference of two arrays va and vb using the -norm, which is defined as the maximum absolute value of va - vb. The -norm, which differs from the 1-norm or 2-norm, does not sum over the dimensions. So, this heuristic, which is dependent on the length of the arrays, is problematic when the length of the arrays are large.

By the way, I don't know if there are some justification to use the magic constant 1E4 as a factor of the heuristic tolerance.

The source code can be found at base/test.jl.

(commit: 72725a1)

[stevengj]

@test_approx_eq should be deprecated in favor of isapprox, which can be called via @test expa ≈ siga for the default (√ɛ) tolerance. (This test correctly fails on your example above.)

[yuyichao]

So dup of #4615?

[stevengj]

Yes, I think so.