Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Numerical Instability in Tests #365

Open
madhavajay opened this issue May 27, 2021 · 0 comments
Open

Numerical Instability in Tests #365

madhavajay opened this issue May 27, 2021 · 0 comments
Labels
Type: Bug 🐛 Some functionality not working in the codebase as intended

Comments

@madhavajay
Copy link
Contributor

Description

I have noticed one of the tests flaps and sometimes fails, I guess its due to some numerical instability?
https://github.com/OpenMined/PyDP/runs/2679233108?check_suite_focus=true

Screen Shot 2021-05-27 at 12 21 47 pm

______ TestLaplaceDistribution.test_check_statistics_for_geo_unit_values _______
[gw1] linux -- Python 3.6.13 /home/runner/.cache/pypoetry/virtualenvs/pydp-KC8mieZ4-py3.6/bin/python

self = <test_distributions.TestLaplaceDistribution object at 0x7fe191d630b8>

    def test_check_statistics_for_geo_unit_values(self):
    
        ld = LaplaceDistribution(epsilon=1.0, sensitivity=1.0)
        samples = [ld.sample(scale=1.0) for _ in range(k_num_geometric_samples)]
        mean = dp.util.mean(samples)
        var = dp.util.variance(samples)
    
        assert expect_near(0.0, mean, 0.01)
        assert expect_near(2.0, var, 0.1)
        assert expect_near(0.0, skew(samples, mean, math.sqrt(var)), 0.1)
>       assert expect_near(3.0, kurtosis(samples, mean, var), 0.1)
E       assert False
E        +  where False = expect_near(3.0, 3.1097853317836956, 0.1)
E        +    where 3.1097853317836956 = kurtosis([-0.18739762318364228, 0.19336953977835947, 2.2439366761800557, -0.120824026083028, 0.3653907802163303, 2.181137632465834, ...], 0.00018447984155104587, 2.0099539175529073)

How to Reproduce

See CI

Expected Behavior

Not failing.

Screenshots

See above

System Information

See Link

Additional Context

See above link

@madhavajay madhavajay added the Type: Bug 🐛 Some functionality not working in the codebase as intended label May 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Bug 🐛 Some functionality not working in the codebase as intended
Projects
None yet
Development

No branches or pull requests

1 participant