ENH: Vectorize ECDF's __call__ method

[Smit-create]

Closes #97

cc @jstac

[pep8speaks]

Hello @Smit-create! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2022-02-15 10:45:20 UTC

[coveralls]

Coverage increased (+0.01%) to 94.461% when pulling 6cc08be on Smit-create:issue_97 into 5be42da on QuantEcon:master.

[jstac]

Hi @Smit-create , many thanks for trying to fix this. What we're looking for is for the following code to execute successfully:

obs = np.random.randn(1000)
e = ECDF(obs)
t = np.linspace(-1, 1, 100)
print(e(t))

The last line should print the value of e(x) for each float x in the array t. (But it should also work of t is just a float.)

Your code still fails this test. I'd be glad if you take another try.

[Smit-create]

I'd be glad if you take another try.

Thanks for the review! I tried to fix that in some other way.

[jstac]

Thanks @Smit-create , it's a nice try. I didn't know about that numpy function.

There's still a slight problem. I wonder if you can help figure it out. If I run

obs = np.random.randn(100)
e = ECDF(obs)
t = np.linspace(0, 1, 10)
print(e(t))

the output looks good, but the dtype of e(t) is np.object rather than float, which will cause performance problems.

What do you think about using these methods? https://numba.pydata.org/numba-doc/latest/user/vectorize.html

We routinely use numba so it's fine to add it as an import here.

[Smit-create]

the output looks good, but the dtype of e(t) is np.object

Oh, I think that's easy to fix. We can use the following diff to fix that:

diff --git a/quantecon/ecdf.py b/quantecon/ecdf.py
index 12a2014..647ce7a 100644
--- a/quantecon/ecdf.py
+++ b/quantecon/ecdf.py
@@ -51,4 +51,4 @@ class ECDF:
         def f(a):
             return np.mean(self.observations <= a)
         vf = np.frompyfunc(f, 1, 1)
-        return vf(x)
+        return vf(x).astype(np.float)

With this above diff I get:

>>> import numpy as np
>>> from quantecon import ECDF
>>> obs = np.random.randn(100)
>>> e = ECDF(obs)
>>> t = np.linspace(0, 1, 10)
>>> print(e(t))
[0.49 0.56 0.6  0.65 0.69 0.71 0.75 0.77 0.8  0.83]
>>> e(t)
array([0.49, 0.56, 0.6 , 0.65, 0.69, 0.71, 0.75, 0.77, 0.8 , 0.83])
>>> t
array([0.        , 0.11111111, 0.22222222, 0.33333333, 0.44444444,
       0.55555556, 0.66666667, 0.77777778, 0.88888889, 1.        ])

What do you think about using these methods?

I am not exactly sure that we need to use vectorize here, because NumPy docs say that it will use broadcasting rules, which in our case may not be applicable always. This commit 22d7f58 used the numpy's vectorize function which was failing on out test in #602 (comment)

[jstac]

Excellent job @Smit-create . Many thanks for this (and sorry for the slow response)!

[Smit-create]

Thanks for the review!