Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failure to evaluate high dimensional data #167

Open
Leon-Noirclerc opened this issue Jul 16, 2024 · 1 comment
Open

Failure to evaluate high dimensional data #167

Leon-Noirclerc opened this issue Jul 16, 2024 · 1 comment

Comments

@Leon-Noirclerc
Copy link

Hello everyone,
I am trying to evaluate a fitted FFTKDE on high dimensional data ( dimension greater than 10), but this fails with an AssertionError.
The error comes from the method autogrid in KDEpy/utils.py. When passing an empty grid it calls:

 if num_points is None:
    num_points = [int(np.power(1024, 1 / dims))] * dims

Which is a list of 1 for dims greater than 10 and will trigger the assert points >= 2 on line 126.
However in the feature summary of your documentation I saw that the supported number of dimensions is set to Any.
Should I evaluate the FFTKDE differently on high dimensional data ? What would be your recommendation here ?
Thanks !

@tommyod
Copy link
Owner

tommyod commented Jul 21, 2024

Create a custom grid, or give a num_points that is large enough.

Beware that FFTKDE discretizes the data onto the grid, and in high dimension many grid points might be empty (no data point is close to it) so you need a very fine grid in order to avoid discretization errors. Using KDEpy.TreeKDE or even scipy.stats.gaussian_kde might be more reasonable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants