Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there a limit on max columns (features) that kmcuda can manage? #110

Open
sbushmanov opened this issue Jun 13, 2020 · 0 comments
Open

Comments

@sbushmanov
Copy link

kmcuda runs well until 12'000 features:

from libKMCUDA import kmeans_cuda
from time import time

X = np.random.rand(10, 12000).astype(dtype=np.float32)

start = time()
centers_, labels_ = kmeans_cuda(X, 10)
print(time() - start)

0.19472670555114746

It never finishes with 13'000 ÷ 60'000 features.

It throws an error right away with 70'000+ features:

from libKMCUDA import kmeans_cuda
from time import time

X = np.random.rand(10, 70000).astype(dtype=np.float32)

start = time()
centers_, labels_ = kmeans_cuda(X, 10)
print(time() - start)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-8e783410a8e6> in <module>
     10 
     11 start = time()
---> 12 centers_, labels_ = kmeans_cuda(X, 10)
     13 print(time() - start)

ValueError: "samples": more than 70000 features is not supported

So my question is:

Is there a limit on horizontal dimension kmcuda can manage or I'm missing something?

I'm running Ubuntu 18.04, conda python 3.7 environment, CUDA 10.2, libKMCuda 6.2.3 installed via pip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant