Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add storing result on drive, when keeping only intermidiate result in memory #3

Open
bukson opened this issue Feb 7, 2020 · 2 comments

Comments

@bukson
Copy link
Owner

bukson commented Feb 7, 2020

Is your feature request related to a problem? Please describe.
It helps to calculate correlations for large arrays with little memory footprint

Describe the solution you'd like
Store ready results on drive, using hd5 or something similar

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
https://stackoverflow.com/questions/30109297/pearson-correlation-on-big-numpy-matrices

@bukson
Copy link
Owner Author

bukson commented Feb 7, 2020

@g3rfx
Copy link

g3rfx commented May 8, 2020

I would appreciate this as well since computing a high-dimensional correlation matrix is not possible at the moment due to memory issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants