Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fetching difumo 1024 statistical map can be memory heavy if it is caclculated on the fly #283

Open
xgui3783 opened this issue Feb 23, 2023 · 3 comments
Labels
maintenance Not a bug or breaking issue. Code maintenance related.

Comments

@xgui3783
Copy link
Member

as of a000337

sample code to reproduce:

import siibra
_map = siibra.get_map("difumo 1024", "mni152", "statistical")
_map.fetch("Inferior frontal sulcus middle right")

expected result: returns nii

actual result: process Killed.

@xgui3783 xgui3783 added bug Something isn't working important urgent labels Feb 23, 2023
@xgui3783 xgui3783 changed the title [bug] cannot fetch difumo statistical map [bug] cannot fetch difumo 1024 statistical map Feb 24, 2023
@xgui3783 xgui3783 removed the urgent label Feb 24, 2023
@xgui3783
Copy link
Member Author

update: it seems memory related.

the code ran successfully after a machine restart, but htop is showing a ram usage of ~10GB.

This is an issue for siibra-api, since it runs siibra-python on a resource constrained pod (~2GB).

@AhmetNSimsek
Copy link
Collaborator

Needs to be checked again after #334 is completed.

@AhmetNSimsek
Copy link
Collaborator

as of 0588aca, the issue is resolved for local machines. https://github.com/FZJ-INM1-BDA/siibra-python/tree/wip_clarifySparsemap aims to solve this by committing the precomputed SparseIndex to memory instead of writing to the local cache.

@AhmetNSimsek AhmetNSimsek removed the bug Something isn't working label Jun 29, 2023
@AhmetNSimsek AhmetNSimsek added maintenance Not a bug or breaking issue. Code maintenance related. and removed important labels Nov 16, 2023
@AhmetNSimsek AhmetNSimsek changed the title [bug] cannot fetch difumo 1024 statistical map fetching difumo 1024 statistical map can be memory heavy if it is caclculated on the fly Nov 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Not a bug or breaking issue. Code maintenance related.
Projects
None yet
Development

No branches or pull requests

2 participants