Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

investigate how S7 object size increases #37

Closed
kelly-sovacool opened this issue Jul 2, 2024 · 6 comments
Closed

investigate how S7 object size increases #37

kelly-sovacool opened this issue Jul 2, 2024 · 6 comments
Assignees
@kelly-sovacool
Labels
NIDAP sync with the NIDAP team's Trello board poc proof of concept [ iteration 1 ] reneeTools RepoName
Milestone
2024-08

Comments

@kelly-sovacool
Copy link
Member

as we add raw, filtered, normalized counts, how does the memory usage of the object increase?
@kelly-sovacool kelly-sovacool self-assigned this Jul 2, 2024
@kopardev kopardev added the reneeTools RepoName label Jul 2, 2024
@kelly-sovacool kelly-sovacool added this to the 2024-07 milestone Jul 9, 2024
@kelly-sovacool kelly-sovacool added the poc proof of concept [ iteration 1 ] label Jul 9, 2024
@kelly-sovacool
Copy link
Member Author

kelly-sovacool commented Jul 15, 2024 •
edited
Loading

step up from 10, 50, 200, to 500 samples.
~58,000 genes.

store raw, filtered, and normalized counts.

make pretty plots showing how object memory size increases with increasing samples.

@kelly-sovacool kelly-sovacool modified the milestones: 2024-07, 2024-08 Aug 1, 2024
@kelly-sovacool
Copy link
Member Author

kelly-sovacool commented Aug 6, 2024 •
edited
Loading

Also compare to average Seurat object (can't be the same dataset as for reneeTools, seurat is for single cell). This should take an order of magnitude less.

@phoman14 phoman14 added the NIDAP sync with the NIDAP team's Trello board label Aug 19, 2024
@sync-by-unito sync-by-unito bot added the bug Something isn't working label Aug 19, 2024
@kelly-sovacool
Copy link
Member Author

kelly-sovacool commented Aug 20, 2024 •
edited
Loading

pryr::object_size() https://rdpeng.github.io/RProgDA/the-role-of-physical-memory.html

@sync-by-unito sync-by-unito bot changed the title investigate how S7 object size increases investigate how S7 object size increases PH Aug 20, 2024
@sync-by-unito sync-by-unito bot changed the title investigate how S7 object size increases PH investigate how S7 object size increases Aug 20, 2024
@kelly-sovacool kelly-sovacool removed the bug Something isn't working label Aug 20, 2024
kelly-sovacool added a commit that referenced this issue Aug 27, 2024
@kelly-sovacool
feat: plot memory usage vs n samples
d68ef86 
resolves #37
@kelly-sovacool
Copy link
Member Author

image

@kelly-sovacool
Copy link
Member Author

now with comparison to seurat (single cell) objects

image
image

@kelly-sovacool
Copy link
Member Author

completed

source code for plots: https://github.com/CCBR/reneeTools/blob/7773ad0530d82871b0573c7fc87f0c48842d389a/vignettes/memory.Rmd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kelly-sovacool kelly-sovacool

Labels
NIDAP sync with the NIDAP team's Trello board poc proof of concept [ iteration 1 ] reneeTools RepoName
Projects
None yet
Milestone
2024-08
Development

No branches or pull requests
3 participants
@kopardev @kelly-sovacool @phoman14