Slow

[scseeman]

I have been having a lot of trouble with geneBasis being really slow when searching for hundreds of genes (~300). It seems to progressively get slower and will take upwards of a week to finish. Do you have any suggestions for speed up? Thanks

[nhuhoa]

It took me ~40 mins for 140 genes selection using 8 cores, 32G ram computer, not bad. I am not sure about the execution time if I increase number of genes, ex: 300 genes as you said.
While waiting for developer, maybe some filtering functions are useful?
Ex:
For gene:

• Using 3000 to 5000 highly variable genes as input genes in sce file?
• Using retain_informative_genes() function from this tool

For cells:

• Downsampling cells per each group, to reduce complexity and make sure algo works first. With large # cells, umap estimation takes time...

[scseeman]

Thanks,

It definitely seems to be connected to the size of the panel you are trying to make, though I still can't get speeds that match yours using our high-performance computing cluster. Typically I am running it with 10 cores and 500GB of memory and for a 100 genes still takes several hours. If I ask for 300 genes but give it the first 200 in genes_base, the job timed out after 24 hours only returning 20. That's a rate of less than 1 gene / hour. My sce file has 3000 genes, but does have a lot of cells. I will try reducing the number of cells to speed things up.