Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Qi: question regarding to monocle3:::find_gene_modules. Thank you so much. #186

Closed
MQMQ2018 opened this issue Aug 17, 2019 · 17 comments
Closed

Qi: question regarding to monocle3:::find_gene_modules. Thank you so much. #186

MQMQ2018 opened this issue Aug 17, 2019 · 17 comments
Labels
waiting on info
Milestone
v0.2.2

Comments

@MQMQ2018
Copy link

Dear Author,

I encountered error messenger showed as below, could you help to give some tips? Thank you so much.

#===#

gene_module_df = monocle3:::find_gene_modules(cds[pr_deg_ids,], resolution=1e-2)

*** caught segfault ***
address 0xfffffffffffffff7, cause 'memory not mapped'

Traceback:
1: RcppParallel::setThreadOptions(numThreads = n_sgd_threads)
2: uwot(X = X, n_neighbors = n_neighbors, n_components = n_components, metric = metric, n_epochs = n_epochs, alpha = learning_rate, scale = scale, init = init, init_sdev = init_sdev, spread = spread, min_dist = min_dist, set_op_mix_ratio = set_op_mix_ratio, local_connectivity = local_connectivity, bandwidth = bandwidth, gamma = repulsion_strength, negative_sample_rate = negative_sample_rate, a = a, b = b, nn_method = nn_method, n_trees = n_trees, search_k = search_k, method = "umap", approx_pow = approx_pow, n_threads = n_threads, n_sgd_threads = n_sgd_threads, grain_size = grain_size, y = y, target_n_neighbors = target_n_neighbors, target_weight = target_weight, target_metric = target_metric, pca = pca, pca_center = pca_center, pcg_rand = pcg_rand, fast_sgd = fast_sgd, ret_model = ret_model, ret_nn = ret_nn, tmpdir = tempdir(), verbose = verbose)
3: uwot::umap(as.matrix(preprocess_mat), n_components = max_components, metric = umap.metric, min_dist = umap.min_dist, n_neighbors = umap.n_neighbors, fast_sgd = umap.fast_sgd, n_threads = cores, verbose = verbose, nn_method = umap.nn_method, ...)
4: monocle3:::find_gene_modules(cds[pr_deg_ids, ], resolution = 0.01)

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
@friedue
Copy link

friedue commented Aug 22, 2019

I have the exact same issue, using the most recent version of the master branch here.

FWIT, here's my error:

gene_module_df <- find_gene_modules(cds[pr_deg_ids,], resolution = 1e-2)

 *** caught segfault ***
address 0xfffffffffffffff7, cause 'memory not mapped'

Traceback:
 1: RcppParallel::setThreadOptions(numThreads = n_sgd_threads)
 2: uwot(X = X, n_neighbors = n_neighbors, n_components = n_components,     metric = metric, n_epochs = n_epochs, alpha = learning_rate,     scale = scale, init = init, init_sdev = init_sdev, spread = spread,     min_dist = min_dist, set_op_mix_ratio = set_op_mix_ratio,     local_connectivity = local_connectivity, bandwidth = bandwidth,     gamma = repulsion_strength, negative_sample_rate = negative_sample_rate,     a = a, b = b, nn_method = nn_method, n_trees = n_trees, search_k = search_k,     method = "umap", approx_pow = approx_pow, n_threads = n_threads,     n_sgd_threads = n_sgd_threads, grain_size = grain_size, y = y,     target_n_neighbors = target_n_neighbors, target_weight = target_weight,     target_metric = target_metric, pca = pca, pca_center = pca_center,     pcg_rand = pcg_rand, fast_sgd = fast_sgd, ret_model = ret_model,     ret_nn = ret_nn, tmpdir = tempdir(), verbose = verbose)
 3: uwot::umap(as.matrix(preprocess_mat), n_components = max_components,     metric = umap.metric, min_dist = umap.min_dist, n_neighbors = umap.n_neighbors,     fast_sgd = umap.fast_sgd, n_threads = cores, verbose = verbose,     nn_method = umap.nn_method, ...)
 4: find_gene_modules(cds[pr_deg_ids, ], resolution = 0.01)

> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-conda_cos6-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux Server release 6.3 (Santiago)

Matrix products: default
BLAS/LAPACK: /pbtech_mounts/homes022/frd2007/miniconda3/envs/monocle/lib/libopenblasp-r0.3.6.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets
[8] methods   base

other attached packages:
 [1] magrittr_1.5                data.table_1.12.2
 [3] monocle3_0.1.3              testthat_2.2.1
 [5] SingleCellExperiment_1.7.4  SummarizedExperiment_1.15.6
 [7] DelayedArray_0.11.4         BiocParallel_1.19.0
 [9] matrixStats_0.54.0          GenomicRanges_1.37.14
[11] GenomeInfoDb_1.21.1         IRanges_2.19.10
[13] S4Vectors_0.23.17           Biobase_2.45.0
[15] BiocGenerics_0.31.5

loaded via a namespace (and not attached):
  [1] Rtsne_0.15               colorspace_1.4-1         deldir_0.1-23
  [4] grr_0.9.5                pryr_0.1.4               class_7.3-15
  [7] rprojroot_1.3-2          XVector_0.25.0           fs_1.3.1
 [10] rstudioapi_0.10          proxy_0.4-23             listenv_0.7.0
 [13] furrr_0.1.0              remotes_2.1.0            ggrepel_0.8.1
 [16] splines_3.6.1            codetools_0.2-16         pscl_1.5.2
 [19] pkgload_1.0.2            speedglm_0.3-2           jsonlite_1.6
 [22] RhpcBLASctl_0.18-205     uwot_0.1.3               pheatmap_1.0.12
 [25] shiny_1.3.2              compiler_3.6.1           httr_1.4.1
 [28] backports_1.1.4          assertthat_0.2.1         Matrix_1.2-17
 [31] lazyeval_0.2.2           limma_3.41.15            cli_1.1.0
 [34] later_0.8.0              htmltools_0.3.6          prettyunits_1.0.2
 [37] tools_3.6.1              igraph_1.2.4.1           coda_0.19-3
 [40] gtable_0.3.0             glue_1.3.1               GenomeInfoDbData_1.2.1
 [43] RANN_2.6.1               reshape2_1.4.3           dplyr_0.8.3
 [46] gmodels_2.18.1           Rcpp_1.0.2               slam_0.1-45
 [49] spdep_1.1-2              gdata_2.18.0             nlme_3.1-141
 [52] DelayedMatrixStats_1.7.1 lmtest_0.9-37            stringr_1.4.0
 [55] globals_0.12.4           ps_1.3.0                 mime_0.7
 [58] irlba_2.3.3              gtools_3.8.1             devtools_2.1.0
 [61] future_1.14.0            LearnBayes_2.15.1        zlibbioc_1.31.0
 [64] MASS_7.3-51.4            zoo_1.8-6                scales_1.0.0
 [67] promises_1.0.1           expm_0.999-4             RColorBrewer_1.1-2
 [70] gridExtra_2.3            memoise_1.1.0            reticulate_1.13
 [73] pbapply_1.4-1            ggplot2_3.2.1            Matrix.utils_0.9.7
 [76] stringi_1.4.3            desc_1.2.0               e1071_1.7-2
 [79] boot_1.3-23              pkgbuild_1.0.4           spData_0.3.0
 [82] rlang_0.4.0              pkgconfig_2.0.2          bitops_1.0-6
 [85] rsample_0.0.5            lattice_0.20-38          purrr_0.3.2
 [88] sf_0.7-6                 htmlwidgets_1.3          processx_3.4.1
 [91] tidyselect_0.2.5         plyr_1.8.4               R6_2.4.0
 [94] generics_0.0.2           DBI_1.0.0                pillar_1.4.2
 [97] withr_2.1.2              units_0.6-3              sp_1.3-1
[100] RCurl_1.95-4.12          tibble_2.1.3             crayon_1.3.4
[103] plotly_4.9.0             viridis_0.5.1            usethis_1.5.1
[106] grid_3.6.1               callr_3.3.1              digest_0.6.20
[109] classInt_0.3-1           pbmcapply_1.5.0          xtable_1.8-4
[112] tidyr_0.8.3              httpuv_1.5.1             RcppParallel_4.4.3
[115] munsell_0.5.0            viridisLite_0.3.0        sessioninfo_1.1.1

@MQMQ2018
Copy link
Author

MQMQ2018 commented Aug 22, 2019 via email

@j-andrews7
Copy link

This is almost certainly a memory issue. How much are you allotting?

@MQMQ2018
Copy link
Author

MQMQ2018 commented Aug 27, 2019 via email

@j-andrews7
Copy link

Depending how many cells you have and how careful you're being with copies of memory-intensive objects, that may not be enough. I have to kick it over 120GB for ~20k cells, though I have a fair amount of other stuff in the environment too.

@friedue
Copy link

friedue commented Aug 27, 2019

I only have about 1800 cells and more than 300 GB memory available...I'll try to see if it would run with a subset, but you're basically saying that you ran into the same error and fixed it by allocating more memory?

@j-andrews7
Copy link

Every time I've ever ran into a segfault due to memory not being mapped or allocated, throwing more memory/fewer cores at it has fixed it. Part of it is due to R's relatively sucky memory management, but a lot of parallelized code tends to increases the memory footprint by a ton - though I doubt that's happening here since you don't appear to be changing cores.

If you have that much memory and are still running into it, then maybe something is broken. This can also occur when R package versions and the version of R being used don't match (i.e. the package was built in another version of R). Uninstalling/reinstalling with the correct version can often help.

@friedue
Copy link

friedue commented Aug 27, 2019

The question is which packages might be clashing...I'm basically using the most recent BioC stuff of everything, but I do rely on devtools::load_all() for loading monocle3, which may cause the issue (however; I sort of doubt that the other OP has a similarly weird set up).

@hpliner
Copy link
Collaborator

hpliner commented Sep 13, 2019

Can you try reinstalling RcppParallel and seeing if that helps?

@hpliner hpliner added this to the v0.2.1 milestone Sep 13, 2019
@ctrapnell
Copy link
Contributor

@MQMQ2018 Do you ever see this behavior when calling reduce_dimension()? I am trying to get a sense for whether it's a problem you encounter with uwot generally, or whether it's particular to find_gene_modules()

@travisalee
Copy link

Hi @ctrapnell I get the same issue when calling reduce_dimension()

cds = reduce_dimension(cds)

*** caught segfault ***
address 0xfffffffffffffff7, cause 'memory not mapped'

Traceback:
1: RcppParallel::setThreadOptions(numThreads = n_threads)
2: uwot(X = X, n_neighbors = n_neighbors, n_components = n_components, metric = metric, n_epochs = n_epochs, alpha = learning_rate, scale = scale, init = init, init_sdev = init_sdev, spread = spread, min_dist = min_dist, set_op_mix_ratio = set_op_mix_ratio, local_connectivity = local_connectivity, bandwidth = bandwidth, gamma = repulsion_strength, negative_sample_rate = negative_sample_rate, a = a, b = b, nn_method = nn_method, n_trees = n_trees, search_k = search_k, method = "umap", approx_pow = approx_pow, n_threads = n_threads, n_sgd_threads = n_sgd_threads, grain_size = grain_size, y = y, target_n_neighbors = target_n_neighbors, target_weight = target_weight, target_metric = target_metric, pca = pca, pca_center = pca_center, pcg_rand = pcg_rand, fast_sgd = fast_sgd, ret_model = ret_model, ret_nn = ret_nn, tmpdir = tempdir(), verbose = verbose)
3: uwot::umap(as.matrix(preprocess_mat), n_components = max_components, metric = umap.metric, min_dist = umap.min_dist, n_neighbors = umap.n_neighbors, fast_sgd = umap.fast_sgd, n_threads = cores, verbose = verbose, nn_method = umap.nn_method, ...)
4: reduce_dimension(cds)

@theboocock
Copy link

theboocock commented Oct 24, 2019 •
edited
Loading

I have the same issue in both reduce_dimension and cluster_genes. It seems to be dependent on the architecture of the system being used. For instance on my macbook pro I have no problems, but on the cluster I use I get the error. It 100% comes from uwot. I can reproduce the error by running this command twice.

library(uwot)
iris_umap <- umap(iris, n_neighbors = 50, learning_rate = 0.5, init = "random")
iris_umap <- umap(iris, n_neighbors = 50, learning_rate = 0.5, init = "random")

On the second time I get the error "address 0xfffffffffffffff7, cause 'memory not mapped'"

If anyone is interested I did make a version of monocle3 that doesn't use uwot but uses the umap package. Its a bit slower but it seems to work fine. A workaround could be to have the option to specify the umap package to use.

https://github.com/theboocock/monocle3

@MLKaufman
Copy link

I had this same problem. I was unable to identify the root, but was able to figure out that it was being caused by the RcppParallel (RcppParallel_4.4.4) installed directly from CRAN. I tried to uninstall and reinstall RcppParallel and all of its components, but still did not work.

Only got past this issue when I installed the same version number using conda and the conda-forge channel, the problem goes away.

conda install r-rcppparallel -c conda-forge

Fixed the crashing of R.

@aldojongejan aldojongejan mentioned this issue Dec 2, 2019
Closed
@jlmelville
Copy link

I can't reproduce the problem, but a new version of uwot (0.1.5) has just been accepted by CRAN, which I hope fixes the underlying cause.

If you are still having issues with this, please retry with the new version. If it still doesn't work, I would also appreciate a bug report (e.g. at jlmelville/uwot#39). I don't check the monocle3 bug reports (I didn't know monocle used uwot) and so didn't see @theboocock's efforts to reproduce the issue until a couple of days ago.

@jlmelville
Copy link

If you are still having issues with crashing, it's likely the new version of uwot won't help you. The good news is that thanks to @theboocock's willingness to do some experimentation with valgrind, we have narrowed down the problem to RcppParallel and @MLKaufman's solution should work: i.e. if you are using anaconda to manage R and its packages, you need to install RcppParallel using conda, not the CRAN version. Conversely, you should probably also avoid using conda to install RcppParallel if you are otherwise installing R packages from CRAN.

benjjneb/dada2#684 describes a very similar issue, so there might be more useful details to be gleaned from there.

@jlmelville
Copy link

Me again. Just a note that the latest version of uwot (0.1.8) no longers uses RcppParallel so problems around installing it should have been solved by the slightly drastic measure of not installing it.

@brgew
Copy link
Collaborator

brgew commented Apr 23, 2020

Is it OK to close this issue?

@hpliner hpliner closed this as completed May 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
waiting on info
Projects
None yet
Milestone
v0.2.2
Development

No branches or pull requests
10 participants
@ctrapnell @theboocock @jlmelville @friedue @hpliner @j-andrews7 @brgew @MLKaufman @MQMQ2018 @travisalee