Using RunUMAP with Anaconda Python [duplicate]

[GHAStVHenry]

Hi,

I can't get RunUMAP working in RStudio.
I get:

Cannot find UMAP, please install through pip (e.g. pip install umap-learn).

I have installed UMAP, I:

conda install -c conda-forge umap-learn

and that worked, I even tried:

pip install umap-learn

and it just said that all requirements are already satisfied.

Seurat 2.3.1

> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux Server release 6.9 (Santiago)

Matrix products: default
BLAS/LAPACK: /cm/shared/apps/intel/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64_lin/libmkl_gf_lp64.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] Seurat_2.3.1  Matrix_1.2-14 cowplot_0.9.2 ggplot2_2.2.1

loaded via a namespace (and not attached):
  [1] diffusionMap_1.1-0   Rtsne_0.13           VGAM_1.0-5           colorspace_1.3-2     ggridges_0.5.0      
  [6] class_7.3-14         modeltools_0.2-21    mclust_5.3           htmlTable_1.11.2     base64enc_0.1-3     
 [11] proxy_0.4-22         rstudioapi_0.7       DRR_0.0.3            flexmix_2.3-14       lubridate_1.6.0     
 [16] prodlim_2018.04.18   mvtnorm_1.0-6        ranger_0.10.1        codetools_0.2-15     splines_3.4.1       
 [21] R.methodsS3_1.7.1    mnormt_1.5-5         robustbase_0.92-7    knitr_1.20           tclust_1.4-1        
 [26] RcppRoll_0.2.2       jsonlite_1.5         Formula_1.2-2        caret_6.0-80         ica_1.0-2           
 [31] broom_0.4.2          ddalpha_1.3.2        cluster_2.0.6        kernlab_0.9-25       png_0.1-7           
 [36] R.oo_1.21.0          sfsmisc_1.1-2        compiler_3.4.1       backports_1.1.2      assertthat_0.2.0    
 [41] lazyeval_0.2.0       lars_1.2             acepack_1.4.1        htmltools_0.3.6      tools_3.4.1         
 [46] bindrcpp_0.2.2       igraph_1.2.1         gtable_0.2.0         glue_1.2.0           RANN_2.5.1          
 [51] reshape2_1.4.3       dplyr_0.7.5          Rcpp_0.12.17         trimcluster_0.1-2    gdata_2.18.0        
 [56] ape_5.1              nlme_3.1-131         iterators_1.0.9      fpc_2.1-11           lmtest_0.9-36       
 [61] psych_1.7.5          timeDate_3043.102    gower_0.1.2          stringr_1.3.1        irlba_2.3.2         
 [66] gtools_3.5.0         DEoptimR_1.0-8       zoo_1.8-1            MASS_7.3-47          scales_0.4.1        
 [71] ipred_0.9-6          doSNOW_1.0.16        parallel_3.4.1       RColorBrewer_1.1-2   reticulate_1.8      
 [76] pbapply_1.3-4        gridExtra_2.3        segmented_0.5-3.0    rpart_4.1-11         latticeExtra_0.6-28 
 [81] stringi_1.2.2        foreach_1.4.4        checkmate_1.8.5      caTools_1.17.1       lava_1.6.1          
 [86] geometry_0.3-6       dtw_1.20-1           SDMTools_1.1-221     rlang_0.2.1          pkgconfig_2.0.1     
 [91] prabclus_2.2-6       bitops_1.0-6         lattice_0.20-35      ROCR_1.0-7           purrr_0.2.5         
 [96] bindr_0.1.1          recipes_0.1.2        htmlwidgets_1.2      CVST_0.2-1           tidyselect_0.2.4    
[101] plyr_1.8.4.9000      magrittr_1.5         R6_2.2.2             snow_0.4-2           gplots_3.0.1        
[106] Hmisc_4.1-1          dimRed_0.1.0         withr_2.1.2          pillar_1.2.2         foreign_0.8-69      
[111] mixtools_1.1.0       fitdistrplus_1.0-9   survival_2.41-3      scatterplot3d_0.3-40 abind_1.4-5         
[116] nnet_7.3-12          tsne_0.1-3           tibble_1.4.2         KernSmooth_2.23-15   grid_3.4.1          
[121] data.table_1.10.4-3  FNN_1.1              ModelMetrics_1.1.0   metap_0.9            digest_0.6.12       
[126] diptest_0.75-7       tidyr_0.8.1          R.utils_2.5.0        stats4_3.4.1         munsell_0.4.3       
[131] magic_1.5-8  

[GHAStVHenry]

fyi, I am running this on an HPC, I don't have access to write to the source python environment, but I cloned it locally, activated it, installed umap-learn, and in that same instance of the command prompt, while that local environment is activated, I run rtudio to try RunUMAP.

[GHAStVHenry]

I uninstalled all the dependencies installed by

conda install -c conda-forge umap-learn

and reinstalled them with

pip install umap-learn

and it seemed to work... thanks for the help :)

[amisharin]

Same problem here, but reinstalling R and Rstudio did not solve it. umap-learn installed via pip. I can run umap via python from Jupiter. Running on the local machine.

> object <- RunUMAP(object, reduction.use = "pca", dims.use = 1:10)
Error in RunUMAP(object, reduction.use = "pca", dims.use = 1:10) : 
  Cannot find UMAP, please install through pip (e.g. pip install umap-learn).

> sessionInfo()
R version 3.5.1 (2018-07-02)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: OS X El Capitan 10.11.6

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] umap_0.1.0.3   bindrcpp_0.2.2 Seurat_2.3.4   Matrix_1.2-14  cowplot_0.9.3  ggplot2_3.0.0 
[7] reticulate_1.9

loaded via a namespace (and not attached):
  [1] tsne_0.1-3          segmented_0.5-3.0   nlme_3.1-137        bitops_1.0-6       
  [5] bit64_0.9-7         RColorBrewer_1.1-2  httr_1.3.1          prabclus_2.2-6     
  [9] tools_3.5.1         backports_1.1.2     R6_2.2.2            irlba_2.3.2        
 [13] rpart_4.1-13        KernSmooth_2.23-15  Hmisc_4.1-1         lazyeval_0.2.1     
 [17] colorspace_1.3-2    trimcluster_0.1-2   nnet_7.3-12         withr_2.1.2        
 [21] tidyselect_0.2.4    gridExtra_2.3       bit_1.1-14          compiler_3.5.1     
 [25] htmlTable_1.12      hdf5r_1.0.0         labeling_0.3        diptest_0.75-7     
 [29] caTools_1.17.1      scales_0.5.0        checkmate_1.8.5     lmtest_0.9-36      
 [33] DEoptimR_1.0-8      mvtnorm_1.0-8       robustbase_0.93-1   ggridges_0.5.0     
 [37] pbapply_1.3-4       dtw_1.20-1          proxy_0.4-22        stringr_1.3.1      
 [41] digest_0.6.15       mixtools_1.1.0      foreign_0.8-70      R.utils_2.6.0      
 [45] base64enc_0.1-3     pkgconfig_2.0.1     htmltools_0.3.6     htmlwidgets_1.2    
 [49] rlang_0.2.1         rstudioapi_0.7      bindr_0.1.1         zoo_1.8-3          
 [53] jsonlite_1.5        ica_1.0-2           mclust_5.4.1        gtools_3.8.1       
 [57] acepack_1.4.1       dplyr_0.7.6         R.oo_1.22.0         magrittr_1.5       
 [61] modeltools_0.2-22   Formula_1.2-3       lars_1.2            Rcpp_0.12.17       
 [65] munsell_0.5.0       ape_5.1             R.methodsS3_1.7.1   stringi_1.2.3      
 [69] MASS_7.3-50         flexmix_2.3-14      gplots_3.0.1        Rtsne_0.13         
 [73] plyr_1.8.4          grid_3.5.1          parallel_3.5.1      gdata_2.18.0       
 [77] crayon_1.3.4        doSNOW_1.0.16       lattice_0.20-35     splines_3.5.1      
 [81] SDMTools_1.1-221    knitr_1.20          pillar_1.3.0        igraph_1.2.1       
 [85] fpc_2.1-11          reshape2_1.4.3      codetools_0.2-15    stats4_3.5.1       
 [89] glue_1.3.0          metap_0.9           latticeExtra_0.6-28 data.table_1.11.4  
 [93] png_0.1-7           foreach_1.4.4       tidyr_0.8.1         gtable_0.2.0       
 [97] RANN_2.6            purrr_0.2.5         kernlab_0.9-26      assertthat_0.2.0   
[101] class_7.3-14        survival_2.42-6     tibble_1.4.2        snow_0.4-2         
[105] iterators_1.0.10    cluster_2.0.7-1     fitdistrplus_1.0-9  ROCR_1.0-7    

[amisharin]

Hi Gervaise, could you, please, reopen the issue, it'd be nice to have some feedback from Seurat team.

[GHAStVHenry]

you could open your own issue, but ok...

I removed all the dependencies:

conda remove --name umap umap-learn numpy scikit-learn scipy numba llvmlite enum34 singledispatch funcsigs six

then reinstalled with:

pip install umap-learn

then it ran perfectly in R. Probably not the same problem for you though if it works in jupyter-notebook

[mojaveazure]

Please see #486 for dealing with UMAP, Seurat/reticulate, and Anaconda. The short answer is that you can't. Reticulate appears to not recognize Anaconda installations, so if you're planning on using UMAP with Seurat, always use your system's Python (or CPython from the folks bring you Python for Windows users) rather than using Anaconda.

[danielruss]

@mojaveazure I know this is an old question. But I was able to get RunUMAP running with anaconda python. I run
library(reticulate)
use_condaenv(condaenv="Renv", conda="/home/druss/anaconda3/bin/conda")
library(Seurat)
then I can use the RunUMAP no problem. (NOTE: This is in Seurat 3)

of course I installed umap-learn into the Renv conda environment. I used pip to install umap-learn.

[Dragonmasterx87]

@danielruss Could you please outline a sequence of events, from installing anaconda to installing umap-learn, so that the code you have outlined will work?

Much Appreciated!

[chlee-tabin]

To whom it may be worth:

I also had issues with installing UMAP and Leiden algorithm using conda installation, and at least in the local situation (MBP Mojave OSX), I could make it work on conda environment. Although it is tricky Anaconda installation can be recognized by reticulate (I forced it, and the use_condaenv above might work as well, I tried it but failed in my particular case):

1. install clean miniconda3
2. install leidenalg and umap-learn by conda

conda install numpy  # somehow I had to do this 
conda install -c conda-forge leidenalg
conda install -c conda-forge umap-learn

3. create an .Renviron file on your home directory

cat ~/.Renviron
RETICULATE_PYTHON = /Users/XXXX/miniconda3/bin/python

4. Check by starting up R (or RStudio)

reticulate::py_config()

should show you at the end the list of packages and python directory to miniconda3.

5. Finally test that leiden and umap are available:

reticulate::py_module_available(module='leidenalg')
reticulate::py_module_available(module='umap-learn')

P.S. The above method does not work well in an HPC environment (with RStudio server) where some of the package installation require specific libraries that are not available and root installation/update is impossible. I did not do it but the manager succeeded in installing everything (including RStudio etc.) again inside the conda environment and got it work as well.

[danielruss]

@Dragonmasterx87 I installed conda quite awhile ago, so I don't remember all the details. However, I just followed the instructions from docs.conda.io . I installed Anaconda3, so the default is python3. @chlee-tabin installed miniconda, which comes with a smaller set of packages, so I am not surprised that chlee-tabin had to install numpy.

After installing Anaconda3, I created a conda environment

• conda create -n Renv python=3.7
  Then I activated the environment
• conda activate Renv
  you may need to install pip (pip3)
• 'conda install pip'
  Installed umap-learn
• pip install umap-learn

Then in R

• library(reticulate)
• use_condaenv(condaenv="Renv", conda="/home/druss/anaconda3/bin/conda")

The conda = argument has to be set appropriately. I installed conda in my home directory, if you have a central conda command (it is a bash function) you need to know where conda is installed. You can get a clue where the file is by looking in your startup files (/.bash_profile,/.bashrc, /etc/profile/, /etc/bashrc ). The startup file will source the conda file, which is nothing more than a bash function that starts up a python program.

-'umap <- reticulate::import("umap")'
-'umap'

Module(umap)

Hope it helps..
BTW: This was on a linux box, but I also had it running on a macbook using both conda and virtualenv

[shwetak01]

Hi,

This comment is directed to Daniel Russ or the admin. I am new to programming, and trying to get UMAP to run in R studio. I had installed Anaconda distribution of Python, and made use of the directions you posted just above. However, when it came to R part, I only made it to this part
library(reticulate)
I did get this error message: Warning message: package ‘reticulate’ was built under R version 3.5.3

After this, I tried this command: use_condaenv(condaenv="Renv",conda="/home/druss/anaconda3/bin/conda")
Obviously, I need to create the filepath as per where my conda (Anaconda?) is installed. Of course this is where I am flummoxed, as I didn't understand your instructions. Can you please elaborate?
Thanks in advance!

[danielruss]

Hi, First of all a few disclaimers. 1) I am not a member of the Satija lab, just a happy Seurat user, so don’t hold them responsible for anything I say that is incorrect 2) these are my comments and not that of my employer. If you installed anaconda, there should be an “Anaconda” directory in your home directory. When you log in a bash function is created that calls the appropriate conda subcommand. That function is named “conda”. If you are interested, you could type “type conda” and you can see the function definition. When the function is defined, an environment variable _CONDA_EXE is also defined. So you can type “echo $_CONDA_EXE” to see the actual commands conda uses. When I log in, this is what I see: echo $_CONDA_EXE /Users/druss/anaconda3/bin/conda This is what goes into that “conda=” param. The condaenv parameter is which of you conda environments have the umap-learn package installed. When you type conda activate “xxx” the “xxx” is the condaenv. When I type “conda env list” I get # conda environments: # base * /home/druss/anaconda3 Renv /home/druss/anaconda3/envs/Renv gpu-tf /home/druss/anaconda3/envs/gpu-tf I installed umap-learn in the Renv conda environment. Hope it helps and a big thank you to the Satija lab, Daniel From: shwetak01 <notifications@github.com> Reply-To: satijalab/seurat <reply@reply.github.com> Date: Wednesday, June 12, 2019 at 5:59 PM To: satijalab/seurat <seurat@noreply.github.com> Cc: "Russ, Daniel (NIH/CIT) [E]" <druss@mail.nih.gov>, Mention <mention@noreply.github.com> Subject: Re: [satijalab/seurat] Using RunUMAP with Anaconda Python [duplicate] (#631) Hi, This comment is directed to Daniel Russ or the admin. I am new to programming, and trying to get UMAP to run in R studio. I had installed Anaconda distribution of Python, and made use of the directions you posted just above. However, when it came to R part, I only made it to this part library(reticulate) I did get this error message: Warning message: package ‘reticulate’ was built under R version 3.5.3 After this, I tried this command: use_condaenv(condaenv="Renv",conda="/home/druss/anaconda3/bin/conda") Obviously, I need to create the filepath as per where my conda (Anaconda?) is installed. Of course this is where I am flummoxed, as I didn't understand your instructions. Can you please elaborate? Thanks in advance! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#631?email_source=notifications&email_token=AA4X26ZJ5Y4DLLW7EBCMVTDP2FWPDA5CNFSM4FK2XLIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXR5IFA#issuecomment-501470228>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AA4X262QMPK4GNUCJSYH45DP2FWPDANCNFSM4FK2XLIA>.

[shwetak01]

Thank you very much Daniel Russ! I was able to figure out the issue.

[davidasiegel]

Daniel Russ, thank you! Nothing seemed to be working for me but your April 8 comment got it done.

[Dragonmasterx87]

@danielruss sorry for the late reply....I somehow forgot about this, but back then I circumvented this by creating a py3.7 env, and then just running Rstudio in that env. UMAP ran fine in R after that. Just came here for the thumbs up!