Fail to install nevergrad following demo

[kennychen83]

Describe issue

I tried to copy-paste demo locally to my Rstudio. However, I haven't been able to get pass the stage of installing nevergrad. Any help is appreciated.

1. USE pip

remotes::install_github("facebookexperimental/Robyn/R")
library(Robyn)
library(reticulate)
set.seed(123)

Sys.setenv(R_FUTURE_FORK_ENABLE="true")
options(future.fork.enable = TRUE)

## Option 1: nevergrad installation via PIP
virtualenv_create("r-reticulate")
use_virtualenv("r-reticulate", required = TRUE)
py_install("nevergrad", pip = TRUE)

Using virtual environment '/Users/kennychen/.virtualenvs/r-reticulate' ...
Requirement already satisfied: nevergrad in ./.virtualenvs/r-reticulate/lib/python3.8/site-packages (0.4.3.post8)
Requirement already satisfied: cma>=2.6.0 in ./.virtualenvs/r-reticulate/lib/python3.8/site-packages (from nevergrad) (3.1.0)
Requirement already satisfied: numpy>=1.15.0 in ./.virtualenvs/r-reticulate/lib/python3.8/site-packages (from nevergrad) (1.21.2)
Requirement already satisfied: typing-extensions>=3.6.6 in ./.virtualenvs/r-reticulate/lib/python3.8/site-packages (from nevergrad) (3.10.0.2)
Requirement already satisfied: bayesian-optimization>=1.2.0 in ./.virtualenvs/r-reticulate/lib/python3.8/site-packages (from nevergrad) (1.2.0)
Requirement already satisfied: scikit-learn>=0.18.0 in ./.virtualenvs/r-reticulate/lib/python3.8/site-packages (from bayesian-optimization>=1.2.0->nevergrad) (1.0)
Requirement already satisfied: scipy>=0.14.0 in ./.virtualenvs/r-reticulate/lib/python3.8/site-packages (from bayesian-optimization>=1.2.0->nevergrad) (1.7.1)
Requirement already satisfied: joblib>=0.11 in ./.virtualenvs/r-reticulate/lib/python3.8/site-packages (from scikit-learn>=0.18.0->bayesian-optimization>=1.2.0->nevergrad) (1.1.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in ./.virtualenvs/r-reticulate/lib/python3.8/site-packages (from scikit-learn>=0.18.0->bayesian-optimization>=1.2.0->nevergrad) (3.0.0)

But it fails during Step 3 (run robyn_run):

Using robyn object location: /Users/kennychen/Desktop
Input data has 208 weeks in total: 2015-11-23 to 2019-11-11
Initial model is built on rolling window of 92 weeks: 2016-11-21 to 2018-08-20
Using geometric adstocking with 18 hyperparameters & 10-fold ridge x-validation on 6 cores
>>> Start running 5 trials with 2000 iterations per trial each with TwoPointsDE nevergrad algorithm...
 Running trial nr. 1 

Error in robyn_mmm(hyper_collect = InputCollect$hyperparameters, InputCollect = InputCollect,  : 
  You must have nevergrad python library installed.

2. USE conda

remotes::install_github("facebookexperimental/Robyn/R")
library(Robyn)
library(reticulate)
set.seed(123)

Sys.setenv(R_FUTURE_FORK_ENABLE="true")
options(future.fork.enable = TRUE)

## Option 2: nevergrad installation via conda
conda_create("r-reticulate") # must run this line once
use_condaenv("r-reticulate")
conda_install("r-reticulate", "nevergrad", pip=TRUE)

and this takes forever with some following messages:

ERROR: Command errored out with exit status 1: /Users/kennychen/Library/r-miniconda/envs/r-reticulate/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/cl/dh6ng9n54497kq338mr97xv40000gp/T/pip-install-2cnal0_e/scipy_89b10482d6b64757a968ef573987162c/setup.py'"'"'; __file__='"'"'/private/var/folders/cl/dh6ng9n54497kq338mr97xv40000gp/T/pip-install-2cnal0_e/scipy_89b10482d6b64757a968ef573987162c/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/cl/dh6ng9n54497kq338mr97xv40000gp/T/pip-record-5w5zk2sn/install-record.txt --single-version-externally-managed --prefix /private/var/folders/cl/dh6ng9n54497kq338mr97xv40000gp/T/pip-build-env-i6uajlu6/overlay --compile --install-headers /private/var/folders/cl/dh6ng9n54497kq338mr97xv40000gp/T/pip-build-env-i6uajlu6/overlay/include/python3.10/scipy Check the logs for full command output.
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/e1/4d/15c3542a17eebf61e48bd71dc55b5f3b5031f1cd0dc4aad1ff9ac9651e49/scikit-learn-1.0.tar.gz#sha256=776800194e757cd212b47cd05907e0eb67a554ad333fe76776060dbb729e3427 (from https://pypi.org/simple/scikit-learn/) (requires-python:>=3.7). Command errored out with exit status 1: /Users/kennychen/Library/r-miniconda/envs/r-reticulate/bin/python /private/var/folders/cl/dh6ng9n54497kq338mr97xv40000gp/T/pip-standalone-pip-5z4uruie/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /private/var/folders/cl/dh6ng9n54497kq338mr97xv40000gp/T/pip-build-env-i6uajlu6/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel 'Cython>=0.28.5' 'oldest-supported-numpy; python_version!='"'"'3.7'"'"' or platform_machine=='"'"'aarch64'"'"' or platform_system=='"'"'AIX'"'"' or platform_python_implementation == '"'"'PyPy'"'"'' 'numpy==1.14.6; python_version=='"'"'3.7'"'"' and platform_machine!='"'"'aarch64'"'"' and platform_system!='"'"'AIX'"'"' and platform_python_implementation != '"'"'PyPy'"'"'' 'scipy>=1.1.0' Check the logs for full command output.

 Using cached scikit-learn-0.24.2.tar.gz (7.5 MB)
  Installing build dependencies: started
  Installing build dependencies: still running...
  Installing build dependencies: still running...

3. USE use_python with pip

## In case nevergrad still can't be imported after installation,
## please locate your python file and run this line with your path:
use_python("~/Library/Frameworks/Python.framework/Versions/3.10/bin/python3")
virtualenv_create("r-reticulate")
use_virtualenv("r-reticulate", required = TRUE)
py_install("nevergrad", pip = TRUE)

still can't run robyn_run:

Using robyn object location: /Users/kennychen/Desktop
Input data has 208 weeks in total: 2015-11-23 to 2019-11-11
Initial model is built on rolling window of 92 weeks: 2016-11-21 to 2018-08-20
Using geometric adstocking with 18 hyperparameters & 10-fold ridge x-validation on 6 cores
>>> Start running 5 trials with 2000 iterations per trial each with TwoPointsDE nevergrad algorithm...
 Running trial nr. 1 

Error in robyn_mmm(hyper_collect = InputCollect$hyperparameters, InputCollect = InputCollect,  : 
  You must have nevergrad python library installed.

4. USE use_python with conda
   Same message with 2.

5. Force Python path for reticulate:
   Not working either.

Environment & Robyn version

R version 4.1.1
Please make sure you're using the latest Robyn version -- YES

[kian-cr-ft]

I'm having the same issue. I'm on OSX and using Robyn 3.0.0.
reticulate can't see the downloaded nevergrad file within the VM even though it has been successfully downloaded by pip and placed correctly in the VM lib/python3.9/site-packages folder.

I tested exactly the same with pandas - from download to install - and it worked fine. It populated the same folder in the VM.

R: 3.6.3
Reticulate: 1.22

library(reticulate)
use_python("~/.virtualenvs/r-reticulate2/bin/python3.9")
use_virtualenv("r-reticulate2", required = TRUE)

py_install("nevergrad", pip = TRUE, envname = "r-reticulate2")
py_module_available("nevergrad") # FALSE

py_install('pandas', pip = TRUE, envname = "r-reticulate2")
py_module_available("pandas") # TRUE

[gufengzhou]

environment issue is really quite tricky. Have you tried forcing python path with this? Sys.setenv(RETICULATE_PYTHON = "~/Library/r-miniconda/envs/r-reticulate/bin/python3.9") ? Adapted to your path of course. Also can you share the outcome of reticulate::py_config()? Mine looks like this:

python: /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate/bin/python
libpython: /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate/lib/libpython3.9.dylib
pythonhome: /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate:/Users/gufengzhou/Library/r-miniconda/envs/r-reticulate
version: 3.9.6 | packaged by conda-forge | (default, Jul 11 2021, 03:36:15) [Clang 11.1.0 ]
numpy: /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate/lib/python3.9/site-packages/numpy
numpy_version: 1.21.1

[gufengzhou]

I believe most Nevergrad installation issue is related to python path specification in reticulate. Could you check this issue and see if it helps?

rstudio/reticulate#367

[Tsurty]

Hello guys, I'm facing similar issues too.
I've now tried with the reticulate::py_config() but what I get is what you can see below.

However, going into these folders via the finder reveleas that everything is there as per your output @gufengzhou (an example below)

I've found these thread out rstudio/reticulate#637
And tried this one command instead reticulate::py_discover_config()

Which actually turned out to print the required output
python: /Users/cristian/.virtualenvs/r-reticulate/bin/python
libpython: [NOT FOUND]
pythonhome: /Users/cristian/.virtualenvs/r-reticulate:/Users/cristian/.virtualenvs/r-reticulate
version: 3.8.9 (default, Aug 21 2021, 15:53:23) [Clang 13.0.0 (clang-1300.0.29.3)]
numpy: /Users/cristian/.virtualenvs/r-reticulate/lib/python3.8/site-packages/numpy
numpy_version: 1.21.2

NOTE: Python version was forced by use_python function

So now I'm trying to understand how to fix that [NOT FOUND] as I'm yet to find the correct way.

[gufengzhou]

To force python path, please try this Sys.setenv(RETICULATE_PYTHON = "~/Library/r-miniconda/envs/r-reticulate/bin/python3.9"), adapted to your path and python version of course.

Then restart R session, re-create and use virtual environment, either via conda with conda_create("r-reticulate") & use_condaenv("r-reticulate") or via PIP with virtualenv_create("r-reticulate") & use_virtualenv("r-reticulate", required = TRUE).

Then re-check py_config() to see if everything found

[Tsurty]

I tried as you suggested @gufengzhou

The env was correctly created (I suppose)

But then py-config returns just the same

I noticed I've bot Anaconda and Miniconda installed may this be an issue?

Moreover if I try conda env list in the terminal it shows the path to the virtual env but not the "r-reticulate" name next to it.

P.S. I'm Cristian Nozzi from the facebook group anyway

[gufengzhou]

This is new to me that not even py_config is working. Could you also try reticulate::py_discover_config()?

[Tsurty]

Yes this still works like before.

[Tsurty]

Sorry, sent the previous one for error.
It's finding a lot of python versions tho, should I delete some of them?

Moreover I'm still trying to fix the [NOT FOUND] lib

[Tsurty]

So guys after many failed tries on fixing the [NOT FOUND] error on the libpython i came to understand that it was probably using a wrong version of python and not the one I meant to.

I restarted R Session again an tried the following

library(Robyn)
library(reticulate)
Sys.setenv(RETICULATE_PYTHON = "/Users/cristian/opt/miniconda3/envs/r-reticulate/bin/python3.10")

@gufengzhou the difference here is that I restarted first and then launched this command (earlier I did the reverse).
This time it worked but the, trying to reticulate::py_config() took me back to my very first error that got me stuck on the 2.0 version of Robyn.

Again, I'm using a M1 Macbook Pro so I'm afraid this may be some CPU limitations here.

[gufengzhou]

Ok I just got it solved...I also ran into nevergrad install problem after I've updated python to 3.10 that is still too new and probably some dependencies are not updated yet. I'm using a macbook.

First, I've removed python 3.10 by removing the folder /Applications/python 3.10 into trash can, then I've opened /Users/gufengzhou/.bash_profile and removed this whole section at the bottom:

# Setting PATH for Python 3.10
# The original version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.10/bin:${PATH}"
export PATH

Then I've uninstalled and re-installed miniconda by just downloading the latest .pkg installer here

Then restart Rstudio, load reticulate library(reticulate) and run py_config(), where the environment is created with an older python version, but well it works... below is the output:

Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate

  added / updated specs:
    - numpy
    - python=3.6


The following NEW packages will be INSTALLED:

  ca-certificates    conda-forge/osx-64::ca-certificates-2021.10.8-h033912b_0
  certifi            conda-forge/osx-64::certifi-2016.9.26-py36_0
  libblas            conda-forge/osx-64::libblas-3.9.0-12_osx64_openblas
  libcblas           conda-forge/osx-64::libcblas-3.9.0-12_osx64_openblas
  libcxx             conda-forge/osx-64::libcxx-12.0.1-habf9029_0
  libffi             conda-forge/osx-64::libffi-3.4.2-he49afe7_4
  libgfortran        conda-forge/osx-64::libgfortran-5.0.0-9_3_0_h6c81a4c_23
  libgfortran5       conda-forge/osx-64::libgfortran5-9.3.0-h6c81a4c_23
  liblapack          conda-forge/osx-64::liblapack-3.9.0-12_osx64_openblas
  libopenblas        conda-forge/osx-64::libopenblas-0.3.18-openmp_h3351f45_0
  libzlib            conda-forge/osx-64::libzlib-1.2.11-h9173be1_1013
  llvm-openmp        conda-forge/osx-64::llvm-openmp-12.0.1-hda6cdc1_1
  ncurses            conda-forge/osx-64::ncurses-6.2-h2e338ed_4
  numpy              conda-forge/osx-64::numpy-1.19.5-py36h08b5fde_2
  openssl            conda-forge/osx-64::openssl-1.1.1l-h0d85af4_0
  pip                conda-forge/noarch::pip-21.3-pyhd8ed1ab_0
  python             conda-forge/osx-64::python-3.6.13-haf480d7_2_cpython
  python_abi         conda-forge/osx-64::python_abi-3.6-2_cp36m
  readline           conda-forge/osx-64::readline-8.1-h05e3726_0
  setuptools         conda-forge/osx-64::setuptools-49.6.0-py36h79c6626_3
  sqlite             conda-forge/osx-64::sqlite-3.36.0-h23a322b_2
  tk                 conda-forge/osx-64::tk-8.6.11-h5dbffcc_1
  wheel              conda-forge/noarch::wheel-0.37.0-pyhd8ed1ab_1
  xz                 conda-forge/osx-64::xz-5.2.5-haf1e3a3_1
  zlib               conda-forge/osx-64::zlib-1.2.11-h9173be1_1013


Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
#
# To activate this environment, use
#
#     $ conda activate /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate
#
# To deactivate an active environment, use
#
#     $ conda deactivate

python:         /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate/bin/python
libpython:      /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate/lib/libpython3.6m.dylib
pythonhome:     /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate:/Users/gufengzhou/Library/r-miniconda/envs/r-reticulate
version:        3.6.13 | packaged by conda-forge | (default, Sep 23 2021, 07:55:15)  [GCC Clang 11.1.0]
numpy:          /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/numpy
numpy_version:  1.19.5

Finally, running reticulate::py_install("nevergrad", pip = TRUE) gets it installed.

[laresbernardo]

Hi there. I've just installed Robyn in my new MacBook Pro, with M1 chip and ran smoothly. These were the steps I followed:

1. remotes::install_github('facebookexperimental/Robyn/R')
2. library("Robyn")
   Here I got a weird error. Something about data.tables and its default options. I simply closed and reopened RStudio.
3. library("Robyn")
4. Installed nevergrad with a virtual environment (no conda needed). Note that I never set manually my python version but reticulate automatically set it up for me:

reticulate::virtualenv_create("r-reticulate")
reticulate::use_virtualenv("r-reticulate", required = TRUE)
reticulate::py_install("nevergrad", pip = TRUE)

5. Ran the demo.R file and everything ran OK.

For further details, I'm sharing my sessionInfo() and py_config():

R version 4.1.1 (2021-08-10)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur 11.6

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] doRNG_1.8.2     rngtools_1.5.2  doFuture_0.12.0 future_1.22.1   foreach_1.5.1   Robyn_3.0.0    

loaded via a namespace (and not attached):
 [1] nlme_3.1-152         matrixStats_0.61.0   lubridate_1.8.0      doParallel_1.0.16   
 [5] RColorBrewer_1.1-2   rprojroot_2.0.2      rstan_2.21.2         tools_4.1.1         
 [9] utf8_1.2.2           R6_2.5.1             DBI_1.1.1            lazyeval_0.2.2      
[13] mgcv_1.8-36          colorspace_2.0-2     withr_2.4.2          tidyselect_1.1.1    
[17] gridExtra_2.3        prettyunits_1.1.1    processx_3.5.2       curl_4.3.2          
[21] compiler_4.1.1       glmnet_4.1-2         textshaping_0.3.5    cli_3.0.1           
[25] labeling_0.4.2       scales_1.1.1         callr_3.7.0          systemfonts_1.0.2   
[29] stringr_1.4.0        digest_0.6.28        StanHeaders_2.21.0-7 extraDistr_1.9.1    
[33] pkgconfig_2.0.3      parallelly_1.28.1    rlang_0.4.12         rstudioapi_0.13     
[37] rPref_1.3            shape_1.4.6          prophet_1.0          generics_0.1.0      
[41] farver_2.1.0         jsonlite_1.7.2       dplyr_1.0.7          inline_0.3.19       
[45] magrittr_2.0.1       loo_2.4.1            patchwork_1.1.1      Matrix_1.3-4        
[49] Rcpp_1.0.7           munsell_0.5.0        fansi_0.5.0          reticulate_1.22     
[53] lifecycle_1.0.1      stringi_1.7.5        pkgbuild_1.2.0       grid_4.1.1          
[57] parallel_4.1.1       listenv_0.8.0        crayon_1.4.1         lattice_0.20-44     
[61] splines_4.1.1        ps_1.6.0             pillar_1.6.4         igraph_1.2.7        
[65] codetools_0.2-18     stats4_4.1.1         glue_1.4.2           V8_3.4.2            
[69] data.table_1.14.2    remotes_2.4.1        RcppParallel_5.1.4   png_0.1-7           
[73] vctrs_0.3.8          nloptr_1.2.2.2       gtable_0.3.0         purrr_0.3.4         
[77] tidyr_1.1.4          assertthat_0.2.1     ggplot2_3.3.5        ragg_1.1.3          
[81] survival_3.2-11      minpack.lm_1.2-1     tibble_3.1.5         iterators_1.0.13    
[85] globals_0.14.0       ellipsis_0.3.2       here_1.0.1  

> reticulate::py_config()
python:         /Users/bernardolares/.virtualenvs/r-reticulate/bin/python
libpython:      /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/config-3.8-darwin/libpython3.8.dylib
pythonhome:     /Users/bernardolares/.virtualenvs/r-reticulate:/Users/bernardolares/.virtualenvs/r-reticulate
version:        3.8.9 (default, Aug 21 2021, 15:53:23)  [Clang 13.0.0 (clang-1300.0.29.3)]
numpy:          /Users/bernardolares/.virtualenvs/r-reticulate/lib/python3.8/site-packages/numpy
numpy_version:  1.21.2
nevergrad:      /Users/bernardolares/.virtualenvs/r-reticulate/lib/python3.8/site-packages/nevergrad

NOTE: Python version was forced by use_python function

[Tsurty]

Hello guys I'll try to keep this short and share my experience so far hoping this can help somebody.
I've been testing for ours starting with @gufengzhou's solution but did not work.

In particular I deleted Python3.10, and configured 3.8.
Deleted miniconda everywhere (like seriously there's no trace in hidden files and such) as well as conda actually but when I try to install miniconda again it tells me "It's already installed in /opt/miniconda3" which was the old path which is not in the $PATH variable, the folder doesn't exist, there's no file referencing it so I'm a bit lost on this.

Then I saw @laresbernardo's solution so I decided to give it a try.
I noticed in particular that the "libpython" item in "py_config()" was being referenced inside Xcode's folder and that's not the first time I've come across this in the past few days.

So I decided to give it a try: I've installed Xcode, created a dummy app, launched once and then closed everything.
After I did this I managed to execute reticulate::py_config() (it used to return an error and tell me I had no shared python library).

So now my py_config works fine and automatically referenced the Xcode's libpython

py_config()
python: /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3.8
libpython: /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/config-3.8-darwin/libpython3.8.dylib
pythonhome: /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8:/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8
version: 3.8.9 (default, Aug 21 2021, 15:53:23) [Clang 13.0.0 (clang-1300.0.29.3)]
numpy: [NOT FOUND]

NOTE: Python version was forced by RETICULATE_PYTHON

reticulate::py_config()
python: /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3.8
libpython: /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/config-3.8-darwin/libpython3.8.dylib
pythonhome: /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8:/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8
version: 3.8.9 (default, Aug 21 2021, 15:53:23) [Clang 13.0.0 (clang-1300.0.29.3)]
numpy: [NOT FOUND]

NOTE: Python version was forced by RETICULATE_PYTHON

I had to force the Python version in order to be sure I was getting the right now.
Now my nevergrad install still fails but I think it's due to numpy's problems.

In particular, between the errors I'm getting, the reason should be this:
numpy.distutils.system_info.NotFoundError: No lapack/blas resources found. Note: Accelerate is no longer supported.

I'm now trying to fix this out and make numpy install with BLAS.

Will keep you posted.

EDIT:
Restarting R Session and not forcing Python path returns me this

reticulate::py_config()
python: /Users/cristian/.virtualenvs/r-reticulate/bin/python
libpython: /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/config-3.8-darwin/libpython3.8.dylib
pythonhome: /Users/cristian/.virtualenvs/r-reticulate:/Users/cristian/.virtualenvs/r-reticulate
version: 3.8.9 (default, Aug 21 2021, 15:53:23) [Clang 13.0.0 (clang-1300.0.29.3)]
numpy: /Users/cristian/.virtualenvs/r-reticulate/lib/python3.8/site-packages/numpy
numpy_version: 1.21.2

python versions found:
/Users/cristian/.virtualenvs/r-reticulate/bin/python
/usr/bin/python3
/usr/bin/python
/Users/cristian/.virtualenvs/r-reticulate-new/bin/python

[Lockee45]

Hey guys, I was also experiencing an error message when trying to install nevergrad with conda.

Problem was coming from Python 3.10 package in my r-reticulate env.

I have just run:

conda_create("r-reticulate","Python 3.8")

to change the version of the Python package, and all is working now. Seems you can use all previous version up to 3.9

Hope it helps.

[Tsurty]

After many testes I'm still stuck here on my M1 Macbook.
I'm now on Python3.8 which shouldn't be a problem, Nevergrad seems installed correctly but yet Robyn won't run.

This is what happens when I try to py_config

Besides the fact that I don't get why py_config itself is not working, I noticed that on @laresbernardo's output there's "nevergrad" right after "numpy" which is missing on mine.

P.S.In the meantime I borrowed an older 2016 Macbook Pro (with Intel CPU ofc) on which I managed to make Robyn run smoothly with one little trick: I was still having troubles with the missing "shared Python library" error.
Simply installing Xcode (without even running it once) fixed it and now it works fine.

[kian-cr-ft]

environment issue is really quite tricky. Have you tried forcing python path with this? Sys.setenv(RETICULATE_PYTHON = "~/Library/r-miniconda/envs/r-reticulate/bin/python3.9") ? Adapted to your path of course. Also can you share the outcome of reticulate::py_config()? Mine looks like this:

python: /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate/bin/python libpython: /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate/lib/libpython3.9.dylib pythonhome: /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate:/Users/gufengzhou/Library/r-miniconda/envs/r-reticulate version: 3.9.6 | packaged by conda-forge | (default, Jul 11 2021, 03:36:15) [Clang 11.1.0 ] numpy: /Users/gufengzhou/Library/r-miniconda/envs/r-reticulate/lib/python3.9/site-packages/numpy numpy_version: 1.21.1

Thanks, this actually solved it for me. For reference I:

1. Restarted the session, and reopened RStudio
2. Then

Sys.setenv(RETICULATE_PYTHON = "~/.virtualenvs/r-reticulate2/bin/python3")
use_python("~/.virtualenvs/r-reticulate2/bin/python3.9")
use_virtualenv("r-reticulate2", required = TRUE)
py_install("nevergrad", pip = TRUE, envname = "r-reticulate2")
py_module_available("nevergrad") # TRUE

My py_config was

> py_config()
python:         /Users/ME/.virtualenvs/r-reticulate2/bin/python3
libpython:      /usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/config-3.9-darwin/libpython3.9.dylib
pythonhome:     /Users/ME/.virtualenvs/r-reticulate2:/Users/ME/.virtualenvs/r-reticulate2
version:        3.9.1 (default, Feb  3 2021, 07:04:15)  [Clang 12.0.0 (clang-1200.0.32.29)]
numpy:          /Users/ME/.virtualenvs/r-reticulate2/lib/python3.9/site-packages/numpy
numpy_version:  1.21.2
nevergrad:      /Users/ME/.virtualenvs/r-reticulate2/lib/python3.9/site-packages/nevergrad

NOTE: Python version was forced by RETICULATE_PYTHON```

[Met0o]

Hey guys, I was also experiencing an error message when trying to install nevergrad with conda.

Problem was coming from Python 3.10 package in my r-reticulate env.

I have just run:

conda_create("r-reticulate","Python 3.8")

to change the version of the Python package, and all is working now. Seems you can use all previous version up to 3.9

Hope it helps.

Thank you. This fix helped me resolve the following error:

"Error in py_module_import(module, convert = convert) : ImportError: DLL load failed while importing _ctypes: The specified module could not be found."

Recreating the r-reticulate environment with python version 3.8 resolved the problem.

[andrey-legayev]

Win10:
I've tried many ways and found that the most easy one is to use Nevergrad via system Python.
Without venv / conda / miniconda / anaconda.
There is no special code required for making it work after proper installation - it just works in R Console, R Studio and in PyCharm with R plugin.

I hope it will help someone -

Win10 Instructions

1. Cleanup:
   • uninstall all Python installations
   • uninstall Miniconda
   • remove folders:
     %HOMEPATH%\.conda
     %HOMEPATH%\Documents\.virtualenvs
     %HOMEPATH%\AppData\Local\r-reticulate
     %HOMEPATH%\AppData\Local\r-miniconda
     any other folders related to venv / miniconda
2. Install latest Python 3.9.* from here: https://www.python.org/downloads/windows/
3. Run powershell as admin: Win+X -> Windows PowerShell (Admin)
4. Install pip: py -m ensurepip --upgrade
5. Upgrade pip: py -m pip install --upgrade pip
6. Install nevergrad: pip install nevergrad
7. Test Nevergrad availability in R console:
   • Run R console
   • Run code:

install.packages('reticulate')
library(reticulate)
if (!reticulate::py_module_available("nevergrad")) {
  cat("This is yout current python config:\n")
  print(reticulate::py_config())
  stop("Nevergrad is not available. Python config has been printed above.")
} else {
  cat("You have nevergrad and it is available to R!\n")
}

8. If you see following question in console - answer “n” (No)

No non-system installation of Python could be found.
Would you like to download and install Miniconda?
Miniconda is an open source environment management system for Python.
See https://docs.conda.io/en/latest/miniconda.html for more details.

Would you like to install Miniconda? [Y/n]: n

[tgtod002]

1. HI Guys,

I have an HP Windows laptop. Running Anaconda 2.1.1
I was finally able to install Nevergrad using the following method:

1. Create a new environment ( i named mine RStudio)
2. Install RStudio
3. Install Powershell Prompt
4. Environments --> RStudio(name of your environment) --> Not installed (searched for reticulate) -- shows up r-reticulate , choose and then apply)
5. Switch to Powershell: in command prompt: pip install nevergrad

[amysheep]

I had similar issue. After I pointed use_python to the right python version and restart a new R session. py_config was able to pick up the correct version and everything worked.

[wamm0909]

After many testes I'm still stuck here on my M1 Macbook. I'm now on Python3.8 which shouldn't be a problem, Nevergrad seems installed correctly but yet Robyn won't run.

This is what happens when I try to py_config

Besides the fact that I don't get why py_config itself is not working, I noticed that on @laresbernardo's output there's "nevergrad" right after "numpy" which is missing on mine.

P.S.In the meantime I borrowed an older 2016 Macbook Pro (with Intel CPU ofc) on which I managed to make Robyn run smoothly with one little trick: I was still having troubles with the missing "shared Python library" error. Simply installing Xcode (without even running it once) fixed it and now it works fine.

my issue display like you:"there's "nevergrad" right after "numpy" which is missing on mine. "do you solve this?ths!

[Tsurty]

After many testes I'm still stuck here on my M1 Macbook. I'm now on Python3.8 which shouldn't be a problem, Nevergrad seems installed correctly but yet Robyn won't run.

This is what happens when I try to py_config

Besides the fact that I don't get why py_config itself is not working, I noticed that on @laresbernardo's output there's "nevergrad" right after "numpy" which is missing on mine.
P.S.In the meantime I borrowed an older 2016 Macbook Pro (with Intel CPU ofc) on which I managed to make Robyn run smoothly with one little trick: I was still having troubles with the missing "shared Python library" error. Simply installing Xcode (without even running it once) fixed it and now it works fine.

my issue display like you:"there's "nevergrad" right after "numpy" which is missing on mine. "do you solve this?ths!

Nope, I've done some other tries over time with no luck (tried uninstalling and re-installing everything aswell) but I now had the chance to switch to an older intel-based mackbook which worked just fine right from the beginning using conda.

I haven't completely given up on my M1 tho, so I'll be glad to share once everything is fixed.

[wamm0909]

After many testes I'm still stuck here on my M1 Macbook. I'm now on Python3.8 which shouldn't be a problem, Nevergrad seems installed correctly but yet Robyn won't run.

This is what happens when I try to py_config

Besides the fact that I don't get why py_config itself is not working, I noticed that on @laresbernardo's output there's "nevergrad" right after "numpy" which is missing on mine.
P.S.In the meantime I borrowed an older 2016 Macbook Pro (with Intel CPU ofc) on which I managed to make Robyn run smoothly with one little trick: I was still having troubles with the missing "shared Python library" error. Simply installing Xcode (without even running it once) fixed it and now it works fine.

my issue display like you:"there's "nevergrad" right after "numpy" which is missing on mine. "do you solve this?ths!

Nope, I've done some other tries over time with no luck (tried uninstalling and re-installing everything aswell) but I now had the chance to switch to an older intel-based mackbook which worked just fine right from the beginning using conda.

I haven't completely given up on my M1 tho, so I'll be glad to share once everything is fixed.

thanks buddy，Looking forward to your good news

[fee1985]

I tried to install nevergrad with Option 1: nevergrad installation via PIP of the demo.R

reticulate is installed and loaded

virtualenv_create("r-reticulate") worked an I have this folder:
C:\Users\name\Documents.virtualenvs\r-reticulate

I also have the pip.exe here
C:\Users\name\Documents.virtualenvs\r-reticulate\Scripts

But when I try py_install("nevergrad", pip = TRUE) I get the following error
C:\Users\SD89F1.LES\DOCUME1\VIRTUA1\R-RETI1\python.exe: No module named pip
Error: Error installing package(s): "nevergrad"

py_config() brings this output
python: C:/Users/s.leschke/Documents/.virtualenvs/r-reticulate/Scripts/python.exe
libpython: C:/Users/s.leschke/Anaconda3/python39.dll
pythonhome: C:/Users/s.leschke/Documents/.virtualenvs/r-reticulate
version: 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)]
Architecture: 64bit
numpy: [NOT FOUND]

NOTE: Python version was forced by use_python function

Installing via Anaconda Prompt with pip install works well.

Would be grateful if someone has an idea how I can install nevergrad.

[Robin-Lord]

Hiya, first off - great project, I'm really impressed.

Second - I'm one of the people who hit a bit of a wall with this and really struggled to get further. I was working on Macbook Pro did find that downgrading to a laptop with a non-Apple chipset meant it breezed through the start phase (but then struggled a bit with the model build, understandably).

Most effective solution I found was creating a cloud-hosted notebook with Google's Vertex AI. Unfortunately Colab (even Colab Pro+) crashed when I got to the model building stage, but the Vertex Jupyter notebook seems to have coped with it all really well. Tried Azure's equivalent but that struggled a bit.

Not to advertise the service but it's roughly 35p per hour for the core-number needed, so could be worth people giving it a try if you're stuck.

[wamm0909]

After many testes I'm still stuck here on my M1 Macbook. I'm now on Python3.8 which shouldn't be a problem, Nevergrad seems installed correctly but yet Robyn won't run.

This is what happens when I try to py_config

Besides the fact that I don't get why py_config itself is not working, I noticed that on @laresbernardo's output there's "nevergrad" right after "numpy" which is missing on mine.
P.S.In the meantime I borrowed an older 2016 Macbook Pro (with Intel CPU ofc) on which I managed to make Robyn run smoothly with one little trick: I was still having troubles with the missing "shared Python library" error. Simply installing Xcode (without even running it once) fixed it and now it works fine.

my issue display like you:"there's "nevergrad" right after "numpy" which is missing on mine. "do you solve this?ths!

Nope, I've done some other tries over time with no luck (tried uninstalling and re-installing everything aswell) but I now had the chance to switch to an older intel-based mackbook which worked just fine right from the beginning using conda.

I haven't completely given up on my M1 tho, so I'll be glad to share once everything is fixed.

hi,buddy, I have successfully installed it on the m1 chip, macos big sur 11.6.1; the r version of rstudio is 4.1.2, the python is 3.8 or 3.9, and the method of “use conda” is used【at the top of this page】, but the last step command cannot use pip, it needs to be changed become：conda_install("r-reticulate", "nevergrad")；hope you can get it！and you can use

After many testes I'm still stuck here on my M1 Macbook. I'm now on Python3.8 which shouldn't be a problem, Nevergrad seems installed correctly but yet Robyn won't run.

This is what happens when I try to py_config

Besides the fact that I don't get why py_config itself is not working, I noticed that on @laresbernardo's output there's "nevergrad" right after "numpy" which is missing on mine.
P.S.In the meantime I borrowed an older 2016 Macbook Pro (with Intel CPU ofc) on which I managed to make Robyn run smoothly with one little trick: I was still having troubles with the missing "shared Python library" error. Simply installing Xcode (without even running it once) fixed it and now it works fine.

my issue display like you:"there's "nevergrad" right after "numpy" which is missing on mine. "do you solve this?ths!

Nope, I've done some other tries over time with no luck (tried uninstalling and re-installing everything aswell) but I now had the chance to switch to an older intel-based mackbook which worked just fine right from the beginning using conda.

I haven't completely given up on my M1 tho, so I'll be glad to share once everything is fixed.

hi,buddy, I have successfully installed it on the m1 chip, macos big sur 11.6.1; the r version of rstudio is 4.1.2, the python is 3.8 or 3.9, and the method of “use conda” is used【at the top of this page】, but the last step command cannot use pip, it needs to be changed become：conda_install("r-reticulate", "nevergrad")；hope you can get it！and you can use command "reticulate::py_module_available("nevergrad")" to find whether the installation was successful, this is more efficient than py_config; remember "library (reticulate)"

[Tsurty]

Hey there, thanks for the suggestion.

I tried it but surprisingly enough I get stuck here:
Collecting package metadata (current_repodata.json): ...working... done Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve. Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): ...working... done Solving environment: ...working...

It just stands there and doesn't progress.
I've done a bunch of other tests with no luck.

The thing is if I open a terminal via Anaconda and I do "pip install nevergrad" in there, it actually works and tells me that Nevergrad is installed.
Then, I can use either use_virtualenv("r-reticulate", required = TRUE) or use_condaenv("r-reticulate") but non of those seem to pickup Nevergrad.

At this point I believe there's just something screwed up somewhere between all the tests I've done.

[wamm0909]

你好，谢谢你的建议。

我试过了，但令人惊讶的是我被困在这里： Collecting package metadata (current_repodata.json): ...working... done Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve. Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): ...working... done Solving environment: ...working...

它只是站在那里，没有进展。 我做了很多其他的测试都没有运气。

问题是，如果我通过 Anaconda 打开一个终端并在其中执行“pip install nevergrad”，它实际上可以工作并告诉我已安装 Nevergrad。 然后，我可以使用其中任何一种use_virtualenv("r-reticulate", required = TRUE)，use_condaenv("r-reticulate")但似乎都不能使用 Nevergrad。

在这一点上，我相信在我所做的所有测试之间存在一些问题。

yes, if you use "pip install nevergrad",it will tell you nevergrad has been install，but it‘s installed in python,You must to make the R version satisfy M1, and the python version 【3.8/3.9】satisfy r, when you use "version" command in rstudio, you will find R version 4.1.2 and arch is aarch 64, may be you can also use "conda env list" in terminal at rstudio to find if you have use "r-reticulate" env【find environment location】; the env of mine is following pictures: and If you still have problems，Maybe the network is not good and your download fails。

[Tsurty]

Hello guys!

It's with great pleasure that I'd like to share with you that I managed to run Robyn on M1 Mackbook for the very first time today.

I did two things specifically and I'm not sure which one made the trick but I believe it's this one.
First of all restart R session and clean variable to have a fresh start.

Then do:
reticulate::install_miniconda()

Once it's installed, make sure you have the latest Robyn's version and when prompted update all depencies:
remotes::install_github("facebookexperimental/Robyn/R")

Restart R session again and proceed to import and create a new env and install Nevergrad:

library(reticulate)
virtualenv_create("r-reticulate-cassandra")
use_virtualenv("r-reticulate-cassandra", required = TRUE)
py_install("nevergrad", pip = TRUE)

At this point it went smoothly and surprisingly I don't see Nevergrad in my py_config (see below)

Then I executed the latest demo.R file and it just worked smoothly running everything in < 5mins

In case this won't work for you, the only other thing that I did in the meantime was a bunch of installations which were not related to Robyn but may have had a positive impact on it maybe so I'll just leave them down here:

##1 install devtools and ausgynth and GeoLift
install.packages("devtools", repos='http://cran.us.r-project.org')
devtools::install_github("ebenmichael/augsynth")
devtools::install_github("facebookincubator/GeoLift")

#1.2. install other packages
install.packages(c("tidyverse", "ggplot2", "plotly", "readr","lubridate", "googlesheets4", "knitr", "gsynth","dplyr","doParallel", "MarketMatching"))

[monathiele]

Hi I had the "CondaSSLError: OpenSSL appears to be unavailable on this machine" error all the time but this works real good for me:

library(reticulate)
# create a new environment 
reticulate:::rm_all_reticulate_state() #fixed SSL issue 
conda_create("r-reticulate")
use_condaenv("r-reticulate")
conda_install("r-reticulate", "nevergrad", pip = TRUE)
py_config() 

[MarouschaD]

I'm also running into issues here :/ I've been trying to follow the demo guide and following the "#### Known potential issues when installing nevergrad and possible fixes" steps, but no luck.

I'm following #### Option 2: nevergrad installation via conda.

Running the > conda_install("r-reticulate", "nevergrad", pip=TRUE) command:

• it doesn't seem to add nevergrad (I was expecting it under add / updated specs or packages).
  

When it finished running it just closes with "nevergrad", making it seem like it works, but then when I'm calling py_config() I just see numpy, and not nevergrad:

Then on a last resort I tried:

reticulate:::rm_all_reticulate_state()
system("pip3 install --upgrade pip")

And I ran:

conda_install("r-reticulate", "nevergrad")
But it gave the error:
Error: Unable to find conda binary. Is Anaconda installed?

Anyone knows what I can do to make this work? I'm running it on a windows 11 Pro.

Thanks in advance!