py_install() Error: lexical error: invalid char in json text. #474

moldach · 2019-04-06T17:08:13Z

I downloaded Anaconda 3.7 for Windows (64-bit Graphical Installer), following these instructions, and then downloaded Python 3.7.3.

After installing reticulate I'm getting an error from py_install():

install.packages("reticulate")
library(reticulate)
Sys.which("python")
                                    python 
"C:\\Users\\Matthew\\ANACON~1\\python.exe" 

py_install("numpy")

Error: lexical error: invalid char in json text.
                                       WARNING: The conda.compat modul
                     (right here) ------^

It looks like this is similar to issue #360 at least in terms of warning at the top. And also the closed issue #100. My conda info --json returns:

WARNING: The conda.compat module is deprecated and will be removed in a future release.
{
  "active_prefix": null,
  "active_prefix_name": null,
  "channels": [
    "https://repo.anaconda.com/pkgs/main/win-64",
    "https://repo.anaconda.com/pkgs/main/noarch",
    "https://repo.anaconda.com/pkgs/free/win-64",
    "https://repo.anaconda.com/pkgs/free/noarch",
    "https://repo.anaconda.com/pkgs/r/win-64",
    "https://repo.anaconda.com/pkgs/r/noarch",
    "https://repo.anaconda.com/pkgs/msys2/win-64",
    "https://repo.anaconda.com/pkgs/msys2/noarch"
  ],
  "conda_build_version": "3.17.8",
  "conda_env_version": "4.6.11",
  "conda_location": "C:\\Users\\Matthew\\Anaconda3\\lib\\site-packages\\conda",
  "conda_prefix": "C:\\Users\\Matthew\\Anaconda3",
  "conda_private": false,
  "conda_shlvl": -1,
  "conda_version": "4.6.11",
  "config_files": [],
  "default_prefix": "C:\\Users\\Matthew\\Anaconda3",
  "env_vars": {
    "CIO_TEST": "<not set>",
    "CONDA_ROOT": "C:\\Users\\Matthew\\Anaconda3",
    "DOCKER_CERT_PATH": "C:\\Users\\Matthew\\.docker\\machine\\machines\\default",
    "DOCKER_TOOLBOX_INSTALL_PATH": "C:\\Program Files\\DockerToolbox",
    "HOMEPATH": "\\Users\\Matthew",
    "NO_PROXY": "<set>",
    "PATH": "C:\\Program Files\\R\\R-3.5.3\\bin\\x64;C:\\Rtools\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOW
S\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program F
iles\\Git\\cmd;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files\\PuTTY\\;C:\\Users\\Matthew\\Anaconda3;C:\\User
s\\Matthew\\Anaconda3\\Library\\mingw-w64\\bin;C:\\Users\\Matthew\\Anaconda3\\Library\\usr\\bin;C:\\Users\\Matthe
w\\Anaconda3\\Library\\bin;C:\\Users\\Matthew\\Anaconda3\\Scripts;C:\\Users\\Matthew\\AppData\\Local\\Microsoft\\
WindowsApps;C:\\Program Files\\DockerToolbox;C:\\Program Files\\qpdf-8.4.0\\bin;C:\\Users\\Matthew\\AppData\\Loca
l\\Pandoc\\;C:\\Users\\Matthew\\Documents\\MiKTeX\\miktex\\bin\\x64\\;C:\\Users\\Matthew\\AppData\\Local\\Program
s\\MiKTeX 2.9\\miktex\\bin\\x64\\;C:\\Users\\Matthew\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:/Program
 Files/Git/bin;C:/Program Files/RStudio/bin/msys-ssh-1000-18",
    "PSMODULEPATH": "C:\\Program Files\\WindowsPowerShell\\Modules;C:\\WINDOWS\\system32\\WindowsPowerShell\\v1.0
\\Modules",
    "REQUESTS_CA_BUNDLE": "<not set>",
    "RMARKDOWN_MATHJAX_PATH": "C:/Program Files/RStudio/resources/mathjax-26",
    "RS_RPOSTBACK_PATH": "C:/Program Files/RStudio/bin/rpostback",
    "SSL_CERT_FILE": "<not set>",
    "VBOX_MSI_INSTALL_PATH": "C:\\Program Files\\Oracle\\VirtualBox\\"
  },
  "envs": [
    "C:\\Users\\Matthew\\Anaconda3"
  ],
  "envs_dirs": [
    "C:\\Users\\Matthew\\Anaconda3\\envs",
    "C:\\Users\\Matthew\\.conda\\envs",
    "C:\\Users\\Matthew\\AppData\\Local\\conda\\conda\\envs"
  ],
  "is_windows_admin": false,
  "netrc_file": null,
  "offline": false,
  "pkgs_dirs": [
    "C:\\Users\\Matthew\\Anaconda3\\pkgs",
    "C:\\Users\\Matthew\\.conda\\pkgs",
    "C:\\Users\\Matthew\\AppData\\Local\\conda\\conda\\pkgs"
  ],
  "platform": "win-64",
  "python_version": "3.7.3.final.0",
  "rc_path": "C:\\Users\\Matthew\\.condarc",
  "requests_version": "2.21.0",
  "root_prefix": "C:\\Users\\Matthew\\Anaconda3",
  "root_writable": true,
  "site_dirs": [],
  "sys.executable": "C:\\Users\\Matthew\\Anaconda3\\python.exe",
  "sys.prefix": "C:\\Users\\Matthew\\Anaconda3",
  "sys.version": "3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)]",
  "sys_rc_path": "C:\\Users\\Matthew\\Anaconda3\\.condarc",
  "user_agent": "conda/4.6.11 requests/2.21.0 CPython/3.7.3 Windows/10 Windows/10.0.17134",
  "user_rc_path": "C:\\Users\\Matthew\\.condarc"
}

py_discover_config() yields the same lexical error.

This is my sessionInfo():

R version 3.5.3 (2019-03-11)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 17134)

Matrix products: default

locale:
[1] LC_COLLATE=English_Canada.1252  LC_CTYPE=English_Canada.1252    LC_MONETARY=English_Canada.1252
[4] LC_NUMERIC=C                    LC_TIME=English_Canada.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] reticulate_1.11.1  usethis_1.4.0.9000 devtools_2.0.1    

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.1         rstudioapi_0.10    magrittr_1.5       pkgload_1.0.2      lattice_0.20-38   
 [6] R6_2.4.0           rlang_0.3.3        tools_3.5.3        pkgbuild_1.0.3     grid_3.5.3        
[11] sessioninfo_1.1.1  cli_1.1.0          withr_2.1.2        remotes_2.0.2      assertthat_0.2.1  
[16] rprojroot_1.3-2    digest_0.6.18      crayon_1.3.4       processx_3.3.0     Matrix_1.2-15     
[21] callr_3.2.0        fs_1.2.7           ps_1.3.0           testthat_2.0.1     memoise_1.1.0.9000
[26] glue_1.3.1         compiler_3.5.3     desc_1.2.0         backports_1.1.3    prettyunits_1.0.2 
[31] jsonlite_1.6

The text was updated successfully, but these errors were encountered:

martj42 · 2019-04-07T20:55:25Z

Have the same issue and from what I can tell it's caused by the warning in the first line

WARNING: The conda.compat module is deprecated and will be removed in a future release.

There is a check for something similar in conda_list() but this one gets through.

reticulate/R/conda.R

Line 50 in bfac80b

    
           if (length(conda_envs) > 0 && grepl("Anaconda Cloud", conda_envs[[1]], fixed = TRUE))

Basically what we'd want is something like this to be added to the code:

  if (length(conda_envs) > 0 && grepl("WARNING", conda_envs[[1]], fixed = TRUE))
    conda_envs <- conda_envs[-1]

MislavSag · 2019-04-08T13:34:04Z

I got the same error after trying to implement LSTM using keras. Everything worked 2 days ago.

jjallaire · 2019-04-08T16:28:43Z

I think this correctly diagnoses both the issue and the fix (extraneous warnings being included in output we are attempting to parse as JSON).

While we could look for and discard the warning, I wonder if there is something we could do here proactively:

reticulate/R/conda.R

Lines 29 to 31 in bfac80b

    
           conda_envs <- suppressWarnings( 
        
             system2(conda, args = c("info", "--json"), stdout = TRUE) 
        
           )

i.e. pass stderr = FALSE to prevent writing to stderr in the first place, or perhaps include a conda command line flag that would surpress this output.

@kevinushey Could you take a look at this ASAP? (I think it's breaking reticulate installs for anyone using Anaconda >= 3.7).

kevinushey · 2019-04-08T20:20:46Z

Will take a look!

kevinushey · 2019-04-08T20:56:56Z

@jjallaire should we prepare a CRAN release of reticulate for this?

gwd999 · 2019-04-08T22:21:34Z

Looking fwd to updates here since I also get the same errors and warnings (imho based upon "WARNING: The conda.compat modul ..." ) as described above. Since up to a few weeks ago ie before I updated Anaconda (to currently 3.7.1) everything ('everything' being the use of the reticulate and keras packages based on conda environments) worked fairly smooth it does seem to be an issue that surfaced vie update along the 'conda_envs' lilnes.

jjallaire · 2019-04-08T23:52:48Z

@kevinushey Yes I think we should prepare a CRAN release for this. Maybe before we update we should confirm that the stderr fix resolves the problem on all platforms?

smouksassi · 2019-04-11T08:22:24Z

I installed from github but I keep getting the same error.
do we need to install a special commit ?

use_python('C:\Users\smouksas\AppData\Local\Continuum\anaconda3\python.exe')
sns <- import('seaborn')
Error: lexical error: invalid char in json text.
WARNING: The conda.compat modul
(right here) ------^

sessionInfo()
R version 3.5.3 (2019-03-11)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] reticulate_1.11.1

kevinushey · 2019-04-11T16:10:23Z

@smouksassi I suspect you need to restart after installing from GitHub. (note that the old CRAN release of reticulate is still loaded in your session)

smouksassi · 2019-04-12T11:48:48Z

yes after restart all went well and could produce my first python outputs thank you !
( I had to deal with and fix another issue regarding qt not found using this code)
import os
os.environ['QT_QPA_PLATFORM_PLUGIN_PATH'] = 'C:/Users/smouksas/AppData/Local/Continuum/anaconda3/Library/plugins/platforms'

kevinushey · 2019-04-12T17:27:20Z

We've just pushed an updated version of reticulate to CRAN (binaries coming soon to macOS / Windows) so this should be fixed across the board then.

NigelKinney · 2019-04-15T05:02:33Z

I got this error when I re-install reticulate from github..
Error in i.p(...) :
(converted from warning) installation of package ‘C:/Users/Nigel/AppData/Local/Temp/RtmpeqiFuU/file2b4c513c44d8/reticulate_1.12.0-9000.tar.gz’ had non-zero exit status

devtools::install_github("rstudio/reticulate")
Downloading GitHub repo rstudio/reticulate@master
√ checking for file 'C:\Users\Nigel\AppData\Local\Temp\RtmpeqiFuU\remotes2b4c50003428\rstudio-reticulate-f3179ab/DESCRIPTION' ...

preparing 'reticulate': (810ms)
√ checking DESCRIPTION meta-information ...
cleaning src
checking for LF line-endings in source and make files and shell scripts (431ms)
checking for empty or unneeded directories
building 'reticulate_1.12.0-9000.tar.gz'

Installing package into ‘C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5’
(as ‘lib’ is unspecified)

installing source package 'reticulate' ...
** libs

*** arch - i386
c:/Rtools/mingw_32/bin/g++ -I"C:/PROGRA~~1/R/R-35~~1.3/include" -DNDEBUG -I"C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/Rcpp/include" -O2 -Wall -mtune=generic -c RcppExports.cpp -o RcppExports.o
c:/Rtools/mingw_32/bin/g++ -I"C:/PROGRA~~1/R/R-35~~1.3/include" -DNDEBUG -I"C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/Rcpp/include" -O2 -Wall -mtune=generic -c event_loop.cpp -o event_loop.o
c:/Rtools/mingw_32/bin/g++ -I"C:/PROGRA~~1/R/R-35~~1.3/include" -DNDEBUG -I"C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/Rcpp/include" -O2 -Wall -mtune=generic -c libpython.cpp -o libpython.o
c:/Rtools/mingw_32/bin/g++ -I"C:/PROGRA~~1/R/R-35~~1.3/include" -DNDEBUG -I"C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/Rcpp/include" -O2 -Wall -mtune=generic -c output.cpp -o output.o
c:/Rtools/mingw_32/bin/g++ -I"C:/PROGRA~~1/R/R-35~~1.3/include" -DNDEBUG -I"C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/Rcpp/include" -O2 -Wall -mtune=generic -c python.cpp -o python.o
c:/Rtools/mingw_32/bin/g++ -I"C:/PROGRA~~1/R/R-35~~1.3/include" -DNDEBUG -I"C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/Rcpp/include" -O2 -Wall -mtune=generic -c readline.cpp -o readline.o
c:/Rtools/mingw_32/bin/g++ -shared -s -static-libgcc -o reticulate.dll tmp.def RcppExports.o event_loop.o libpython.o output.o python.o readline.o -LC:/PROGRA~~1/R/R-35~~1.3/bin/i386 -lR
installing to C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/reticulate/libs/i386

*** arch - x64
c:/Rtools/mingw_64/bin/g++ -I"C:/PROGRA~~1/R/R-35~~1.3/include" -DNDEBUG -I"C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/Rcpp/include" -O2 -Wall -mtune=generic -c RcppExports.cpp -o RcppExports.o
c:/Rtools/mingw_64/bin/g++ -I"C:/PROGRA~~1/R/R-35~~1.3/include" -DNDEBUG -I"C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/Rcpp/include" -O2 -Wall -mtune=generic -c event_loop.cpp -o event_loop.o
c:/Rtools/mingw_64/bin/g++ -I"C:/PROGRA~~1/R/R-35~~1.3/include" -DNDEBUG -I"C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/Rcpp/include" -O2 -Wall -mtune=generic -c libpython.cpp -o libpython.o
c:/Rtools/mingw_64/bin/g++ -I"C:/PROGRA~~1/R/R-35~~1.3/include" -DNDEBUG -I"C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/Rcpp/include" -O2 -Wall -mtune=generic -c output.cpp -o output.o
c:/Rtools/mingw_64/bin/g++ -I"C:/PROGRA~~1/R/R-35~~1.3/include" -DNDEBUG -I"C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/Rcpp/include" -O2 -Wall -mtune=generic -c python.cpp -o python.o
c:/Rtools/mingw_64/bin/g++ -I"C:/PROGRA~~1/R/R-35~~1.3/include" -DNDEBUG -I"C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/Rcpp/include" -O2 -Wall -mtune=generic -c readline.cpp -o readline.o
c:/Rtools/mingw_64/bin/g++ -shared -s -static-libgcc -o reticulate.dll tmp.def RcppExports.o event_loop.o libpython.o output.o python.o readline.o -LC:/PROGRA~~1/R/R-35~~1.3/bin/x64 -lR
installing to C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/reticulate/libs/x64
Error in file.copy(files, dest, overwrite = TRUE) :
(converted from warning) problem copying .\reticulate.dll to C:\Users\Nigel\OneDrive\Documents\R\win-library\3.5\reticulate\libs\x64\reticulate.dll: Permission denied

removing 'C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/reticulate'
restoring previous 'C:/Users/Nigel/OneDrive/Documents/R/win-library/3.5/reticulate'
Error in file.copy(lp, dirname(pkgdir), recursive = TRUE, copy.date = TRUE) :
(converted from warning) problem copying C:\Users\Nigel\OneDrive\Documents\R\win-library\3.5\00LOCK-reticulate\reticulate\libs\x64\reticulate.dll to C:\Users\Nigel\OneDrive\Documents\R\win-library\3.5\reticulate\libs\x64\reticulate.dll: Permission denied
Execution halted
In R CMD INSTALL

kevinushey · 2019-04-16T19:55:11Z

Error in file.copy(files, dest, overwrite = TRUE) :
(converted from warning) problem copying .\reticulate.dll to C:\Users\Nigel\OneDrive\Documents\R\win-library\3.5\reticulate\libs\x64\reticulate.dll: Permission denied

You need to make sure you're installing reticulate from a fresh R session (without reticulate already loaded). Also make sure you don't have any other R sessions open with reticulate loaded. (Simplest way forward is to just restart the computer and then install reticulate in a clean R session)

dfalbel mentioned this issue Apr 7, 2019

lexical error during Keras installation rstudio/keras3#731

Closed

dfalbel mentioned this issue Apr 8, 2019

Keras failure in R after using in python with Anaconda rstudio/keras3#733

Closed

kevinushey closed this as completed in 01e22ce Apr 8, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

py_install() Error: lexical error: invalid char in json text. #474

py_install() Error: lexical error: invalid char in json text. #474

moldach commented Apr 6, 2019 •

edited

Loading

martj42 commented Apr 7, 2019

MislavSag commented Apr 8, 2019

jjallaire commented Apr 8, 2019

kevinushey commented Apr 8, 2019

kevinushey commented Apr 8, 2019

gwd999 commented Apr 8, 2019 •

edited

Loading

jjallaire commented Apr 8, 2019

smouksassi commented Apr 11, 2019

kevinushey commented Apr 11, 2019

smouksassi commented Apr 12, 2019

kevinushey commented Apr 12, 2019

NigelKinney commented Apr 15, 2019

kevinushey commented Apr 16, 2019

py_install() Error: lexical error: invalid char in json text. #474

py_install() Error: lexical error: invalid char in json text. #474

Comments

moldach commented Apr 6, 2019 • edited Loading

martj42 commented Apr 7, 2019

MislavSag commented Apr 8, 2019

jjallaire commented Apr 8, 2019

kevinushey commented Apr 8, 2019

kevinushey commented Apr 8, 2019

gwd999 commented Apr 8, 2019 • edited Loading

jjallaire commented Apr 8, 2019

smouksassi commented Apr 11, 2019

kevinushey commented Apr 11, 2019

smouksassi commented Apr 12, 2019

kevinushey commented Apr 12, 2019

NigelKinney commented Apr 15, 2019

kevinushey commented Apr 16, 2019

moldach commented Apr 6, 2019 •

edited

Loading

gwd999 commented Apr 8, 2019 •

edited

Loading