Exception: Unknown Error code xxxx xxx using reticulate with python and nrel-pysam

[minimenchmuncher]

I'm attempting to use reticulate within R to interface with NREL's SAM library via PySAM (nrel-pysam on pypi); however, I cannot interface with any of the compute modules, resulting in an error that says simply:

Exception: Unknown code s5wu 64

Though the exact code changes each time I run it, making me think it has something to do with RAM addressing.

Here's an example of what I've tried, using AlmaLinux 9.1 (a clean, fresh install), which comes with python version 3.9:

## install R
sudo dnf install python3-pip epel-release
sudo dnf config-manager --set-enabled crb
sudo dnf install R

## set up python dependencies
python -m venv env
source env/bin/activate
pip install nrel-pysam

## Python example, for reference
import PySAM.Pvwattsv8 as pv
pv.new()
## executes with no problem

## within R
install.packages("reticulate")
library(reticulate)
use_virtualenv("~/env")
pv <- import("PySAM.Pvwattsv8")
pvwatts_model <- pv$new()

Error in py_call_impl(callable, call_args$unnamed, call_args$named) : 
  Exception: Unknown code Rb8s 32

a full traceback yields something that looks like

3: stop(structure(list(message = "Exception: Unknown code Rb8s 32\n", 
       call = NULL, cppstack = structure(list(file = "", line = -1L, 
           stack = c("/home/vagrant/R/x86_64-redhat-linux-gnu-library/4.3/reticulate/libs/reticulate.so(+0x1d688) [0x7f6027827688]", 
           "/home/vagrant/R/x86_64-redhat-linux-gnu-library/4.3/reticulate/libs/reticulate.so(+0x11685) [0x7f602781b685]", 
           "/home/vagrant/R/x86_64-redhat-linux-gnu-library/4.3/reticulate/libs/reticulate.so(+0x1be24) [0x7f6027825e24]", 
           "/home/vagrant/R/x86_64-redhat-linux-gnu-library/4.3/reticulate/libs/reticulate.so(_reticulate_py_call_impl+0xda) [0x7f602782f94a]", 
           "/usr/lib64/R/lib/libR.so(+0xf7cde) [0x7f603f1dbcde]", 
           "/usr/lib64/R/lib/libR.so(+0x13c49e) [0x7f603f22049e]", 
           "/usr/lib64/R/lib/libR.so(Rf_eval+0x178) [0x7f603f207838]", 
           "/usr/lib64/R/lib/libR.so(+0x12267e) [0x7f603f20667e]", 
           "/usr/lib64/R/lib/libR.so(Rf_applyClosure+0x268) [0x7f603f207588]", 
           "/usr/lib64/R/lib/libR.so(+0x13fc7c) [0x7f603f223c7c]", 
           "/usr/lib64/R/lib/libR.so(Rf_eval+0x178) [0x7f603f207838]", 
           "/usr/lib64/R/lib/libR.so(+0x12267e) [0x7f603f20667e]", 
           "/usr/lib64/R/lib/libR.so(Rf_applyClosure+0x268) [0x7f603f207588]", 
           "/usr/lib64/R/lib/libR.so(Rf_eval+0x2a5) [0x7f603f207965]", 
           "/usr/lib64/R/lib/libR.so(Rf_ReplIteration+0x322) [0x7f603f2646f2]", 
           "/usr/lib64/R/lib/libR.so(+0x180ab0) [0x7f603f264ab0]", 
           "/usr/lib64/R/lib/libR.so(run_Rmainloop+0x49) [0x7f603f264b69]", 
           "/usr/lib64/R/bin/exec/R(main+0x1f) [0x561e47aa10af]", 
           "/lib64/libc.so.6(+0x3feb0) [0x7f603ef12eb0]", "/lib64/libc.so.6(__libc_start_main+0x80) [0x7f603ef12f60]", 
           "/usr/lib64/R/bin/exec/R(_start+0x25) [0x561e47aa10e5]"
           )), class = "Rcpp_stack_trace")), class = c("Rcpp::exception", 
   "C++Error", "error", "condition")))
2: py_call_impl(callable, dots$args, dots$keywords)
1: pv$new()

I have attempted to do more or less the following on a variety of OSes (Fedora 38, Ubuntu 20.04, AlmaLinux 9.1 to name a fiew) with a variety of python versions (3.8 - 3.11), I've tried both the CRAN version of reticulate and the current (as of 2023-05-08) main branch version on Github, I've installed nrel-pysam with pip and with conda (not having both at the same time of course), have only ever seen this error message. Here's a sample session info:

> sessionInfo()
R version 4.3.0 (2023-04-21)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: AlmaLinux 9.1 (Lime Lynx)

Matrix products: default
BLAS/LAPACK: FlexiBLAS OPENBLAS-OPENMP;  LAPACK version 3.9.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: UTC
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] reticulate_1.28-9000

loaded via a namespace (and not attached):
[1] compiler_4.3.0 Matrix_1.5-4   cli_3.6.1      Rcpp_1.0.10    grid_4.3.0    
[6] jsonlite_1.8.4 rlang_1.1.1    png_0.1-8      lattice_0.21-8

Happy to include any/all other information as necessary.

[t-kalinowski]

Hi, thank you for reporting. I can reproduce locally.

This issue will require some investigation. In a debugger I was able to trace the error to this function. Specifically from the PySAM_TECH_ATTR() macro therein. I searched across the Python, reticulate, and PySAM sources for error messages that match the one we're seeing, but didn't find anything. Most likely the error is coming from what PySAM is wrapping. (I lost the trail looking to see where the function new_error() is defined).

The PySAM.Pvwattsv8 is not a conventional python module, it is constructed entirely in C. I'm not sure what the ultimate cause of the issue is right now. Two guesses:

1. Defining a conventional module in Python with Python code does some additional work that isn't done by the the C code initializing that module, and it's causing reticulate to raise an error
2. Something that reticulate is doing is causing PySAM C code to not load the correct Python API symbols.

This looks like a tricky issue requiring investigation. I'm happy to patch what's needed in reticulate to make this work, but any help with investigation and diagnosing the root cause would be appreciated.

[minimenchmuncher]

Update here, this appears NOT to be an issue on windows.

I conducted a similar test using python 3.11.3 and R 4.3.0 on a windows 10 vm:

> use_virtualenv("~/sam_test/env")
> pvwattsv8 <- import("PySAM.Pvwattsv8")
> model <- pvwattsv8$new()
## if there were a problem it would yield an error here, but it didn't.
> sessionInfo()
R version 4.3.0 (2023-04-21 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default


locale:
[1] LC_COLLATE=English_United States.utf8
[2] LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8

time zone: America/Los_Angeles
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] reticulate_1.28

loaded via a namespace (and not attached):
[1] compiler_4.3.0  here_1.0.1      Matrix_1.5-4    rprojroot_2.0.3
[5] Rcpp_1.0.10     grid_4.3.0      jsonlite_1.8.4  png_0.1-8
[9] lattice_0.21-8

[minimenchmuncher]

Cross-listed this issue on NREL/pysam:
NREL/pysam#150