-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Version 2.0.0 fails to install on R 4.0.2 #104
Comments
Hi. Thanks for reporting this. It is the same error reported in #100. You are using a version of |
thanks for the speedy reply! I understand the fix, and thanks for that, but just as a remark this is a rather problematic upgrade for us as it's a system level dependency and we are using the latest version of R Studio Workbench Enterprise on AWS SageMaker - literally just released a few weeks ago - and R Studio has chosen Ubuntu 18.04 LTS as the base distro, which has cmake 3.10. It's faster and easier to install nlopt as a system library than try to upgrade cmake. But thanks for the fix, we will also look into how to do this. |
Well, the reason that we need |
You could also test for that and use |
I tried to remove |
@frasernutricia, CMake provides pre-built binaries for Linux, so installation is simple: cd /path/to/software
curl -L -O https://github.com/Kitware/CMake/releases/download/v3.19.3/cmake-3.19.3-Linux-x86_64.tar.gz
tar xf cmake-3.19.3-Linux-x86_64.tar.gz Then add its bin/ folder to the PATH=/path/to/software/cmake-3.19.3-Linux-x86_64/bin:$PATH
export PATH and you're good to go. If you can set that up on the server, then that might be better than having to maintain a system/site-wide nloptr install. |
(Or maybe use a more current distro. Tossing random newer binaries into an older LTS may have other side effects. Lots of competing tradeoffs here.) |
One thought I keep having: given that a) the set of files in nlopt is fixed and b) cmake does not seem to do that much 'discovery' could we possibly do away with cmake and just list an explicit list of dependencies of libnlopt.a in src/Makevars? The Matrix package had long done that (across different subdirectories / sublibraries), I more recently converted RDieHarder to build its lib that way. Or am I overlooking some "critical" parts for which we really need cmake? This could and should really build "widely" as we do not depend on anything crazy. Apart from, well, one component of the build chain. And maybe that is a price worth paying. Not my call here. |
We might do that in a future release yes. Looking at the |
Yes. Package Matrix even has a tool that creates the snippet then included into src/Makevars. When I set up a simpler one I started from a simple recursive find for files... |
Yes that is why I updated |
I just gave that a spin and it is far from a cake walk. |
You mean listing in |
Yes. Small points are e.g. that each dir in ## find src/{algs,api,util} -type f | grep -v "pdf\|README\|COPYRIGHT\|AUTHORS\|COPYING\|\.h\|\.txt\|\.3"
SOURCES = \
nlopt-src/src/algs/crs/crs.c \
nlopt-src/src/algs/newuoa/newuoa.c \
nlopt-src/src/algs/neldermead/nldrmd.c \
nlopt-src/src/algs/neldermead/sbplx.c \
nlopt-src/src/algs/cquad/cquad.c \
nlopt-src/src/algs/cdirect/hybrid.c \
nlopt-src/src/algs/cdirect/cdirect.c \
nlopt-src/src/algs/ags/tst.cc \
nlopt-src/src/algs/ags/ags.cc \
nlopt-src/src/algs/ags/local_optimizer.cc \
nlopt-src/src/algs/ags/solver.cc \
nlopt-src/src/algs/ags/evolvent.cc \
nlopt-src/src/algs/mma/mma.c \
nlopt-src/src/algs/mma/ccsa_quadratic.c \
nlopt-src/src/algs/subplex/subplex.c \
nlopt-src/src/algs/direct/DIRect.c \
nlopt-src/src/algs/direct/DIRserial.c \
nlopt-src/src/algs/direct/tstc.c \
nlopt-src/src/algs/direct/direct_wrap.c \
nlopt-src/src/algs/direct/DIRsubrout.c \
nlopt-src/src/algs/praxis/praxis.c \
nlopt-src/src/algs/auglag/auglag.c \
nlopt-src/src/algs/mlsl/mlsl.c \
nlopt-src/src/algs/luksan/plip.for \
nlopt-src/src/algs/luksan/pnet.c \
nlopt-src/src/algs/luksan/pnet.for \
nlopt-src/src/algs/luksan/plis.c \
nlopt-src/src/algs/luksan/plis.for \
nlopt-src/src/algs/luksan/mssubs.for \
nlopt-src/src/algs/luksan/pssubs.for \
nlopt-src/src/algs/luksan/mssubs.c \
nlopt-src/src/algs/luksan/pssubs.c \
nlopt-src/src/algs/luksan/plip.c \
nlopt-src/src/algs/cobyla/cobyla.c \
nlopt-src/src/algs/isres/isres.c \
nlopt-src/src/algs/slsqp/slsqp.c \
nlopt-src/src/algs/esch/esch.c \
nlopt-src/src/algs/bobyqa/bobyqa.c \
nlopt-src/src/algs/stogo/prog.cc \
nlopt-src/src/algs/stogo/global.cc \
nlopt-src/src/algs/stogo/tst.cc \
nlopt-src/src/algs/stogo/tstc.c \
nlopt-src/src/algs/stogo/testros.cc \
nlopt-src/src/algs/stogo/stogo.cc \
nlopt-src/src/algs/stogo/local.cc \
nlopt-src/src/algs/stogo/linalg.cc \
nlopt-src/src/algs/stogo/tools.cc \
nlopt-src/src/api/options.c \
nlopt-src/src/api/f77api.c \
nlopt-src/src/api/deprecated.c \
nlopt-src/src/api/general.c \
nlopt-src/src/api/optimize.c \
nlopt-src/src/util/rescale.c \
nlopt-src/src/util/mt19937ar.c \
nlopt-src/src/util/nlopt-getopt.c \
nlopt-src/src/util/qsort_r.c \
nlopt-src/src/util/stop.c \
nlopt-src/src/util/mt19937ar_test.c \
nlopt-src/src/util/sobolseq_test.c \
nlopt-src/src/util/sobolseq.c \
nlopt-src/src/util/redblack_test.c \
nlopt-src/src/util/timer.c \
nlopt-src/src/util/redblack.c
# nlopt-src/src/algs/direct/DIRparallel.c \
OBJECTS = $(SOURCES:.c=.o) $(SOURCES:.cc=.o) $(SOURCES:.for:.o)
$(SHLIB): $(OBJECTS) and I currently have PKG_CPPFLAGS = -I../inst/include -Inlopt-src/src -I. -Isrc -Inlopt-src/src/util/ -Inlopt-src/src/algs/praxis -Inlopt-src/src/algs/direct -Inlopt-src/src/algs/stogo
PKG_LIBS = $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) It also generates a bazillion and a half warnings. Also needed |
We could use the sources listed in the I am not quite sure why he is including the header files into the We could also start with the includes he lists here: And see if that is sufficient. |
Sorry I may have been unclear. Establishing the list of needed files is somewhat trivial and the first thing I did. But it's the old "necessary but not sufficient": as this is not enough to build A related problem is that if we "own" the full |
Oh I understand. It was worth trying that way in case |
Yep there is some extra 'sauce' in the CMakeLists.txt files we'd need to pick up. Might be enoght to let |
Hi @astamm - this is my ignorance here, but do you really need to move to cmake? We don't have a choice in the distro being used - Ubuntu 18.04.5 LTS - released Aug 2020 - as this was chosen by R Studio as part of their new SageMaker AWS offering. This distro does not seem so very old. We've installed a whole host of other cran packages with no issues, including rstan which has a parallel build process. nloptr seems an unfortunate outlier. But I fully realize I'm not actually offering help here! and we can workaround. |
@frasernutricia Ubuntu 18.04 LTS is already an outdated (even if still supported) LTS as 20.04 is current (and we are weeks away from 22.04). By being on an older version (not entirely by your choice, I understand) you must expect some "delta" to creep in. If I were you I'd clone the instance and see if an update to 20.04 (which I happen to run at AWS) is feasible. |
Thanks @eddelbuettel, we're using a new version of R Workbench which is built inside SageMaker (and the underlying EC2 instance is destroyed after each session, so any updates/system libraries need re-installed each time, but personal R libraries are persistent between sessions). A bring your own container model (docker etc) is on the R Workbench SageMaker roadmap for sometime in 2022 Q2 so fingers-crossed that will allow us something similar to a CRAN build. |
I happily used RStudio Cloud for a few years to teach (but campus appears to be moving away from it), generally liked it -- and that deployed a newer / more reasonable Ubuntu. Oh well. You can also 'persist' a local library and/or rely on binaries. As mentioned, 18.04 starts you at a disadvantage but the c2d4u.team repo has you covered on this one. A quick root@88688a322f3b:/work# apt-cache search r-cran-nloptr
r-cran-nloptr - GNU R package "R Interface to NLopt"
root@88688a322f3b:/work# apt-cache policy r-cran-nloptr
r-cran-nloptr:
Installed: (none)
Candidate: 2.0.0-1cran1.1804.0
Version table:
2.0.0-1cran1.1804.0 500
500 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 Packages
1.0.4-1build1 500
500 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages
root@88688a322f3b:/work# And better still thanks to the magic of bspm you even get installed via root@88688a322f3b:/work# install.r nloptr # install.r is a little wrapper around install.packages() from littler
(loaded the methods namespace)
Loading required package: utils
Tracing function "install.packages" in package "utils"
Install system packages as root...
Reading package lists... Done
Building dependency tree
Reading state information... Done
Hit http://archive.ubuntu.com/ubuntu bionic InRelease
Hit http://security.ubuntu.com/ubuntu bionic-security InRelease
Hit http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic InRelease
Hit http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Hit http://archive.ubuntu.com/ubuntu bionic-backports InRelease
Hit http://ppa.launchpad.net/edd/r-4.0/ubuntu bionic InRelease
Hit http://ppa.launchpad.net/marutter/rrutter4.0/ubuntu bionic InRelease
Fetched 0 B in 0s (0 B/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
Get:1 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 r-cran-brio amd64 1.1.3-1cran1.1804.0 [32.0 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libnlopt0 amd64 2.4.2+dfsg-4 [162 kB]
Get:3 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 r-cran-ps amd64 1.6.0-1cran1.1804.0 [270 kB]
Get:4 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 r-cran-r6 all 2.5.1-1cran1.1804.0 [86.1 kB]
Get:5 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 r-cran-processx amd64 3.5.2-1cran1.1804.0 [303 kB]
Get:6 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 r-cran-callr all 3.7.0-1cran1.1804.0 [421 kB]
Get:7 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 r-cran-glue amd64 1.6.1-1cran1.1804.0 [143 kB]
Get:8 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 r-cran-cli amd64 3.1.1-1cran1.1804.0 [1038 kB]
Get:9 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 r-cran-crayon all 1.4.2-1cran1.1804.0 [151 kB]
Get:10 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 r-cran-rprojroot all 2.0.2-1cran1.1804.0 [92.1 kB]
[.... lots of lines omitted here ...]
Setting up r-cran-vctrs (0.3.8-1cran1.1804.0) ...
Setting up r-cran-lifecycle (1.0.1-1cran1.1804.0) ...
Setting up r-cran-cli (3.1.1-1cran1.1804.0) ...
Setting up r-cran-callr (3.7.0-1cran1.1804.0) ...
Setting up r-cran-pkgload (1.2.4-1cran1.1804.0) ...
Setting up r-cran-pillar (1.6.5-1cran1.1804.0) ...
Setting up r-cran-tibble (3.1.6-1cran1.1804.0) ...
Setting up r-cran-rematch2 (2.1.2-1cran1.1804.0) ...
Setting up r-cran-waldo (0.3.1-1cran1.1804.0) ...
Setting up r-cran-testthat (3.1.2-1cran1.1804.0) ...
Setting up r-cran-nloptr (2.0.0-1cran1.1804.0) ...
Processing triggers for libc-bin (2.27-3ubuntu1.2) ...
root@88688a322f3b:/work#
root@88688a322f3b:/work# R -q
Loading required package: utils
Tracing function "install.packages" in package "utils"
> library(nloptr)
Warning message:
package ‘nloptr’ was built under R version 4.1.2
> Not bad for free, eh? More on |
Thanks to the mention, I noticed that I had this package failing to build in Fedora. Added the cmake dependency and all good. |
If I may add another use-case where updating cmake or the OS is not very evident: I am facing the same issue in trying to use nloptr as a dependency in a containerized lambda (public.ecr.aws/lambda/provided) which is based on CentOS 7 - CentOS 7 is supported until June 2024. I know this is probably comparable to the 18.04 vs 20.04 LTS debate above, but still it is a pain. I will see how I can change my dockerfile based upon the suggestions above while trying to retain a container of limited size and adequate robustness. |
Just in case it wasn't clear, So if you can get a recent enough NLopt onto such a CentOS 7 system you are fine. Of course, NLopt also uses |
Thank you @eddelbuettel , that did clarify things. The current EPEL version of NLopt and NLopt-devel is 2.4.2, but this did not seam recent enough to install |
Right -- it might be easier for us to relax the hard dependency on 2.7.0 or later (to permit your use with 2.4.2) and maybe condition out a method or two requiring 2.6.* or 2.7.* than to replace The fact of the matter remains that brandnew released in 2022 software may have diffculties on dated / very conservative host system. You may also want to consider an older |
For CentOS 7, I stumbled upon a workaround using cmake3 in CentOS, which is recent enough 3.17. In case of potential link rot I will just re-iterate the fix here, if someone else would run into the same issue with nloptr on public.ecr.aws/lambda/provided. Adding the below lines to the Dockerfile will allow you to build nloptr: RUN yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm \
&& yum -y install cmake3 NLopt NLopt-devel \
&& ln -s /usr/bin/cmake3 /usr/bin/cmake |
Nice. I'd recommend making the soft-link in |
@FMKerckhof thank you a lot for you suggestion ! -- did not work completely , but at least I managed to get to 96% 😅 ...where I get this error
any idea? this seems to be related with #105 |
I'd start there |
@ggrrll see above reply by @eddelbuettel - normally, if you have a recent enough cmake you don't need to have a system-level NLopt, as nloptr has an included copy of NLopt. As you can see from my code above I downloaded the latest epel release for CentOS 7 from https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm - but indeed this would result in NLopt 2.4.2. |
ok, so basically the error I am getting there (afterall, this does not seem too bad...)
|
🎊 also posted https://stackoverflow.com/a/71407734/5533078 |
Nice to read you managed to solve your issue. We have an ongoing PR #106 that should fix an issue with earlier versions of R. I will merge it soon. |
In case it's useful for others, the following snippet prior to
|
This is excellent, because RedHat/CentOS has I didn't know about the I think improvement that can be done this package is to look for
That should be backward compatible, and automagically solve it for, at least, everyone on RHEL/CentOS running into this problem. |
Thanks @HenrikBengtsson for the suggestion. I might implement it, it seems indeed a good idea. |
Hi,
as of 2.0.0 nloptr does not install on our R system. I've put the error below and sessionInfo(). I don't understand the error. Any advice much appreciated! thanks. Fraser
** package ‘nloptr’ successfully unpacked and MD5 sums checked
** using staged installation
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ -std=gnu++11 accepts -g... yes
checking how to run the C++ preprocessor... g++ -std=gnu++11 -E
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ -std=gnu++11 accepts -g... (cached) yes
checking for pkg-config... /usr/bin/pkg-config
checking if pkg-config knows NLopt... no
using NLopt via local cmake build on x86_64
set CMAKE_BIN=/usr/bin/cmake
set CC=gcc
set CFLAGS=-I/usr/local/include -fpic -g -O2
set CXX=g++
set CXXFLAGS=-std=gnu++11 -I/usr/local/include -fpic -g -O2
set LDFLAGS=-L/usr/local/lib
CMake Error: The source directory "/tmp/RtmpEKY5cd/R.INSTALL44a59ba24ad/nloptr/src/nlopt-build" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
Unknown argument -j
Unknown argument 2
Usage: cmake --build [options] [-- [native-options]]
Options: = Project binary directory to be built. --target = Build instead of default targets. May only be specified once. --config = For multi-configuration tools, choose . --clean-first = Build target 'clean' first, then build. (To clean only, use --target 'clean'.) --use-stderr = Ignored. Behavior is default in CMake >= 3.0. -- = Pass remaining options to the native tool.
CMake Error: The source directory "/tmp/RtmpEKY5cd/R.INSTALL44a59ba24ad/nloptr/src/nlopt" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
cp: cannot stat 'nlopt/include/*': No such file or directory
configure: creating ./config.status
config.status: creating src/Makevars
** libs
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods base
other attached packages:
[1] pcalg_2.7-4 graph_1.68.0 BiocGenerics_0.36.1
loaded via a namespace (and not attached):
[1] bdsmatrix_1.3-4 Rcpp_1.0.7 corpcor_1.6.10 RBGL_1.66.0 ggm_2.5 stats4_4.0.2
[7] magrittr_2.0.2 robustbase_0.93-9 fastICA_1.2-3 tools_4.0.2 igraph_1.2.11 DEoptimR_1.0-10
[13] sfsmisc_1.1-12 abind_1.4-5 compiler_4.0.2 clue_0.3-60 pkgconfig_2.0.3 cluster_2.1.0
The text was updated successfully, but these errors were encountered: