Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing LibLUV dependency for most recent neovim versions #64400

Closed
rvolosatovs opened this issue Jul 6, 2019 · 52 comments
Closed

Missing LibLUV dependency for most recent neovim versions #64400

rvolosatovs opened this issue Jul 6, 2019 · 52 comments

Comments

@rvolosatovs
Copy link
Member

Issue description

Since neovim/neovim#10123 neovim requires libluv for builds.
See https://github.com/neovim/neovim/wiki/Following-HEAD#20190610

There's no released version yet containing this change, but it's nice to prepare for this already in order to simplify the update later.

The build on rvolosatovs@2b7a3bb fails for me with:

CMake Error at /nix/store/06zj4w9fdhwfpz9grbm01fqwgsl1ig4g-cmake-3.13.4/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find LibLUV (missing: LIBLUV_LIBRARY LIBLUV_INCLUDE_DIR)
  (Required is at least version "1.30.0")
Call Stack (most recent call first):
  /nix/store/06zj4w9fdhwfpz9grbm01fqwgsl1ig4g-cmake-3.13.4/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindLibLUV.cmake:34 (find_package_handle_standard_args)
  CMakeLists.txt:386 (find_package)

I have no experience with either CMake or Lua, so it's quite hard for me to reason about this and debug, but as far as I understand the issue is that build requires luv.h(https://github.com/luvit/luv/blob/1.30.0-0/src/luv.h ???) to be present at build time and luaPackages.luv does not contain it:

tree $(nix eval --raw nixpkgs.luaPackages.luv)
/nix/store/gkszplgfs3z05765pbqqq9ahhsv1j8wv-lua5.2-luv-1.29.1-2
├── lib
│   └── lua
│       └── 5.2
│           └── luv.so
├── luv-1.29.1-2-rocks
│   ├── luv
│   │   └── 1.29.1-2
│   │       ├── doc
│   │       │   ├── docs.md
│   │       │   ├── LICENSE.txt
│   │       │   └── README.md
│   │       ├── luv-1.29.1-2.rockspec
│   │       └── rock_manifest
│   └── manifest
└── nix-support
    └── propagated-build-inputs

Refs neovim/neovim#10181 neovim/neovim#10359 (comment)

cc @manveru

@rvolosatovs
Copy link
Member Author

Update of luv to version 1.30.0 did not help.
master...rvolosatovs:feature/nightly-neovim

@gloaming
Copy link
Contributor

gloaming commented Jul 6, 2019

They didn't include a pkgconfig until after that version: luvit/luv@df03f2a

You can specify it manually, but our version of luv does not include the header files, since that requires a different build configuration (see just after the pkgconfig part). Changing the build configuration for lua packages is apparently a problem? @dtzWill @Shados

luv = super.luv.override({
# Use system libuv instead of building local and statically linking
# This is a hacky way to specify -DWITH_SHARED_LIBUV=ON which
# is not possible with luarocks and the current luv rockspec
# While at it, remove bundled libuv source entirely to be sure.
# We may wish to drop bundled lua submodules too...
preBuild = ''
sed -i 's,\(option(WITH_SHARED_LIBUV.*\)OFF,\1ON,' CMakeLists.txt
rm -rf deps/libuv
'';
propagatedBuildInputs = [
pkgs.libuv
];
});

@teto
Copy link
Member

teto commented Jul 8, 2019

There is a lot going on between libluv's buildsystem and neovim's. It's urgent to wait as some say.
neovim/neovim#10449 is another way closer to a solution.

@blueyed
Copy link
Contributor

blueyed commented Jul 8, 2019

@teto
While a step to better integration, neovim/neovim#10449 should only help when not using LuaJIT.

@gloaming
Copy link
Contributor

gloaming commented Jul 8, 2019

@teto I don't think that's relevant to us in any case. That discussion is about how to build the "bundled" luv, which we aren't using anyway.

@Shados
Copy link
Member

Shados commented Jul 9, 2019

@gloaming I think we will want to wait until luvit/luv#344 and neovim/neovim#10292 both play out, and a new luv luarock has been published. If necessary, we can then just add a postInstall phase to luv's existing overrides, to perform the make BUILD_MODULE=0 install (with appropriate make flags).

@blueyed
Copy link
Contributor

blueyed commented Jul 9, 2019

Waiting for luvit/luv#344 is not really necessary, is it? Also, from what I can tell LuaRocks 3 has several issues (at least within Neovim's bundled build system, for both Linux/Windows)).

@Shados
Copy link
Member

Shados commented Jul 9, 2019

@blueyed nixpkgs internally uses Luarocks 3.1.3 currently, so if post-1.3.0 luv changes have caused issues building with Luarocks 3.1.3 in general (rather than just in neovim's bundled build system), then yes, we'd want to wait for you/someone else to figure that out :P.

More broadly though, I am interested in knowing what issues you've run into with Luarocks 3, at least on the Linux side.

EDIT: Feel free to poke me on Freenode (same nick) if you want to discuss.

@blueyed
Copy link
Contributor

blueyed commented Jul 9, 2019

@Shados
I see.. :)

For Linux it's only that we have to switch to "make install" instead of "make bootstrap". luarocks/luarocks#1043 should fix this.
That's a good work around I guess - I am not sure why "bootstrap" was used in the first place, or what benefits it would provide.

The (main) issue on Windows is fixed via luvit/luv@9d28d38 (released already).

@teto
Copy link
Member

teto commented Jul 17, 2019

Seems like there was a new release luvit/luv@b9b096d , might be worth trying again.

@rvolosatovs
Copy link
Member Author

rvolosatovs commented Jul 27, 2019

The issue persists on rvolosatovs@19e8bf7 (see diff here: master...rvolosatovs:19e8bf796eeacfaa3d44818fa4cc2ccf029c0d20):

Build log
nix-build -A neovim
these derivations will be built:
  /nix/store/h5h2qj4jg02br9qrkx6p43wfpvwfm7g8-ruby2.5.5-msgpack-1.2.6.drv
  /nix/store/q68c9gpk31906bp8xl2i40lrdam329wz-ruby2.5.5-neovim-0.8.0.drv
  /nix/store/7vd8yq0a5cl6hjh4rn18vd3hc98pl2q6-neovim-ruby-env.drv
  /nix/store/ngrp0rnwpi1js3qyyw6pzf1qf1pax5l2-neovim-unwrapped-nightly.drv
  /nix/store/4zibm78fbh75h4rqvqxy9y2ykkrs5psz-neovim-nightly.drv
building '/nix/store/ngrp0rnwpi1js3qyyw6pzf1qf1pax5l2-neovim-unwrapped-nightly.drv'...
building '/nix/store/h5h2qj4jg02br9qrkx6p43wfpvwfm7g8-ruby2.5.5-msgpack-1.2.6.drv'...
unpacking sources
patching sources
configuring
no configure script, doing nothing
installing
buildFlags:
unpacking sources
unpacking source archive /nix/store/flslb5ngm36x22jdfjl5s93y7q931njq-source
WARNING:  You build with buildroot.
  Build root: /
  Bin dir: /nix/store/ws471an5m3ysi7mng7hlh99jj3p7270d-ruby2.5.5-msgpack-1.2.6/lib/ruby/gems/2.5.0/bin
  Gem home: /nix/store/ws471an5m3ysi7mng7hlh99jj3p7270d-ruby2.5.5-msgpack-1.2.6/lib/ruby/gems/2.5.0
Building native extensions. This could take a while...
source root is source
patching sources
applying patch /nix/store/jnp9n2x9cdvl47da3656illrd3p2qvb5-system_rplugin_manifest.patch
patching file runtime/autoload/remote/host.vim
patching file runtime/plugin/rplugin.vim
configuring
cmake flags: -DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON -DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_SKIP_BUILD_RPATH=ON -DBUILD_TESTING=OFF -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/m8c89zhvwl7c62zvlmgl1f9d8jsnzikc-neovim-unwrapped-nightly/include -DCMAKE_INSTALL_LIBDIR=/nix/store/m8c89zhvwl7c62zvlmgl1f9d8jsnzikc-neovim-unwrapped-nightly/lib -DCMAKE_INSTALL_NAME_DIR=/nix/store/m8c89zhvwl7c62zvlmgl1f9d8jsnzikc-neovim-unwrapped-nightly/lib -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_OSX_DEPLOYMENT_TARGET= -DCMAKE_OSX_SYSROOT= -DCMAKE_FIND_FRAMEWORK=last -DCMAKE_STRIP=/nix/store/mgdjnsrkqgmxqjaqaxgqyqm7fwyi96fk-binutils-2.31.1/bin/strip -DCMAKE_RANLIB=/nix/store/mgdjnsrkqgmxqjaqaxgqyqm7fwyi96fk-binutils-2.31.1/bin/ranlib -DCMAKE_AR=/nix/store/mgdjnsrkqgmxqjaqaxgqyqm7fwyi96fk-binutils-2.31.1/bin/ar -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_INSTALL_PREFIX=/nix/store/m8c89zhvwl7c62zvlmgl1f9d8jsnzikc-neovim-unwrapped-nightly -DGPERF_PRG=/nix/store/xckpvzqf07fmsc666p89rybay2bi32d0-gperf-3.1/bin/gperf -DLUA_PRG=/nix/store/adjypv75qi2x8b5zhwg306sir901jiij-luajit-2.1.0-beta3-env/bin/lua -DUSE_BUNDLED_LUV=ON
-- The C compiler identification is GNU 7.4.0
-- Check for working C compiler: /nix/store/hpzj855nkgjvg58nrhq4910sb9q3kss1-gcc-wrapper-7.4.0/bin/gcc
-- Check for working C compiler: /nix/store/hpzj855nkgjvg58nrhq4910sb9q3kss1-gcc-wrapper-7.4.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- CMAKE_INSTALL_PREFIX=/nix/store/m8c89zhvwl7c62zvlmgl1f9d8jsnzikc-neovim-unwrapped-nightly
-- CMAKE_BUILD_TYPE=Release
-- MIN_LOG_LEVEL not specified, default is 1 (INFO)
-- Replacing -O3 in CMAKE_C_FLAGS_RELEASE with -O2
-- Performing Test HAS_OG_FLAG
-- Performing Test HAS_OG_FLAG - Success
-- Performing Test HAS_ACCEPTABLE_FORTIFY
-- Performing Test HAS_ACCEPTABLE_FORTIFY - Success
-- Performing Test HAVE_EXECINFO_BACKTRACE
-- Performing Test HAVE_EXECINFO_BACKTRACE - Success
-- Performing Test HAVE_BUILTIN_ADD_OVERFLOW
-- Performing Test HAVE_BUILTIN_ADD_OVERFLOW - Success
-- Performing Test HAS_WIMPLICIT_FALLTHROUGH_FLAG
-- Performing Test HAS_WIMPLICIT_FALLTHROUGH_FLAG - Success
-- Performing Test HAS_WVLA_FLAG
-- Performing Test HAS_WVLA_FLAG - Success
-- Performing Test HAS_FSTACK_PROTECTOR_STRONG_FLAG
-- Performing Test HAS_FSTACK_PROTECTOR_STRONG_FLAG - Success
-- Performing Test HAS_FSTACK_PROTECTOR_FLAG
-- Performing Test HAS_FSTACK_PROTECTOR_FLAG - Success
-- Performing Test HAS_DIAG_COLOR_FLAG
-- Performing Test HAS_DIAG_COLOR_FLAG - Success
-- Found PkgConfig: /nix/store/01a3dppfi3rv03m8yx2nwxbkklggyzf9-pkg-config-0.29.2/bin/pkg-config (found version "0.29.2")
-- Looking for dlopen in dl
-- Looking for dlopen in dl - found
-- Looking for kstat_lookup in kstat
-- Looking for kstat_lookup in kstat - not found
-- Looking for kvm_open in kvm
-- Looking for kvm_open in kvm - not found
-- Looking for gethostbyname in nsl
-- Looking for gethostbyname in nsl - found
-- Looking for perfstat_cpu in perfstat
-- Looking for perfstat_cpu in perfstat - not found
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - found
-- Looking for sendfile in sendfile
-- Looking for sendfile in sendfile - not found
-- Found LibUV: /nix/store/43gl6cfmzagx5plly02hswb7yhrl9fyn-libuv-1.30.1/lib/libuv.so (Required is at least version "1.28.0")
-- Found Msgpack: /nix/store/823rqv68q4aw29k1kl71kx5b7vfcs7xb-msgpack-3.0.1/lib/libmsgpackc.so (found suitable version "3.0.1", minimum required is "1.0.0")
CMake Error at /nix/store/y09hh3g31pnm68cfb6rnn5pql04si6mp-cmake-3.14.5/share/cmake-3.14/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find LibLUV (missing: LIBLUV_LIBRARY LIBLUV_INCLUDE_DIR)
  (Required is at least version "1.30.0")
Call Stack (most recent call first):
  /nix/store/y09hh3g31pnm68cfb6rnn5pql04si6mp-cmake-3.14.5/share/cmake-3.14/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindLibLUV.cmake:34 (find_package_handle_standard_args)
  CMakeLists.txt:375 (find_package)


-- Configuring incomplete, errors occurred!
See also "/build/source/build/CMakeFiles/CMakeOutput.log".
See also "/build/source/build/CMakeFiles/CMakeError.log".
builder for '/nix/store/ngrp0rnwpi1js3qyyw6pzf1qf1pax5l2-neovim-unwrapped-nightly.drv' failed with exit code 1
cannot build derivation '/nix/store/4zibm78fbh75h4rqvqxy9y2ykkrs5psz-neovim-nightly.drv': 1 dependencies couldn't be built
error: build of '/nix/store/4zibm78fbh75h4rqvqxy9y2ykkrs5psz-neovim-nightly.drv' failed

@gloaming
Copy link
Contributor

@rvolosatovs could you kindly wrap long logs in

? :)

Well, nothing relevant changed upstream, so it's not surprising it doesn't work.

@rvolosatovs
Copy link
Member Author

@gloaming updated

@gloaming
Copy link
Contributor

I have it working here: gloaming@9302546

To do it more nicely, we would need luv to provide a unified build configuration that produces all outputs and a single pkgconfig (and rockspec, I assume).

Preferably, we should also get the Neovim build to accept a dynamic library.

@blueyed
Copy link
Contributor

blueyed commented Jul 28, 2019

Preferably, we should also get the Neovim build to accept a dynamic library.

Are you referring to Neovim itself, or your build of it?
(Neovim itself should not require a static library - it only does so internally when using its bundled deps / third-party build, but that should not be used usually with distributions in the first place)

I have this for example (in CMakeCache.txt): LIBLUV_LIBRARY:FILEPATH=/usr/lib/libluv.so.

@gloaming
Copy link
Contributor

Ah. The problem is that our library is named luv.so instead of libluv.so, but Neovim's CMake does not pass the actual filename to gcc:

gcc [...] -L/nix/store/ywhwnh7x6yi331d7x46s8k4wnxgqnkdm-luajit-2.1.0-beta3-luv-1.30.0-0/lib/lua/5.1 -lluv -luv -lrt -lpthread -lnsl -ldl -ldl -lnsl -lrt -lmsgpackc -lvterm -ltermkey -lunibilium -lpthread -lm -lutil -lluajit-5.1 -lpthread -lmsgpackc -lvterm -ltermkey -lunibilium -lpthread -lm -lutil -lluajit-5.1
/nix/store/glhymgljg23hkyii6dgc5b276ajafm2x-binutils-2.31.1/bin/ld: cannot find -lluv
collect2: error: ld returned 1 exit status
make[2]: *** [src/nvim/CMakeFiles/nvim.dir/build.make:3218: bin/nvim] Error 1

So it implicitly requires that the file is called libluv.so. I think that's a Neovim bug.

@blueyed
Copy link
Contributor

blueyed commented Jul 28, 2019

The problem is that our library is named luv.so instead of libluv.so

Why?

Neovim's CMake looks for "luv" as the library name, so this translates to "libluv.so" then.

You could use -DLIBLUV_LIBRARY:FILEPATH=/path/to/luv.so I guess, but I wonder why not using libluv.so from the beginning.

@teto
Copy link
Member

teto commented Jul 29, 2019

@gloaming your patch works thanks !

I think nixpkgs' luv.so should be named libluv.so as it's the convention. Why isn't it the case already? Is that an issue with luv's installer ?

NB: I merged the libluv 1.30 bump thanks

@gloaming
Copy link
Contributor

I checked several other Lua libraries and they are all named the same way. It appears to be the convention:

https://github.com/brimworks/lua-zip/blob/e01fac77e7ebf1feceed8f95a5de96aa2758e06f/CMakeLists.txt#L34-L37
libmpack/libmpack-lua@1009ed3

@vcunat
Copy link
Member

vcunat commented Jul 30, 2019

Yes, that's so. Also, I've never seen trying to link a lua library the normal C way – they're always open dynamically by require('luv') (or an equivalent to this lua code).

@blueyed
Copy link
Contributor

blueyed commented Jul 30, 2019

Sure require('luv') makes sense.
Also the include location is "luv" for me on Arch Linux:

libluv /usr/
libluv /usr/include/
libluv /usr/include/luv/
libluv /usr/include/luv/lhandle.h
libluv /usr/include/luv/lreq.h
libluv /usr/include/luv/luv.h
libluv /usr/include/luv/util.h
libluv /usr/lib/
libluv /usr/lib/libluv.so
libluv /usr/lib/libluv.so.1
libluv /usr/lib/libluv.so.1.30.0

According to https://cmake.org/cmake/help/latest/command/find_library.html CMake should find it though:

Each library name given to the NAMES option is first considered as a library file name and then considered with platform-specific prefixes (e.g. lib) and suffixes (e.g. .so). Therefore one may specify library file names such as libfoo.a directly. This can be used to locate static libraries on UNIX-like systems.

Code ref in Neovim: https://github.com/neovim/neovim/blob/6e01ed6a4c859ff915b42422622f1ba3cc80efd5/cmake/FindLibLUV.cmake#L23-L26

@gloaming
Copy link
Contributor

It does find it: -- Found LibLUV: /nix/store/3vq3irgvpx00cvp7y8aay64vk7rkwbdl-luajit-2.1.0-beta3-luv-1.30.0-0/lib/lua/5.1/luv.so (Required is at least version "1.30.0")

The problem is the linker options passed to gcc. It should be -l:luv.so instead of -lluv, see ld(1):

-l namespec
--library=namespec
Add the archive or object file specified by namespec to the list of files to link. This option may be used any number of times. If namespec is of the form :filename, ld will search the library path for a file called filename, otherwise it will search the library path for a file called libnamespec.a.

On systems which support shared libraries, ld may also search for files other than libnamespec.a. Specifically, on ELF and SunOS systems, ld will search a directory for a library called libnamespec.so before searching for one called libnamespec.a. (By convention, a ".so" extension indicates a shared library.) Note that this behavior does not apply to :filename, which always specifies a file called filename.

@blueyed
Copy link
Contributor

blueyed commented Jul 30, 2019

Ah, I see.
I would assume CMake to handle that.. we're using target_link_libraries here: https://github.com/blueyed/neovim/blob/e107f94ea2e8dd4e664a7598a69eb0dd1b1c5f09/src/nvim/CMakeLists.txt#L411.
According to https://cmake.org/cmake/help/latest/command/target_link_libraries.html:

There are some cases where CMake may ask the linker to search for the library (e.g. /usr/lib/libfoo.so becomes -lfoo), such as when a shared library is detected to have no SONAME field. See policy CMP0060 for discussion of another case.

I assume we could set the policy to new maybe?
https://cmake.org/cmake/help/latest/policy/CMP0060.html#policy:CMP0060

Having a SONAME might be a better workaround though?

blueyed added a commit to blueyed/neovim that referenced this issue Jul 30, 2019
This is meant to help with linking against "luv.so", which does not work
with "-lluv", if "luv.so" should be used.

Ref: NixOS/nixpkgs#64400 (comment)
@blueyed
Copy link
Contributor

blueyed commented Jul 30, 2019

Can you try the branch from neovim/neovim#10661?

@gloaming
Copy link
Contributor

No change; it looks like CMake is determined that it knows better in the case where a library has no SONAME. This is annoying for us, since under Nix, packages are not relocatable by definition, so there's no reason to try and prevent it. I think ideally we would get CMake to provide a way to disable this behaviour.

On the other hand, I don't know whether there's any particular reason that our Lua libraries don't have a SONAME in the first place. Presumably this is standard Lua behaviour and nobody needed it since they weren't linking in the first place.

@blueyed
Copy link
Contributor

blueyed commented Jul 30, 2019

According to https://cmake.org/cmake/help/latest/policy/CMP0060.html#policy:CMP0060 it should use the new behavior always then. I assume you have CMake >= 3.3?
Another option might be to use IMPORTED as mentioned with the policy.
Can you add a message(STATUS ${NVIM_EXEC_LINK_LIBRARIES}) to check that it is still absolute there, and that the policy gets set actually (i.e. put it in the if block there).

blueyed added a commit to blueyed/neovim that referenced this issue Jul 30, 2019
This is meant to help with linking against "luv.so", which does not work
with "-lluv", if "luv.so" should be used.

Ref: NixOS/nixpkgs#64400 (comment)
@gloaming
Copy link
Contributor

Yes, we're on CMake 3.14.5.

I don't think it says CMP0060 will cause it to always happen, see here: https://cmake.org/cmake/help/latest/command/target_link_libraries.html#command:target_link_libraries

There are some cases where CMake may ask the linker to search for the library (e.g. /usr/lib/libfoo.so becomes -lfoo), such as when a shared library is detected to have no SONAME field. See policy CMP0060 for discussion of another case.

I forced the build by manually passing linker flags, and I noticed that the resulting binary in our case has the expected reference:

 0x0000000000000001 (NEEDED)             Shared library: [luv.so]

(Although it doesn't acquire the RUNPATH for some reason.)


In any case, it looks like this is not a NixOS-specific issue - it will arise on any distribution that doesn't rename the lua shared object. I think you guys need to take this up with CMake and the Lua build system to figure out the right way forward.

@blueyed
Copy link
Contributor

blueyed commented Jul 31, 2019

Have you tried the latest branch? (from yesterday)

luv explicitly requests to have no prefix here with -DBUILD_MODULE=1: https://github.com/luvit/luv/blob/d7e0afb3f35877fde31e0f31e001388013310fb5/CMakeLists.txt#L133

This is what the Arch User Repo package is using:

build() {
    mkdir "${srcdir}/build"
    cd "${srcdir}/build"
    cmake -DWITH_SHARED_LIBUV=ON -DLUA_BUILD_TYPE=System \
        -DBUILD_MODULE=OFF -DBUILD_SHARED_LIBS=ON \
        -DCMAKE_INSTALL_PREFIX=/usr \
        "${srcdir}/luv-${pkgver}-${pkgrel}"
    make
}

package() {
    cd "${srcdir}/build"
    DESTDIR="${pkgdir}" make install
}

Maybe you could combine it / build it twice to have both variants?

As far as I can see luv.so is only useful for Lua, and there it would go into a non-standard location (/usr/lib/lua/5.3/luv.so) then.

I forced the build by manually passing linker flags

How exactly?

For reference, this is the CMake code:

bool cmComputeLinkInformation::CheckImplicitDirItem(std::string const& item)
{
  // We only switch to a pathless item if the link type may be
  // enforced.  Fortunately only platforms that support link types
  // seem to have magic per-architecture implicit link directories.
  if (!this->LinkTypeEnabled) {
    return false;
  }

  // Check if this item is in an implicit link directory.
  std::string dir = cmSystemTools::GetFilenamePath(item);
  if (this->ImplicitLinkDirs.find(dir) == this->ImplicitLinkDirs.end()) {
    // Only libraries in implicit link directories are converted to
    // pathless items.
    return false;
  }

  // Only apply the policy below if the library file is one that can
  // be found by the linker.
  std::string file = cmSystemTools::GetFilenameName(item);
  if (!this->ExtractAnyLibraryName.find(file)) {
    return false;
  }

  // Check the policy for whether we should use the approach below.
  switch (this->Target->GetPolicyStatusCMP0060()) {
    case cmPolicies::WARN:
      if (this->CMP0060Warn) {
        // Print the warning at most once for this item.
        std::string const& wid = "CMP0060-WARNING-GIVEN-" + item;
        if (!this->CMakeInstance->GetPropertyAsBool(wid)) {
          this->CMakeInstance->SetProperty(wid, "1");
          this->CMP0060WarnItems.insert(item);
        }
      }
    case cmPolicies::OLD:
      break;
    case cmPolicies::REQUIRED_ALWAYS:
    case cmPolicies::REQUIRED_IF_USED:
    case cmPolicies::NEW:
      return false;
  }

  // Many system linkers support multiple architectures by
  // automatically selecting the implicit linker search path for the
  // current architecture.  If the library appears in an implicit link
  // directory then just report the file name without the directory
  // portion.  This will allow the system linker to locate the proper
  // library for the architecture at link time.
  this->AddUserItem(file, false);

  // Make sure the link directory ordering will find the library.
  this->OrderLinkerSearchPath->AddLinkLibrary(item);

  return true;
}

In any case, it looks like this is not a NixOS-specific issue - it will arise on any distribution that doesn't rename the lua shared object.

Well, it depends on the usage of BUILD_MODULE=1.

This is from luv's README:

#### Build as shared library

If you want to build luv as a shared library run make with:

~/Code/luv> BUILD_MODULE=OFF BUILD_SHARED_LIBS=ON make

@gloaming
Copy link
Contributor

gloaming commented Aug 1, 2019

Oh, now I get it. Other distros are providing a libluv installable outside of the Lua ecosystem, right?

I tried blueyed/neovim@8172f6ffb, it still gives the same error (cannot link -lluv). I am fairly sure the CMake documentation is not lying on this one.

How exactly?

gloaming/neovim@4f06dd4
Although it works, and isn't even that much of a hack, I don't think it's reasonable to expect packages to jump through hoops like this. I think it's best to just provide a libluv output packages can consume and be done with it: gloaming/nixpkgs@081abe5...8e0f135

@gloaming
Copy link
Contributor

gloaming commented Aug 1, 2019

On what platform is there still an actual distinction between SHARED and MODULE libraries? I don't think there is one. It seems bizarre to provide multiple copies of the library just to get a different filename and SONAME. But improving this situation is a question of whether anyone wants to overhaul luv's build script.

@vcunat
Copy link
Member

vcunat commented Aug 1, 2019

IIRC on darwin they differ in the extension (.so and .dylib).

@blueyed
Copy link
Contributor

blueyed commented Aug 3, 2019

This works:

make CMAKE_EXTRA_FLAGS='-DLIBLUV_LIBRARY:STRING=-Wl,/path/to/luv.so'

I.e. set our/nvim's LIBLUV_LIBRARY to a string value, which passes on an argument for the linker directly. This skips version checks etc then though, but that's ok when you know what you are doing.

blueyed added a commit to blueyed/neovim that referenced this issue Aug 8, 2019
Use "luv" as imported library to work around "-lluv" being used due to
missing SONAME.

Fixes neovim#10407.
Ref: https://gitlab.kitware.com/cmake/cmake/issues/19578
Ref: NixOS/nixpkgs#64400 (comment)
@blueyed
Copy link
Contributor

blueyed commented Aug 8, 2019

I think I've found a workaround: neovim/neovim#10661 (comment) - please try the branch there again.

blueyed added a commit to blueyed/neovim that referenced this issue Aug 8, 2019
Use "luv" as imported library to work around "-lluv" being used due to
missing SONAME.

Fixes neovim#10407.
Ref: https://gitlab.kitware.com/cmake/cmake/issues/19578
Ref: NixOS/nixpkgs#64400 (comment)
@blueyed
Copy link
Contributor

blueyed commented Aug 11, 2019

Gentle ping to provide feedback on neovim/neovim#10661.

blueyed added a commit to neovim/neovim that referenced this issue Aug 12, 2019
Use "luv" as imported library to work around "-lluv" being used due to
missing SONAME.

Fixes #10407.
Ref: https://gitlab.kitware.com/cmake/cmake/issues/19578
Ref: NixOS/nixpkgs#64400 (comment)
@gloaming
Copy link
Contributor

Yeah, it works 👍

Note that Nix needs the second build configuration to get the headers anyway, so I think it's best for us to go that route. (Which means the changes on your end would be for other users' benefit, not us.)

Unfortunately the libluv build fails with the new luv release:

CMake Error: File /build/luv-1.30.1-0/luv-1.30.1-0/libluv.pc.in does not exist.
CMake Error at CMakeLists.txt:206 (configure_file):
  configure_file Problem configuring file


-- Configuring incomplete, errors occurred!
See also "/build/luv-1.30.1-0/luv-1.30.1-0/build.luarocks/CMakeFiles/CMakeOutput.log".

Error: Build error: Failed cmake.

builder for '/nix/store/5d36r8yvv4aklr5fysb3ph00hv4mjjjl-luajit-2.1.0-beta3-luv-1.30.1-0.drv' failed with exit code 1
error: build of '/nix/store/5d36r8yvv4aklr5fysb3ph00hv4mjjjl-luajit-2.1.0-beta3-luv-1.30.1-0.drv' failed

Someone should look into that.

@rvolosatovs
Copy link
Member Author

@tex
Copy link
Contributor

tex commented Sep 5, 2019

This works for me too.

@garbas
Copy link
Member

garbas commented Sep 5, 2019

works for me as well (commit).

@teto
Copy link
Member

teto commented Sep 12, 2019

2 or 3 days ago, libvterm deps got updated so neovim 0.4 will need that libvterm https://github.com/teto/home/blob/4f4a3bc8a2bdd69bcbd2e3f78bcd72e8addd771d/config/nixpkgs/overlays/neovim.nix#L155 as well

@ecklf
Copy link
Member

ecklf commented Sep 16, 2019

New release version (0.4.2) will now fail building because of missing
Edit: (this is brew not nix btw sorry)

LIBLUV_LIBRARY / LIBLUV_INCLUDE_DIR

==> cmake .. -DCMAKE_C_FLAGS_RELEASE=-DNDEBUG -DCMAKE_CXX_FLAGS_RELEASE=-DNDEBUG -DCMAKE_INSTALL_PREFIX=/usr/local/Cellar/neovim/0.4.2 -DCMA
Last 15 lines from /Users/lynx/Library/Logs/Homebrew/neovim/04.cmake:
-- Looking for sendfile in sendfile - not found
-- Found LibUV: /usr/local/lib/libuv.dylib (Required is at least version "1.28.0")
-- Found Msgpack: /usr/local/lib/libmsgpackc.dylib (found suitable version "3.2.0", minimum required is "1.0.0")
CMake Error at /usr/local/Cellar/cmake/3.15.3/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find LibLUV (missing: LIBLUV_LIBRARY LIBLUV_INCLUDE_DIR)
  (Required is at least version "1.30.0")
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.15.3/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindLibLUV.cmake:29 (find_package_handle_standard_args)
  CMakeLists.txt:375 (find_package)

@rvolosatovs
Copy link
Member Author

@impulse this is what is necessary to build 0.4.2: #68882

@doronbehar
Copy link
Contributor

Hey guys, I'm trying to build a specific branch of Neovim in a git checkout in order to check a certain Neovim PR. I used neovim-unwrapped's default.nix as a reference for my shell.nix (I use direnv) and I get the following error:

/var/src/neovim/src/nvim/lua/executor.c:35:10: fatal error: luv/luv.h: No such file or directory
 #include "luv/luv.h"
          ^~~~~~~~~~~
compilation terminated.
make[2]: *** [src/nvim/CMakeFiles/nvim.dir/build.make:567: src/nvim/auto/lua/executor.c.generated.h] Error 1
make[1]: *** [CMakeFiles/Makefile2:3433: src/nvim/CMakeFiles/nvim.dir/all] Error 2

This error appears although Cmake runs fine. I use cmakeFlags just as we use in neovim-unwrapped's default.nix.

I tried to test that branch's build by directly editing neovim-unwrapped's default.nix in my checkout of nixpkgs and the error didn't appear. If you are curios, the build wasn't totally successful but I've had other errors with it which seem to "advance" this one. This is my shell.nix:

with import <nixpkgs> {};
pkgs.mkShell rec {
  neovimLuaEnv = lua.withPackages(ps:
    (with ps; [
      lpeg
      luabitop
      mpack
      nvim-client
      luv
      coxpcall
      busted
      luafilesystem
      penlight
      inspect
      luaffi
    ])
  );
  cmakeFlags = [
    "-DGPERF_PRG=${gperf}/bin/gperf"
    "-DLUA_PRG=${neovimLuaEnv.interpreter}"
    "-DPREFER_LUA=ON"
    "-DBUSTED_PRG=${neovimLuaEnv}/bin/busted"
    
    "-DLIBLUV_LIBRARY=${lua.pkgs.luv}/lib/lua/${lua.luaversion}/luv.so"

    # This isn't part of neovim-unwrapped's default.nix but it's my attempt to
    # fix the missing header file as that's where I've found that luv.h
    "-DLIBLUV_INCLUDE_DIR=${neovimLuaEnv}/include"
  ];
  buildInputs = [
    gettext
    gperf
    libtermkey
    libuv
    libvterm
    luaPackages.luv
    msgpack
    ncurses
    neovimLuaEnv
    unibilium
    jemalloc
    libiconv
    procps
    glibcLocales
    cmake
    pkgconfig
  ];
}

BTW, I expected this issue to be closed now, since the 0.4.2 upgrade was merged. Are there any other luv related issues besides mine people are having?

@gloaming
Copy link
Contributor

@doronbehar You've got luv in your buildInputs, should be lua.pkgs.luv.libluv

@doronbehar
Copy link
Contributor

Thanks @gloaming, you've eventually helped me to find the problem - my channels were'nt updated enough and that's why lua.pkgs.luv.libluv gave me a missing attribute error.

I've also workaround any other issues with a much simpler shell.nix:

with import <nixpkgs> {};
pkgs.mkShell rec {
  cmakeFlags = neovim-unwrapped.cmakeFlags;
  buildInputs = neovim-unwrapped.buildInputs ++ neovim-unwrapped.nativeBuildInputs;
}

Anyway, to conclude, I don't experience any other issues related to luv so as I said before, I think this issue can be closed.

@vegerot
Copy link

vegerot commented Nov 11, 2019

Similar error in macOS Catalina when trying to run cmake -G Xcode neovim

-- Found LibUV: /usr/local/Cellar/libuv/1.31.0/lib/libuv.dylib (Required is at least version "1.28.0")
-- Found Msgpack: /usr/local/Cellar/msgpack/3.2.0/lib/libmsgpackc.dylib (found suitable version "3.2.0", minimum required is "1.0.0")
CMake Error at /usr/local/Cellar/cmake/3.15.5/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find LibLUV (missing: LIBLUV_LIBRARY LIBLUV_INCLUDE_DIR)
  (Required is at least version "1.30.0")
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.15.5/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindLibLUV.cmake:29 (find_package_handle_standard_args)
  CMakeLists.txt:371 (find_package)

And when trying to run "make ."

config.status: linking /Users/maxcoplan/Documents/workspace/neovim/.deps/build/src/libuv/test/fixtures/empty_file to test/fixtures/empty_file
config.status: linking /Users/maxcoplan/Documents/workspace/neovim/.deps/build/src/libuv/test/fixtures/load_error.node to test/fixtures/load_error.node
config.status: executing depfiles commands
config.status: executing libtool commands
ninja: build stopped: subcommand failed.
gmake: *** [Makefile:101: deps] Error 1

@doronbehar
Copy link
Contributor

@vegerot is this the error you get when running cmake from nix-shell / direnv? What is your shell.nix?

@ersinakinci
Copy link
Contributor

I'm getting the following error when trying to run nix-env -f '<nixpkgs>' -iA neovim on macOS--is it related? If so, how can I work around or fix it?

installing 'neovim-0.4.3'
these derivations will be built:
  /nix/store/479s9v0v0ma0wl3157hq6ppnvxk5ykzb-luajit-2.1.0-beta3-luv-1.30.0-0.drv
  /nix/store/5cswwdn7ljsnxy1xayvqvxyvqgxjcjkv-neovim-ruby-env.drv
  /nix/store/cz124m6x406nxhvyc3hvs3ffn9prck32-python-2.7.17-env.drv
  /nix/store/gw6080mnwgys53h0sw43bp26xksbx33k-python3-3.7.6-env.drv
  /nix/store/nr8zhi7hjf0g0rgvnknkn02grdfk3lyx-neovim-unwrapped-0.4.3.drv
  /nix/store/851qma1nnxd1wbviwr3rayqmpkl5bi43-neovim-0.4.3.drv
building '/nix/store/5cswwdn7ljsnxy1xayvqvxyvqgxjcjkv-neovim-ruby-env.drv'...
created 12 symlinks in user environment
building '/nix/store/479s9v0v0ma0wl3157hq6ppnvxk5ykzb-luajit-2.1.0-beta3-luv-1.30.0-0.drv'...
unpacking sources
unpacking source archive /nix/store/iqsk16lkjsixwmrq9w3b6szh4nm3mpcf-luv-1.30.0-0.src.rock

Done. You may now enter directory
luv-1.30.0-0/luv-1.30.0-0
and type 'luarocks make' to build.
source root is ./luv-1.30.0-0/luv-1.30.0-0
setting SOURCE_DATE_EPOCH to timestamp 1561806879 of file ./luv-1.30.0-0/luv-1.30.0-0/src/work.c
patching sources
configuring
building
patching script interpreter paths in .
./deps/lua-compat-5.3/tests/test.lua: interpreter directive changed from "/usr/bin/env lua" to "/nix/store/h3wc33sf3snh2whhn1q5n428kw2j6l1z-luajit-2.1.0-beta3/bin/lua"
installing

luv 1.30.0-0 depends on lua >= 5.1 (5.1-1 provided by VM)
Warning: unmatched variable LUA_LIBDIR
-- The C compiler identification is Clang 7.1.0
-- The ASM compiler identification is Clang
-- Found assembler: /nix/store/2xv7ha3rsc8cd06hyajmhys91fnir3xg-clang-wrapper-7.1.0/bin/clang
-- Check for working C compiler: /nix/store/2xv7ha3rsc8cd06hyajmhys91fnir3xg-clang-wrapper-7.1.0/bin/clang
-- Check for working C compiler: /nix/store/2xv7ha3rsc8cd06hyajmhys91fnir3xg-clang-wrapper-7.1.0/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found LIBUV: /nix/store/7skf0l31yzjxsnw9wymjd93sl3yr69dg-libuv-1.34.2/lib/libuv.dylib
-- Lua: using information from luarocks
-- LUA_LIBDIR:
-- LUA_INCDIR: /nix/store/h3wc33sf3snh2whhn1q5n428kw2j6l1z-luajit-2.1.0-beta3/include/luajit-2.1
-- LUA: /nix/store/h3wc33sf3snh2whhn1q5n428kw2j6l1z-luajit-2.1.0-beta3/bin/luajit
-- Lua library: LUA_LIBRARIES-NOTFOUND
-- Configuring done
CMake Warning (dev):
  Policy CMP0042 is not set: MACOSX_RPATH is enabled by default.  Run "cmake
  --help-policy CMP0042" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.

  MACOSX_RPATH is not specified for the following targets:

   luv

This warning is for project developers.  Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: /tmp/nix-build-luajit-2.1.0-beta3-luv-1.30.0-0.drv-0/luv-1.30.0-0/luv-1.30.0-0/build.luarocks
Scanning dependencies of target luv
[ 50%] Building C object CMakeFiles/luv.dir/src/luv.c.o
[100%] Linking C shared library libluv.dylib
[100%] Built target luv
[100%] Built target luv
Install the project...
-- Install configuration: ""
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.1.30.0.dylib
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.1.dylib
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.dylib
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/luv.h
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/util.h
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/lhandle.h
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/lreq.h
cp: cannot stat '/nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.1.dylib': No such file or directory

Error: Failed copying /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.1.dylib to /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/lib/lua/5.1/libluv.1.dylib
builder for '/nix/store/479s9v0v0ma0wl3157hq6ppnvxk5ykzb-luajit-2.1.0-beta3-luv-1.30.0-0.drv' failed with exit code 1
cannot build derivation '/nix/store/nr8zhi7hjf0g0rgvnknkn02grdfk3lyx-neovim-unwrapped-0.4.3.drv': 1 dependencies couldn't be built
building '/nix/store/cz124m6x406nxhvyc3hvs3ffn9prck32-python-2.7.17-env.drv'...
cannot build derivation '/nix/store/851qma1nnxd1wbviwr3rayqmpkl5bi43-neovim-0.4.3.drv': 1 dependencies couldn't be built
error: build of '/nix/store/851qma1nnxd1wbviwr3rayqmpkl5bi43-neovim-0.4.3.drv' failed

@NilsIrl
Copy link
Member

NilsIrl commented Mar 5, 2020

cp: cannot stat '/nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.1.dylib': No such file or directory

seems related to me. Actually wrong issue. Take a look at #81206

@vcunat
Copy link
Member

vcunat commented Mar 5, 2020

Except for darwin there's now a different (yet similar) error after solving that one; see https://hydra.nixos.org/job/nixpkgs/trunk/neovim-unwrapped.x86_64-darwin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests