Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

neovim fails to build on master (due to lua / luarocks) #79870

Closed
bjornfor opened this issue Feb 11, 2020 · 15 comments
Closed

neovim fails to build on master (due to lua / luarocks) #79870

bjornfor opened this issue Feb 11, 2020 · 15 comments
Labels
0.kind: bug Something is broken 0.kind: regression Something that worked before working no longer 6.topic: lua 6.topic: vim

Comments

@bjornfor
Copy link
Contributor

Describe the bug
neovim no longer builds on master (tested f77e057), broken since commit d4f09b6 (by git bisect, ping @doronbehar, author of latest luarocks update).

$ nix-build -A neovim
these derivations will be built:
  /nix/store/4wx9xn4zysrqkia9n55vb46jfxpi6a99-luajit-2.1.0-beta3-luv-1.30.0-0.drv
  /nix/store/3adbwmw56z7qgjqzm7vw7kqxr5hmflw6-neovim-unwrapped-0.4.3.drv
  /nix/store/3yi4yxsws4g3qyjjphhsbx1hlicwnf77-neovim-0.4.3.drv
building '/nix/store/4wx9xn4zysrqkia9n55vb46jfxpi6a99-luajit-2.1.0-beta3-luv-1.30.0-0.drv'...
unpacking sources
unpacking source archive /nix/store/iqsk16lkjsixwmrq9w3b6szh4nm3mpcf-luv-1.30.0-0.src.rock

Done. You may now enter directory
luv-1.30.0-0/luv-1.30.0-0
and type 'luarocks make' to build.
source root is ./luv-1.30.0-0/luv-1.30.0-0
setting SOURCE_DATE_EPOCH to timestamp 1561806879 of file ./luv-1.30.0-0/luv-1.30.0-0/src/work.c
patching sources
configuring
building
patching script interpreter paths in .
./deps/lua-compat-5.3/tests/test.lua: interpreter directive changed from "/usr/bin/env lua" to "/nix/store/vy5x75rdxcr8g4jk2xv31zvd9mfcyb6v-luajit-2.1.0-beta3/bin/lua"
installing

luv 1.30.0-0 depends on lua >= 5.1 (5.1-1 provided by VM)
Warning: unmatched variable LUA_LIBDIR
-- The C compiler identification is GNU 9.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /nix/store/3fncfqvh6mz5x2my2s3aiqa9w0rmvcy4-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/3fncfqvh6mz5x2my2s3aiqa9w0rmvcy4-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/3fncfqvh6mz5x2my2s3aiqa9w0rmvcy4-gcc-wrapper-9.2.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found LIBUV: /nix/store/f711zxf89ga3in4gph12jclqdryc5n34-libuv-1.34.2/lib/libuv.so
-- Lua: using information from luarocks
-- LUA_LIBDIR:
-- LUA_INCDIR: /nix/store/vy5x75rdxcr8g4jk2xv31zvd9mfcyb6v-luajit-2.1.0-beta3/include/luajit-2.1
-- LUA: /nix/store/vy5x75rdxcr8g4jk2xv31zvd9mfcyb6v-luajit-2.1.0-beta3/bin/luajit
-- Lua library: LUA_LIBRARIES-NOTFOUND
-- Configuring done
-- Generating done
-- Build files have been written to: /build/luv-1.30.0-0/luv-1.30.0-0/build.luarocks
Scanning dependencies of target luv
[ 50%] Building C object CMakeFiles/luv.dir/src/luv.c.o
[100%] Linking C shared library libluv.so
[100%] Built target luv
[100%] Built target luv
Install the project...
-- Install configuration: ""
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so.1.30.0
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so.1
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/luv.h
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/util.h
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/lhandle.h
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/lreq.h
cp: cannot stat '/nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so': No such file or directory

Error: Failed copying /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so to /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/lib/lua/5.1/libluv.so
builder for '/nix/store/4wx9xn4zysrqkia9n55vb46jfxpi6a99-luajit-2.1.0-beta3-luv-1.30.0-0.drv' failed with exit code 1
cannot build derivation '/nix/store/3adbwmw56z7qgjqzm7vw7kqxr5hmflw6-neovim-unwrapped-0.4.3.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/3yi4yxsws4g3qyjjphhsbx1hlicwnf77-neovim-0.4.3.drv': 1 dependencies couldn't be built
error: build of '/nix/store/3yi4yxsws4g3qyjjphhsbx1hlicwnf77-neovim-0.4.3.drv' failed

Expected behavior
I expect neovim to (continue to) build when I upgrade my system.

@bjornfor bjornfor added 0.kind: bug Something is broken 0.kind: regression Something that worked before working no longer labels Feb 11, 2020
@doronbehar
Copy link
Contributor

Warning: unmatched variable LUA_LIBDIR

This seems to be the root cause of the issue.

That's a hard issue to debug. What I've done in the meantime is test luarocks install luv (with the right environment from a shell.nix) and it ran flawlessly. Moreover, I couldn't figure out the difference between libluv and luv - they both install different .so files but I can't point a difference between the two upstream.

I've opened an issue on Luarocks' end: luarocks/luarocks#1155 .

Besides that, I noticed that https://github.com/luvit/luv (whether that's libluv or luv) is outdated - we have 1.30.0 and they have 1.34.1. @teto perhaps we should update it so we'd be more confident the issue wasn't fixed upstream and hasn't landed here yet?

I'm ccing @manveru, @rvolosatovs and @Ma27 as the maintainers of Neovim which is the sole user of lua.pkgs.libluv.

@teto
Copy link
Member

teto commented Feb 12, 2020

updating luv is definitely worth a try as their build system required us to do some contortions. Hopefully they've since then made some changes that improve things.

@doronbehar doronbehar mentioned this issue Feb 16, 2020
10 tasks
@ArdaXi
Copy link
Contributor

ArdaXi commented Feb 17, 2020

I can confirm that after overriding luv with 1.34.1-1 as below I can build neovim again.

{
      version = "1.34.1-1";
      src = self.fetchurl {
        url = https://luarocks.org/luv-1.34.1-1.src.rock;
        sha256 = "044cyp25xn35nj5qp1hx04lfkzrpa6adhqjshq2g7wvbga77p1q0";
      };
    }

@mjlbach
Copy link
Contributor

mjlbach commented Feb 17, 2020

@bjornfor please try on latest master after #80369

@bjornfor
Copy link
Contributor Author

bjornfor commented Feb 17, 2020

@bjornfor please try on latest master after #80369

Unfortunately:

$ git pull      # updated to 69efb5cd343a947502bf2372c0385e0afc84c570
$ nix-build -A neovim
...
-- Installing: /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/lhandle.h
-- Installing: /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/lreq.h
shrinking RPATHs of ELF executables and libraries in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1
shrinking /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1/lib/lua/5.1/luv.so
strip is /nix/store/xpzdkp9y8fnliwrs2g5pygx2qmmbrv8w-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1/lib
patching script interpreter paths in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1
checking for references to /build/ in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1...
cp: cannot stat '/nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks/luv/1.34.1-1/lib/libluv.so.1': No such file or directory

Error: Failed copying /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks/luv/1.34.1-1/lib/libluv.so.1 to /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/lib/lua/5.1/libluv.so.1
builder for '/nix/store/r1r1sprc1r1yb29w17vfyfnmi2r09zjv-luajit-2.1.0-beta3-luv-1.34.1-1.drv' failed with exit code 1
cannot build derivation '/nix/store/0xasdnwd3fvxzl7qrs2zf5wsh2f86yrj-neovim-unwrapped-0.4.3.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/6v9iv9jrlwik9a3m5v4hn6gwhph1sc27-neovim-0.4.3.drv': 1 dependencies couldn't be built
error: build of '/nix/store/6v9iv9jrlwik9a3m5v4hn6gwhph1sc27-neovim-0.4.3.drv' failed

I looked inside the store path it's trying to copy from, there is not a single *.so file there:

$ find /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/ -ls
 12867484      4 drwxr-xr-x   4 nixbld1  nixbld       4096 Feb 17 20:15 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/
 12867706      4 drwxr-xr-x   3 nixbld1  nixbld       4096 Feb 17 20:15 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include
 12867707      4 drwxr-xr-x   2 nixbld1  nixbld       4096 Feb 17 20:15 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv
 12867710      8 -rw-r--r--   1 nixbld1  nixbld       5678 Jan 18 06:04 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/luv.h
 12867713      4 -rw-r--r--   1 nixbld1  nixbld       1534 Jan 18 06:04 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/lreq.h
 12867711      4 -rw-r--r--   1 nixbld1  nixbld       1744 Jan 18 06:04 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/util.h
 12867712      4 -rw-r--r--   1 nixbld1  nixbld       2225 Jan 18 06:04 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/lhandle.h
 12867491      4 drwxr-xr-x   2 nixbld1  nixbld       4096 Feb 17 20:15 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks

@jokogr
Copy link
Contributor

jokogr commented Feb 17, 2020

I can confirm @bjornfor's findings, it's not working for me, too.

I can also find the missing library at /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1/lib/lua/5.1/luv.so.

@ArdaXi
Copy link
Contributor

ArdaXi commented Feb 17, 2020

I uhh… don't know what to tell you:

HEAD is now at eaa47284113 luaPackages.luv: 1.30.0-0 -> 1.34.1-1
hiro:nixpkgs:% nix-build --check -A luajitPackages.luv                                                                                                                                     <eaa47284113>
checking outputs of '/nix/store/vr6kd59vkzz0n2lrw6d3kssh4cqz1zvq-luajit-2.1.0-beta3-luv-1.34.1-1.drv'...
unpacking sources
unpacking source archive /nix/store/f0az73f8fi7fp7cz55iapzwr0wh1kd93-luv-1.34.1-1.src.rock

Done. You may now enter directory
luv-1.34.1-1/luv-1.34.1-1
and type 'luarocks make' to build.
source root is ./luv-1.34.1-1/luv-1.34.1-1
setting SOURCE_DATE_EPOCH to timestamp 1579323846 of file ./luv-1.34.1-1/luv-1.34.1-1/src/work.c
patching sources
configuring
building
patching script interpreter paths in .
./deps/lua-compat-5.3/tests/test.lua: interpreter directive changed from "/usr/bin/env lua" to "/nix/store/zhsgiagaqx27hp5j94hy6i46rv3gnh75-luajit-2.1.0-beta3/bin/lua"
installing

luv 1.34.1-1 depends on lua >= 5.1 (5.1-1 provided by VM)
Warning: unmatched variable LUA_LIBDIR
-- The C compiler identification is GNU 9.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /nix/store/fqhjxf9ii4w4gqcsx59fyw2vvj91486a-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/fqhjxf9ii4w4gqcsx59fyw2vvj91486a-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/fqhjxf9ii4w4gqcsx59fyw2vvj91486a-gcc-wrapper-9.2.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found LIBUV: /nix/store/gk304dpcw0x4fx7vp3di9v1lv9s8rhdc-libuv-1.34.2/lib/libuv.so
-- Lua: using information from luarocks
-- LUA_LIBDIR:
-- LUA_INCDIR: /nix/store/zhsgiagaqx27hp5j94hy6i46rv3gnh75-luajit-2.1.0-beta3/include/luajit-2.1
-- LUA: /nix/store/zhsgiagaqx27hp5j94hy6i46rv3gnh75-luajit-2.1.0-beta3/bin/luajit
-- Lua library: LUA_LIBRARIES-NOTFOUND
-- Configuring done
-- Generating done
-- Build files have been written to: /build/luv-1.34.1-1/luv-1.34.1-1/build.luarocks
Scanning dependencies of target luv
[ 50%] Building C object CMakeFiles/luv.dir/src/luv.c.o
[100%] Linking C shared module luv.so
[100%] Built target luv
[100%] Built target luv
Install the project...
-- Install configuration: ""
-- Installing: /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks/luv/1.34.1-1/lib/luv.so
No existing manifest. Attempting to rebuild...
luv 1.34.1-1 is now installed in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1 (license: Apache 2.0)

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1
shrinking /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1/lib/lua/5.1/luv.so
strip is /nix/store/xpzdkp9y8fnliwrs2g5pygx2qmmbrv8w-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1/lib
patching script interpreter paths in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1
checking for references to /build/ in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1...
/nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1

@mjlbach
Copy link
Contributor

mjlbach commented Feb 17, 2020

I just installed neovim on a clean vm using the most recent nixpkgs tarball without errors...

@jokogr
Copy link
Contributor

jokogr commented Feb 17, 2020

@ArdaXi could you try to build luajit.pkgs.luv.libluv as used there:

(lua is overridden with luajit in all-packages.nix for neovim).

@ArdaXi
Copy link
Contributor

ArdaXi commented Feb 18, 2020

@jokogr That one fails, but I'm not on Darwin so that's not a dependency of Neovim for me.

# FIXME: this is verry messy and strange.
++ optional (!stdenv.isDarwin) "-DLIBLUV_LIBRARY=${lua.pkgs.luv}/lib/lua/${lua.luaversion}/luv.so"
++ optional (stdenv.isDarwin) "-DLIBLUV_LIBRARY=${lua.pkgs.luv.libluv}/lib/lua/${lua.luaversion}/libluv.dylib"

Messy and strange indeed.

EDIT: I'm not entirely sure why a Darwin build would be trying to install .so files anyway…

@doronbehar
Copy link
Contributor

What I don't get is what's the difference between libluv and luv - both packages seem to come from the same rockspec but it's unclear what's the difference between the two and why does nix-build -A luajit.pkgs.luv doesn't fail while nix-build -A luajit.pkgs.luv.libluv does.

@chkno
Copy link
Member

chkno commented Feb 19, 2020

The difference between luajit.pkgs.luv and luajit.pkgs.luv.libluv:

sdi() { nix show-derivation "$(nix-instantiate -A "$1")" | sed -r 's,/nix/store/[^/]{32},/nix/store/...,g'; }
diff -U999 <(sdi luajit.pkgs.luv) <(sdi luajit.pkgs.luv.libluv)

Cleaned up a little bit to make it easier to read:

 {
     "inputDrvs": {
       ...
+      "/nix/store/...-hook.drv": [
+        "out"
+      ],
     },
     ...
     "env": {
+      "NIX_LDFLAGS": "",
       ...
-      "nativeBuildInputs": "",
+      "nativeBuildInputs": "/nix/store/...-hook",
       ...
-      "preBuild": "sed -i 's,\\(option(WITH_SHARED_LIBUV.*\\)OFF,\\1ON,' CMakeLists.txt\nrm -rf deps/libuv\n",
+      "preBuild": "sed -i 's,\\(option(WITH_SHARED_LIBUV.*\\)OFF,\\1ON,' CMakeLists.txt\nrm -rf deps/libuv\n
+                   sed -i 's,\\(option(BUILD_MODULE.*\\)ON,\\1OFF,' CMakeLists.txt\n
+                   sed -i 's,\\(option(BUILD_SHARED_LIBS.*\\)OFF,\\1ON,' CMakeLists.txt\n
+                   sed -i 's,${INSTALL_INC_DIR},/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9/include/luv,' CMakeLists.txt\n",
       ...
     }
   }
 }

(I'm unable to build neovim on x86_64-linux on master.)

@chkno
Copy link
Member

chkno commented Feb 21, 2020

neovim on x86_64-linux on master works again since staging-merge 9d88ee0.

@bjornfor
Copy link
Contributor Author

And it builds on release-20.03 (tested 3b9b66d) too. Thanks!

@teto
Copy link
Member

teto commented Feb 27, 2020

it builds on release because it doesn't contain d4f09b6, master still seems faulty #81206 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: bug Something is broken 0.kind: regression Something that worked before working no longer 6.topic: lua 6.topic: vim
Projects
None yet
Development

No branches or pull requests

8 participants