Linking Failure Flann LZ4

[SergioRAgostinho]

Windows builds on Azure fail (e.g. this one) with the following error:

Creating library D:/a/build/lib/pcl_kdtree.lib and object D:/a/build/lib/pcl_kdtree.exp
kdtree_flann.obj : error LNK2001: unresolved external symbol __imp__LZ4_resetStreamHC [D:\a\build\kdtree\pcl_kdtree.vcxproj]
kdtree_flann.obj : error LNK2001: unresolved external symbol __imp__LZ4_setStreamDecode [D:\a\build\kdtree\pcl_kdtree.vcxproj]
kdtree_flann.obj : error LNK2001: unresolved external symbol __imp__LZ4_decompress_safe_continue [D:\a\build\kdtree\pcl_kdtree.vcxproj]
kdtree_flann.obj : error LNK2001: unresolved external symbol __imp__LZ4_decompress_safe [D:\a\build\kdtree\pcl_kdtree.vcxproj]
kdtree_flann.obj : error LNK2001: unresolved external symbol __imp__LZ4_compress_HC_continue [D:\a\build\kdtree\pcl_kdtree.vcxproj]
D:\a\build\bin\pcl_kdtree.dll : fatal error LNK1120: 5 unresolved externals [D:\a\build\kdtree\pcl_kdtree.vcxproj]

It is is very likely that this is the cause flann-lib/flann#399

[jasjuang]

That PR for flann also breaks Ubuntu build for me. If I build flann with that PR and compile with the latest pcl below is the error I got

../lib/libpcl_kdtree.so.1.9.1.99: undefined reference to `LZ4_resetStreamHC'
../lib/libpcl_kdtree.so.1.9.1.99: undefined reference to `LZ4_setStreamDecode'
../lib/libpcl_kdtree.so.1.9.1.99: undefined reference to `LZ4_decompress_safe'
../lib/libpcl_kdtree.so.1.9.1.99: undefined reference to `LZ4_decompress_safe_continue'
../lib/libpcl_kdtree.so.1.9.1.99: undefined reference to `LZ4_compress_HC_continue'

[taketwo]

Same errors were also reported in #804.

[SergioRAgostinho]

Useful info microsoft/vcpkg#6294

[SergioRAgostinho]

I dug a little more into this topic. Here's what I found:

1. Flann 1.9.1 still ships its own lz4. Until there's a new official version, there's no point in caring about this because the development of that lib is practically stale.
2. The windows CI started having issues because the vcpkg team decided to use the development version instead of waiting for an official release. Installation of pcl from within vcpkg is patched to work with their changes, i.e. they strip out our custom FindFLANN.cmake module because the flann development version already exports a configuration file.
3. When compiling the flann (dev) target withing vcpkg, finding lz4 is now required. There's no version increment we can look for in the flann config file they export do distinguish between the official version and their custom one, which means we cannot patch with that in mind. Users who want to compile PCL from source, using vcpkg for the dependencies are pretty much in a tough spot at this point.

This is vcpkg specific problem I believe we have no way of fixing things here. It needs to be done upstream.

Reference microsoft/vcpkg@69e86fd#diff-65789af0d8e619c9aa1c332d420d5760

[taketwo]

Useful info microsoft/vcpkg#6294

Thanks for digging this out. It would be nice if we got a heads up from the vcpkg people when this issue was discovered. I guess I'll post a kind message there to ask them for the future.

Users who want to compile PCL from source, using vcpkg for the dependencies are pretty much in a tough spot at this point.

... and unfortunately our CI is one such user.

they strip out our custom FindFLANN.cmake module because the flann development version already exports a configuration file.

Well, we've got a lead. Availability of a config file can help us to distinguish release from development version. In our custom finder script we may first do find_package(flann CONFIG). If this succeeds, then we are basically done. (Just need to create an alias to match our FLANN target name with the official one.)

[jspricke]

Would a new Flann version help? We could ask @mariusmuja to tag one.

[SergioRAgostinho]

The current master needs a couple more patches. I'm currently unable to compile it on OSX, for instance. I would avoid tagging given the current state.

[taketwo]

Would a new Flann version help?

While tagging new versions from time to time is generally good, I don't think it's going to help in this particular case. Even if a new version is tagged, we will still need to support both old manual "module-based" and new "config-based" lookup.

[taketwo]

Fixed by #3202.

[corot]

I have exactly the same error reported by @jasjuang compiling latest master on ubuntu 20.04 (didn't happen on 18.04):

[ 45%] Building CXX object tools/CMakeFiles/pcl_vtk2obj.dir/vtk2obj.cpp.o
/usr/bin/ld: ../lib/libpcl_kdtree.so.1.12.0.99: undefined reference to `LZ4_resetStreamHC'
/usr/bin/ld: ../lib/libpcl_kdtree.so.1.12.0.99: undefined reference to `LZ4_setStreamDecode'
/usr/bin/ld: ../lib/libpcl_kdtree.so.1.12.0.99: undefined reference to `LZ4_decompress_safe'
/usr/bin/ld: ../lib/libpcl_kdtree.so.1.12.0.99: undefined reference to `LZ4_decompress_safe_continue'
/usr/bin/ld: ../lib/libpcl_kdtree.so.1.12.0.99: undefined reference to `LZ4_compress_HC_continue'
collect2: error: ld returned 1 exit status
make[2]: *** [tools/CMakeFiles/pcl_compute_cloud_error.dir/build.make:117: bin/pcl_compute_cloud_error] Error 1
make[1]: *** [CMakeFiles/Makefile2:3131: tools/CMakeFiles/pcl_compute_cloud_error.dir/all] Error 2
make[1]: *** Waiting for unfinished jo

I double checked that I have the #3202 fix. flann version is:

libflann-dev is already the newest version (1.9.1+dfsg-9build1).

Disabling VTK (not a big problem, as I don't even know what it is!) just move the error to the next tool:

[ 45%] Linking CXX executable ../bin/pcl_compute_hausdorff
/usr/bin/ld: ../lib/libpcl_kdtree.so.1.12.0.99: undefined reference to `LZ4_resetStreamHC'
/usr/bin/ld: ../lib/libpcl_kdtree.so.1.12.0.99: undefined reference to `LZ4_setStreamDecode'
/usr/bin/ld: ../lib/libpcl_kdtree.so.1.12.0.99: undefined reference to `LZ4_decompress_safe'
/usr/bin/ld: ../lib/libpcl_kdtree.so.1.12.0.99: undefined reference to `LZ4_decompress_safe_continue'
/usr/bin/ld: ../lib/libpcl_kdtree.so.1.12.0.99: undefined reference to `LZ4_compress_HC_continue'
collect2: error: ld returned 1 exit status
make[2]: *** [tools/CMakeFiles/pcl_compute_hausdorff.dir/build.make:100: bin/pcl_compute_hausdorff] Error 1
make[1]: *** [CMakeFiles/Makefile2:3194: tools/CMakeFiles/pcl_compute_hausdorff.dir/all] Error 2

Can u reopen the issue, @taketwo? or should I open a new one?

[jspricke]

Hi Jorge,

which flann version do you use?

[corot]

he he ,, just noticed that info was missing
libflann-dev is already the newest version (1.9.1+dfsg-9build1).

[corot]

thanks for reopening!
better change the title: windows -> ubuntu 20.04

[jspricke]

I can't reproduce this on Debian unstable, will setup a Ubuntu 20.04 VM later.

[jspricke]

Can't reproduce this in Ubuntu 20.04 either, can you try a build on a fresh system? ;).

[kunaltyagi]

@corot Do you have a reproduction in a dockerfile? I was also unable to reproduce it on my 20.04 and in the docker image (The CI is green as well)

[corot]

shame on me..... 😞 worked after deleting build... sorry....

[kdheejb7]

Hello, @corot

I have exactly the same problem now...
How did you solve this problem?

[kunaltyagi]

worked after deleting build.

Most likely performed a fresh build

[kdheejb7]

Thanks for the quick response!
I did a fresh build but not work ... :(
All versions(flann, pcl, and vtk) are same, and I did re-building flann, pcl, and vtk all, but cannot solve it

[kunaltyagi]

Please use discord for such questions. Live chat is easier that way.

[jackyko1991]

same issue here

fresh build even custom lz4 build on flann does not help. possibly flann problem using older version of lz4?