Native library bundling and CI job for os dependent wheel distribution

[cansik]

As already mentioned in #157 , this PR adds support for native libraries bundled within the package, as well as a multi-os (Windows / Unix) build CI task for github.

At the moment the build task creates wheels for x64 Windows and Linux operating systems and uploads the resulting wheel files into a new release on github. I guess it would make sense to have the following features implemented as well, but it would make sense if the maintainer would define / approve them first:

• Use tag trigger to create a new release by setting a tag
• Add pypi-publish to the build CI to directly upload the binary files to pypi
• Implement an add-all method to directly load all .so files in the package directory (as on Windows) to not be dependent on version numbers
• Maybe re-implement the possibility to use the system Azure libs on Windows (currently I replaced them by the bundled ones)?

[cansik]

@etiennedub I released the bundled library on PyPI: pyk4a-bundle #157

[ZdenekM]

@cansik Great! Just when using pyk4a-bundle, mypy complains for some reason (with "ordinary" pyk4a, it is ok):

src/python/arcor2_kinect_azure/kinect_azure.py:6: error: Skipping analyzing
"pyk4a": module is installed, but missing library stubs or py.typed marker
    import pyk4a

[ZdenekM]

And when trying to open the sensor, I'm unfortunately getting

Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assertion `needed != NULL' failed!

[cansik]

@ZdenekM Thanks for your reports, I will investigate a bit more. I assume you are working on Ubuntu?

I also noticed that currently numpy 1.22.1 is used to compile the wheel packages, which could be a problem for projects with older numpy libraries installed.

[ZdenekM]

Well, I actually tried it within a docker container based on "python:3.9-slim-buster", but on Ubuntu 20.04, I'm getting the same. My numpy is at version 1.22.1.

[shagren]

@cansik , please add pyk4a/py.typed to package_data. It will solve problem.

[cansik]

I have to check why it is not included, I did not change the options in the setup.py, so it should be included as mentioned here:
https://github.com/cansik/pyk4a/blob/master/setup.cfg#L23-L25

[cansik]

• Numpy version dependency conflict has been resolved by using the lowest possible numpy version per python version.
• @shagren py.typed should be included again

Released version pyk4a-bundle 1.3.0.1

@ZdenekM I am now working on the linux support, this may take some time

[ZdenekM]

Thank you for your effort!

[cansik]

@ZdenekM I tested the package on a Ubuntu 20.04.3 LTS and at least the color stream seems to work, after installing the device rule. There are still problems with the dynamic loading of the depthengine, but I will work on that.

Update
It seems that the auditwheel step only includes the pyk4a and pyk4a-record so lib, but not the depthengine. Manually putting the libdepthengine.so.2.0 into the pyk4a_bundle.libs fixes all issues and the camera works.

I guess the auditwheel step makes copying the libs directly into the module unnecessary for Linux.

[cansik]

After some try-and-error to include the libdepthengine.so.2.0 into the package by auditwheel, I now just use wheel unpack & pack to adjust the wheel after auditing. Definitely not the best way, but it works and the library runs on Linux, also with depth support.

Please try it out by installing version pyk4a-bundle 1.3.0.2

[ZdenekM]

I updated but still getting the same error Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assertion `needed != NULL' failed! (both in docker and on my localhost). The mypy error is gone.

[ZdenekM]

This is the output I got from LD_DEBUG=all python3.9 -c "import pyk4a" - not sure if it could be helpful or not...

pyk4a_ld_debug_all.zip

[ZdenekM]

Maybe this pypa/auditwheel#103 could be related?

[cansik]

@ZdenekM Thanks, which glibc version do you have installed? You can check it by using this command:

ldd --version

[ZdenekM]

ldd (Ubuntu GLIBC 2.31-0ubuntu9.2) 2.31 on my localhost and ldd (Debian GLIBC 2.28-10) 2.28 in the docker image, in which I also tried it.

[cansik]

@rajkundu Thank you for trying it out. I am aware that Unix is still a problem, maybe someone with many-linux pip package experience can help here. I mainly work on MacOS & Windows, so Linux support was more an additional thing.

@lpasselin Interesting, I am not sure what the problem could be there, but I will give it a try later. Thanks for mentioning!

[rajkundu]

Thank you for your help! Do you have any advice in the meantime for me to be able to get the depth engine working on Linux? For me, the SDK works (no import error) when I simply run pip install pyk4a (no ImportError), but it keeps returning None when I try to do transformations, retrieve depth, etc.

I saw this post:

It seems that the auditwheel step only includes the pyk4a and pyk4a-record so lib, but not the depthengine. Manually putting the libdepthengine.so.2.0 into the pyk4a_bundle.libs fixes all issues and the camera works.

How can I replicate this with just regular pyk4a? I mean, what steps did you take on Linux to get pyk4a working with depth?

[lpasselin]

@rajkundu
Does your LD_LIBRARY_PATH contain a directory containing the depthengine.so ?
https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/depthengine.md#linux

[rajkundu]

I installed the SDK using apt install libk4a1.4. Here are the relevant files:

raj@taco:~$ ls -la /usr/lib/x86_64-linux-gnu/ | grep k4a
drwxr-xr-x   2 root root      4096 Feb  7 19:25 libk4a1.4
lrwxrwxrwx   1 root root        19 Jun 15  2020 libk4arecord.so -> libk4arecord.so.1.4
lrwxrwxrwx   1 root root        21 Jun 15  2020 libk4arecord.so.1.4 -> libk4arecord.so.1.4.1
-rw-r--r--   1 root root  13810400 Jun 15  2020 libk4arecord.so.1.4.1
lrwxrwxrwx   1 root root        13 Jun 15  2020 libk4a.so -> libk4a.so.1.4
lrwxrwxrwx   1 root root        15 Jun 15  2020 libk4a.so.1.4 -> libk4a.so.1.4.1
-rw-r--r--   1 root root   6916616 Jun 15  2020 libk4a.so.1.4.1
raj@taco:~$ ls -la /usr/lib/x86_64-linux-gnu/libk4a1.4/
total 3980
drwxr-xr-x   2 root root    4096 Feb  7 19:25 .
drwxr-xr-x 111 root root  126976 Mar  1 06:20 ..
-rw-r--r--   1 root root 3937792 Jun 15  2020 libdepthengine.so.2.0

My $LD_LIBRARY_PATH is /usr/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu/libk4a1.4:/home/raj/torch/install/lib:/usr/lib/x86_64-linux-gnu:/home/raj/torch/install/lib:. I added /usr/lib/x86_64-linux-gnu/libk4a1.4, as that is the path directly containing libdepthengine.so.2.0, but I still get a depth engine error.

This error is:

[2022-03-01 16:16:33.477] [error] [t=359439] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/tewrapper/tewrapper.c (61): transform_engine_start_helper(). Transform engine create and initialize failed with error code: 204.

and here is the relevant GitHub issue. Seems to me like it could be an issue with the SDK, not pyk4a. What do you think?

Finally, I'm running on a headless environment, which I believe is also known to come with its own set of issues due to OpenGL. I'll try the same pyk4a code on Windows and see if that works instead.

[ZdenekM]

Any news on this? :-)

[rosesolow]

@cansik

regular python 3.9 Works! The k4a SDK must be installed otherwise drivers are not there and Windows does not recognize the device. Installing pyk4a-bundle before installing the drivers works as expected.

anaconda python 3.9 and python 3.8 I tested both 3.6 and 3.9 since the DLL loading is different when using <3.8 and >=3.8 Both result in the same "DLL not found" error mentionned in a previous comment.

Any ideas why the anaconda environment is problematic?

I am trying to install pyk4a with anaconda and running into the dll not found issues. Is there a solution to this issue?

[lpasselin]

No solution for Anaconda unfortunately. Someone would need to dig into this. I don't have much knowledge about the internals of conda.

[lpasselin]

Any ideas why the anaconda environment is problematic?

A user found the fix: #167

[lpasselin]

@cansik would you be able to rebase your branch on develop and reset the PR target branch on develop?

[cansik]

@lpasselin I did a rebase of my master to origin/develop and changed the target of this PR. Hopefully this works, otherwise I could also create a new clean branch from develop and a new PR.

[lpasselin]

Thank you for that. Most of the work is done and I am now confident we will be able to merge soon. Let's take our time to properly review the final steps of this PR. To be 100% sure we aren't breaking any regular pyk4a install.

I would like to try installing the package from source on a linux system. Will only have access to my linux machine in 2 weeks. Can anyone try to install cansik's branch on linux? pip install git+https://github.com/cansik/pyk4a.git (might need your typical extra arguments or environement variables)