NetVips Native Linux on Ubuntu 18.04 and Unity - cannot register existing type 'gchar' #90
Comments
|
Just to follow up on this, I decided to try to use standard packages for libvips (I believe it installed 8.4.5-1build1)
I then modified Unity's config: As you can see, I mapped This seems to work! - although a few undefined symbols, but no crash. (This now makes me wonder on a correct packaging strategy for our app). |
|
Hello @jsm174, Sorry for the delay, I was quite busy with other projects. I could not reproduce this on my Fedora machine with Unity 2020.1.2. I followed this tutorial to include the Unity project as a package. After that, I had to fix the With the above commit, the example project could be built and executed. Do I need to import a VPX file to reproduce it? If so, is there a sample file I can use? |
|
@kleisauke, no worries. Thank you for looking into this! Correct, you will need to import a VPX file before any NetVips calls are executed. You should be able to grab the VPX table here: |
|
One other question, on your PR. (had something similar, but didn't submit since it wasn't fully working). Just curious, did Unity's Editor UI allow you to modify the |
|
I'm able to import the above linked VPX file (using the
If you want, I can also test this on Ubuntu 18.04 running in a VM.
I haven't submitted a PR either. Feel free to use these changes!
I manually modified these |
|
A PR would be wonderful! Thanks guys for looking into this. |
|
Thanks. Just to follow up Unity does have it's own mono installation. Two questions
I'm going to try again in Unity 2020.1.2.. |
|
So I just had a chance to test Unity 2020.1.2f1 in Ubuntu 18.04. Using your branch, without any modifications to Here is what I see:
|
|
I somehow missed those errors in the console, sorry. I also see this crash with commit kleisauke/VisualPinball.Engine@b19c2c2 (which removes the need for those DllMap configuration files). Unfortunately, there is not much that NetVips can do for this. This crash happens because the GType register is in a flat global namespace. It can conflict when another GLib instance (running in the same process) registers a GObject type (for e.g.
Making the importer self-contained (1) might be the best option if you want to use the pre-built libvips binaries. Python for Unity has a out-of-process API that could perhaps be used, I'm not sure if there is a similar API available for C#. Alternatively, you could parse the VPX files with an intermediate step. I noticed that NetVips is only used to calculate the opaque/translucent/transparent stats, perhaps you could write that to a JSON file and have Unity import that? The advantage of this approach is that you can easily perform this step on any platform. For example, I'm currently working on WebAssembly bindings for libvips, so that you can run it in the browser without the hassle of installing pre-built binaries, see: Install libvips via the package manager is another option (2). The only downside is that on some distributions the libvips package is quite old. For example, Debian's policy is never to update packages (except for security fixes and critical bugs) because they prioritize stability. If you want a system with up-to-date binaries, you need to pick a different Linux or compile libvips manually (3). |
|
Do we know what registers GType in a flat global namespace? Is it libvips itself, hence solution 3) to recompile it in a separate namespace? |
|
With a fresh project in 2020.1.2f1, (vpe package NOT installed), I looked at open files for the Unity process: fwiw:
Also, I did the same test on the Mac: |
|
@kleisauke Hello again. I had a chance to look at this some more. I decided to make a clone of NetVips for Unity: Basically we
This allowed us to not have to make any code changes to anything inside of root NetVips directory. It also gets us around dllmap issues. (we really don't want people having to edit their Unity So now we should be able to include native dlls that are compatible with Unity. Currently, we are using your latest native dlls (https://github.com/jsm174/net-vips-unity/tree/master/deps/libvips) Our current observations are:
|
Solution 3) is similar to 2) by ensuring that the system-wide GLib dependency is used to avoid conflicts with different GLib's.
That's interesting! This is probably because Unity decided to not link against Homebrew's GLib. Instead, they seem to provide their own statically linked GLib (which is the same strategy as NetVips). This might be the reason why it works on macOS out-of-the box.
I agree that editing the Unity configuration file should not be necessary, but I'm not sure a "hard-fork" for NetVips is desired here. My opinion is that this check has to be done at compile-time, and therefore different DLLs needs to be deployed on different operating systems (i.e. deploying
That's curious, I've not encountered this error on Travis' macOS runner using the pre-built binaries, see: Perhaps you need to set Lines 17 to 21 in 656691e
|
Unity does not use I did end up working on the Everything is compiling and testing successfully for all platforms.
The final hurdle now is using the Testing it locally, I'm seeing: I'll keep working on it... |
Ah, I didn't know that. If it recompiles every This commit basically removes all dependencies of NetVips and merges the net-vips/src/NetVips/Interop/Libraries.cs Lines 5 to 34 in e13972e Feel free to use these changes (it could perhaps be useful as a git submodule). Note that you must call
It looks like |
|
@kleisauke - Thank you again for all your help! Just so I'm understanding, if we were to use the preprocessing directives approach, we would take your Unity branch as a submodule and Unity would build its own NetVips? (hence why you removed the dependencies?) I did take all your changes from the (Having the dependencies removed helped out. Otherwise I would have had to repack with ILRepack) I then added a OSX: Linux: |
|
You should be careful with distributing libvips that links against system libraries. It usually works when used on the same distro on which it is compiled, but it can easily break if used on other distros (or even older versions of the same distro). That's why NetVips (and sharp) distributes libvips with every dependency statically-linked. This also ensures that newer dependencies can easily be incorporated (for e.g. libspng). For Ubuntu a PPA would be more appropriate, see libvips/libvips#987. For macOS, you could use libvips from Homebrew or use the
Correct. This to avoid having to maintain a separate fork.
As long as you keep this package unlisted, I'm fine with it.
Note that removing the
It makes more sense to move this to the constructor to prevent libvips from being initialized several times (although libvips has protection against
After having taken a closer look, it's because you specified these libraries in your P/Invoke methods. Try changing these libraries to |
Yes, this crossed my mind several times over the weekend 😃 . I just wanted to get something working before I could start testing other scenarios. If I end up using the libs here: https://github.com/lovell/sharp-libvips/releases/tag/v8.10.0 (which are your NetVips.Native ones), aren't I still going to have the gchar issues that started this whole thread?
I'm going to go put these dependencies back in as we are now using the latest
Yes, this was just a quick test for me to see if it would work. Again, I truly appreciate the help and patience! |
|
Updated NetVips.Unity to 0.0.4.
All pipelines pass: (https://github.com/jsm174/net-vips-unity/actions/runs/222260220) Updated Visual Pinball Engine to use new NetVips.Unity to 0.0.4 (https://github.com/jsm174/VisualPinball.Engine/tree/net-vips-unity-2) All pipelines pass: (https://github.com/jsm174/VisualPinball.Engine/runs/1021784269) Running in actual Unity:
So for now, I can only see a custom compiled |
|
After some investigation, I think I have found a solution for this issue! See commit kleisauke/VisualPinball.Engine@47ccb5b. Could you try the I also simplified the implementation of including different DLLs for different OSes, see: |
kleisauke
added
blocked-upstream-dependency
|
@kleisauke - Wow. I just gave this a try on my Ubuntu install and it worked! I'll test the other OS's too and report back! I'll need to look at your code because I'm curious about the how NetVips is picking up the correct dll filenames now. EDIT: So I'm looking at the code and I'm seeing unique Also, I like what you did with the |
|
Great! Thanks for testing, I'm also glad it works now, phew! 😅
I'll publish the new NetVips.Native.* NuGet packages somewhere this week.
You can simply do this to ensure that the proper NetVips DLLs are deployed in the corresponding RID directories. cd VisualPinball.Engine
dotnet build -r win-x64
dotnet build -r win-x86
dotnet build -r osx-x64
dotnet build -r linux-x64
The only thing that might not be ideal is that the (quite large) libvips binaries are checked into version control. The entire plugin directory should probably be placed in |
Fantastic!! So I'm assuming NetVips.Native.* will now start containing the different
Perfect. Never even thought about that!
Agreed, and hopefully that will change in the future. Currently as the project is gaining more interest, table designers are grabbing the latest from the repo and adding directly to Unity - so the DLLs need to be there. Only the developers are using Visual Studio / Rider - which creates and refreshes that directory. One last observation, and this may or may not be an issue.. For Windows, in Unity we have to enable the
We don't need to do that for any other platform, or even for |
The NetVips NuGet package contains the DLLs for different runtimes (within the I cannot merge this into one package because users sometimes prefer to use a system-wide installed libvips (for e.g. to support more image formats).
I could reproduce this on my Windows PC. Could you reproduce this with your Perhaps adding |
|
Apologies, I thought I looked through the nuget before, but I see them now:
And they already have the correct library names...
Yes, this does also happen with |
kleisauke
removed
the
blocked-upstream-dependency
|
NetVips.Native v8.10.0 is now available. 🎉 I've updated the |
|
Awesome. I honestly can't thank you enough. I will work on getting this back into VPE! |
|
@kleisauke Cheers man, you're saving the day again :) Any objections if I merge your branch upstream and add you to the contributors list? |
|
Sorry, but I just noticed one last thing when going through builds. I was trying to run this on Mac and Linux: In the
I was trying to switch our github workflows from Should I open a new issue? |
I have no objections.
It's only necessary on Mono to copy the files to the output directory. If you switch the |
|
I thought I had issues running NUnit tests inside of VS Mac, which was why the test projects were at Thanks again! |
|
This has been fixed within VPE with freezy/VisualPinball.Engine#170, I'll close this issue. fwiw, the above mentioned WebAssembly example can be viewed here: |
|
@kleisauke can I ask a last favor to give your green light to relicense VPE to GPLv3 at freezy/VisualPinball.Engine#159? You're contributor now and we need everybody's authorization. Sorry that it bit you back so quickly ;) |
Hello.
I'm helping on the Visual Pinball Engine project getting it to run on Mac and Linux. You helped us a few weeks ago and added support for mono, and we were successfully able to get it to work on OSX.
We are currently trying to get this running on Ubuntu 18.04.
If we do a base install of Ubuntu 18.04 and then add Mono 6 (
Mono JIT compiler version 6.10.0.104) and run our test cases, we can seelibvipsis working.NetVips.dll.config:Launch test cases:
MONO_LOG_LEVEL=debug MONO_LOG_MASK=dll mono ~/.nuget/packages/nunit.consolerunner/3.11.1/tools/nunit3-console.exe VisualPinball.Engine.Test/.bin/Release/net472/VisualPinball.Engine.Test.dllConsole:
In Unity, when
libvipstries to load it immediately crashes.Since Unity doesn't seem to load individual dll maps, we updated its config (just like we did for OSX)
~/Unity/Hub/Editor/2020.1.0b16/Editor/Data/MonoBleedingEdge/etc/mono/config:Launch Unity:
MONO_LOG_LEVEL=debug MONO_LOG_MASK=dll /home/jsm174/Unity/Hub/Editor/2020.1.0b16/Editor/Unity -projectpath "/home/jsm174/New Unity Project (5)" -hubIPC -cloudEnvironment productionUnity Editor Log:
I know the dllmap is working, because if I omit the entries:
I also know that other native libraries are loading successfully:
Is it possible, Unity's has its own
libglibandlibgobjectthat are conflicting? The mono config file has these entries:The text was updated successfully, but these errors were encountered: