NetMQ + Unity3D, am I wasting my time?

[Sohojoe]

I have been trying to implement a basic REQ/REP model with Unity3D (Mac OS X) acting as the server and python as the client. It works OK as a standalone Unity program. However Unity will soft-lock after 10-40 frames if i run with the Mono Develop debugger attached (requiring a Force Quit)

I have read through the numerous open and closed issues with regards to problems with NetMQ and Unity3D;

1. Is there a fundamental problem with Unity3D + NetMQ (i.e. I wasting my time with NetMQ and should look for an alternative)

2. or, is there a localized problem only when attaching Mono Develop debugger to Unity3D + NetMQ (i.e. NetMQ is good for production but I should look for another solution for debugging)

3. or, it's something hookey with what I'm trying to do (so either we debug that or I change my architecture)

Things I have tried

• building NetMQ and AsyncIO from master
• adding AsyncIO.ForceDotNet.Force ();
• adding NetMQConfig.ManualTerminationTakeOver();
• adding NetMQConfig.ContextCreate(true);
• using RouterSocket instead of ResponseSocket
• running server code on a seperate thread (per one of the example in the Issues

Environment

NetMQ Version:

• NetMQ 4.0.0-rc5
• AsyncIO 0.1.26
  (Note: I started with the Nuget version of NetMQ and AsyncIO; then I built locally from git.Master)

Operating System:

• MacOS 10.12.1

.NET Version:

• Unity 5.4.3f1
• Python 2.7 (for the client)

Expected behaviour

Actual behaviour

Steps to reproduce the behaviour

[Supergeek]

I have a lot of the same concerns. I'm just learning NetMQ/ZMQ and it feels very fragile on Unity. Some guidelines and best practices to avoid editor freezes and crashes would be very helpful.

[dogagithub]

Same problem to me.Unity will stuck.

[flisky1]

Try closing all sockets and adding NetMQConfig.ContextTerminate(); to the OnDestroy() event of the MonoBehaviour. This fixed all of my freezing issues. I am using AsyncIO.ForceDotNet.Force(); NetMQConfig.ManualTerminationTakeOver(); NetMQConfig.ContextCreate(true); and a single RequestSocket for my development scenario under Unity 5.4.1.

[tobi-tobsen]

Most of the issues of using netmq with Unity is stemming from not properly initializing it (using AsyncIO.ForceDotNet.Force(); ) and not properly shutting down / disposing the netmq sockets and context (e.g. in OnDisable() or OnApplicationQuit() methods of the MonoBehaviour).

So, in my opinion you are not wasting your time and people are successfully using netmq with unity. You could try what others have been using.

Are there any additional things which could help you getting started with unity? Maybe post your solution or add documentation to help others?

[Sohojoe]

@tobi-tobsen, @flisky1

• I have created a sample project that only has AsyncIO, NetMQ.
• I created a Req/Rep example based on @tobi-tobsen's link
• It runs OK when running from Unity
• When I attach MonoDebug, and run from Unity, it locks after 5 or 6 cycles
• This is consistent with what I've seen in every test I have done
• Can you check if you get the same problem (if so, are you on Mac or PC?)

Code - https://github.com/Sohojoe/UnityMQ

[tobi-tobsen]

@Sohojoe I quickly skimmed through the code and as far as I can tell it should work that way.
Since I have neither unity nor a mac at my hands, I cannot reproduce the behaviour.
I am not convinced that this is related to netmq but may be related to unity + monodebug on mac since others are still having issues debugging (and here) without netmq involved.

[Sohojoe]

@tobi-tobsen I didn't get the sense that this problem was related to those other MonoDev issues. I all I do is click run which the debugger attached; I'm not stepping through code or actually 'debugging'

These are my ideas to move this forward, do you have other ideas?

• Step through code to find where it locks
• Check if bug presents on windows
• Submit bug to Unity (I think we should at least step through the code first)

I will try and get some more time on this over the holidays

[Sohojoe]

1. I have Reproduced on Windows (when debugging with VisualStudio)
2. debugged and I can confirmed code locks within NetMQ
3. I have submitted a bug report to Unity

I also implemented the ReliablePubSub patterns; which also soft locks when using the debugger.

[Mystfit]

@Sohojoe The included UnityMQ example locks up on second run, which I believe happens when you call NetMQConfig.ManualTerminationTakeOver().

This is behaviour that I experience when I have more than two sockets active. When I call close on the sockets, the actual low level socket object never receives a shutdown message, and so the context will never terminate as it awaits a successful shutdown message from each socket. Since you have the cleanup code at the start, it doesn't lock when you stop the editor (which triggers OnApplicationQuit()), but it will lock when the old context has yet to close due to the frozen sockets.

Tested using Unity 5.5.0f3 on WIndows 10.

[valkjsaaa]

I just went a long way to figure this problem out.

Check this

The key is to have AsyncIO.ForceDotNet.Force(); before you start and NetMQConfig.Cleanup(); before you end.

[erdalpekel]

@Sohojoe Can you please tell me how you actually installed netmq on Mac OS X for use with Unity.
And how you included it in the project.

Thanks

[Sohojoe]

@erdalpekel I had created a repro. I gave up on NetMQ for the problem I was hoping to solve, but I did update the repro with Unity 2017.1 / .Net4.6 - I think it is working now, but I didn't do deep testing

https://github.com/Sohojoe/UnityMQ

[overthrowrobotics]

I need to publish from Unity -> Python and also publish from Python -> Unity. I used @valkjsaaa code and it worked great for the Unity client and I can publish from Python. I haven't been able to figure out how to publish from Unity though, the server example is for req/res, not pub/sub.

@Sohojoe I tried your code and the sample where it publishes and subscribes in a single script works fine. When I use my python script to try to listen to that IP and port it doesn't work. Is reliable server/client somehow different than pub/sub?

[Sohojoe]

@overthrowrobotics I gave up NetMQ and went with a mem cache solution (Redis) which is complex but well suited to my use case (Reinforcement Learning where I need to sync at up 1000 per seconds)

What are you trying to do? There are a few different approaches. If you are not too worried about latency (and dont need to transfer too much data) then PubNub is super easy to set up. If you need something faster then it maybe worth looking at Unity's ML which uses sockets to comunicate between Unity and Python (it came out after I had figure out my path so I have not tried it but I belive it is open source). Initially I used HTTP / Rest which is pretty simple but not great for performance.

[overthrowrobotics]

Building a boxing robot. Here's an older version of it. https://www.youtube.com/watch?v=Bwv_wJbEbVE

I have 5 Teensy (32-bit arduino clone) that I'm using for various encoders, pressure sensors, IMUs, LEDs, dollar bill acceptor, etc. Standard System.IO.Ports sucks. Super unreliable and I've tried 5 other 3rd party serial libraries and they are all problematic. Pyserial works pretty well.

I managed to get mqtt/paho/mosquitto working last night. I'm only doing about 20hz with small <20 byte messages so I think it's going to be ok.

I was thinking about the Unity ML but I couldn't find any documentation on how to hijack it's sockets.

I use SARSA (similar to Q-Learning) in the robot through Aforge.net library since it was long before they released their ML stuff. It's not deep RL but for basic stuff it works. The problem is that there are a lot of new RL algorithms (even outside of deep) like contextual bandits that might work well and Aforge isn't being updated.