communication Modelica ↔ Unity

[thorade]

At UdK Berlin (@UdK-VPT, @jraedler, @nytschgeusen) we work on visualization of building simulation results and for this, we are interested in communication between Modelica and Unity. This communication could be sent via UDP, using the MDD UDP sockets. A similar approach was mentioned in section 3.1 and figure 2 of Yamaura et al. 2016.

Yamaura et al did not mention what serialiser/deserialiser was used, and they did not mention how the UDP socket and serialiser/deserialiser were implemented on Unity side. As Unity supports C#, the approach we currently investigate is to use SWIG to generate a C# (and Python) wrapper for these two packages.

Is this SWIG wrapper something that we should contribute back to you in the long term?
Are there any recommendations what we should or should not do to make integration easier?

I have seen issue #112, we are not yet fully sure whether it is the best approach to bring your serialiser to the Unity-CSharp world or whether there are some more widely used serialisers (maybe XDR) that could be integrated in MDD.

Any feedback welcome.

[bernhard-thiele]

Since you interested in a communication between Modelica and Unity for visualization, this issue might also be interesting for @tbellmann, who had thoughts whether one should work at a dedicated visualization interface for Modelica (he is one of the authors of the DLR Visualization library, http://www.dlr.de/rmc/rm/Portaldata/52/Resources/Software/MotSimVR/DLR_Visualization_Library.pdf). There are also visualization efforts within the OpenModelica project (@vwaurich, @sjoelund), so maybe the Modelica community could benefit from the definition and development of a common interface that can be used (and extended) by different visualization solutions.

The serial packager in MDD was developed having a rather simple communication use-case in mind: periodic sending or receiving of fixed size packages which map to fixed size input and output signals in the Modelica model. This fits well for certain application.

There have been discussions to integrate more flexible approaches. However, there are currently no concrete develop efforts in this direction. Having an additional serialiser, which can be used optionally (instead of the current one), would be fine. I expect that there is not a single solution which is perfect for all use-cases, so it would be nice if we could provide some choice.

The initial idea was that one should be able to use different serializers with the same communication block. However, if one reuses the interface of the communication blocks (Blocks.Interfaces.PackageIn and PackageOut) the design of an alternative serialiser is subject to the constraints of these interfaces. This constrains the design space severely. To escape that constraints, one might need clones of the communication blocks with a different interface. This is not nice, but possibly a necessary trade-off. On the other hand, if one only uses functions (forgets about the things below subpackage Blocks), there are less constraints for adding a new serialiser.

Whatever you decide for, it would be great if you could bring your solution to the attention of the MDD project. Maybe we can integrate it directly into MDD, or maybe we can simply reference to some project of yours (e.g., the SWIG wrappers could be an own project that we reference in the documentation, or they could be shipped with the MDD library). We would need to figure out what works best, once there is something available.

In any case, thanks for the information and for considering to contribute back to MDD.

[thorade]

Thanks for that comprehensive reply & new input.

For myself, a short list of references:

• Modelica 3D annotations: Engelson 2000
• MultiBody Library: Otter, Elmqvist, Mattsson 2003
• X3D Modelica annotations: Hoeft, Nytsch-Geusen 2008
• DLR ExternalDevices Library: Bellmann 2009
• Modelica3D: Höger, Mehlhase, Nytsch-Geusen et al 2012
• Modelica Synchronous: Otter, Thiele, Elmqvist 2012
• DLR Visualization Library: Hellerer, Bellmann, Schlegel 2014
• ADAS Unity: Yamaura et al. 2016
• Modelica DeviceDrivers Library: Thiele, Beutlich, Waurich, Sjölund, Bellmann 2017

And some webpages:

• https://github.com/DLR-SR
• https://dlr-sr.github.io/
• https://youtu.be/QCGcWOwV8Qs
• http://www.bausch-gall.de/vi1
• https://github.com/OpenModelica/modelica3d
• https://en.wikipedia.org/wiki/X3D
• http://www.openscenegraph.org/
• https://unity3d.com/
• https://www.oculus.com/
• https://www.vive.com/

So, if I understand correclty, both the ModelicaDeviceDrivers Library and the DLR Visualization Library are based on the DLR ExternalDevices Library, but only the DeviceDrivers is available under an open license.

[bernhard-thiele]

Yes, initially the ExternalDevices library included both aspects (visualization and device drivers). These two aspects were separated, leading to two internal DLR libraries. Later, one of these libraries evolved to the commercial DLR Visualization library, the other to the open-source MDD library (first open-source release was in August, 2012).

[thorade]

Our code is here:
https://github.com/jraedler/Modelica_DeviceDrivers/tree/swig
It works for us, so we can exchange data to and from Modelica, Python and CSharp.
Before sending a Pull Request, we should probably do some cleanup.

[bernhard-thiele]

Very nice :-)