Node Editor in Android

[zardchim]

Hi everyone :)

I am trying to build a Android application with Node Editor features to perform basic logical application (e.g. if / else)

However since Smartphone only allows pressing operation, I am trying to figure out how to add node (e.g. AllRoundNode) by pressing a button on SideGui() as demonstrated in RunTimeNodeEditor.

Could anyone please guide me a way to achieve such operation? I am a bit lost after reading the "ContextCallback" function ....

[pmhpereira]

ContextCallback receives a NodeEditorMenuCallback input variable to decide what to do.
NodeEditorMenuCallback has 4 fields: message, canvas, editor and contextClickPos.

message is a string representation of the command (deleteNode, duplicateNode, startTransition) or the ID of the node to add (allaroundNode for your case).

The next two fields are the canvas and editor where the operation will be performed. If you only have one canvas/editor, then you can simple pass NodeEditor.curNodeCanvas and NodeEditor.curEditorState.
The last field contextClickPos is the position in your screen where the node will be added/changed.

To summarize, you can just call:

NodeEditorMenuCallback cb = new NodeEditorMenuCallback("allaroundNode", NodeEditor.curNodeCanvas, NodeEditor.curEditorState);
cb.contextClickPos = new Vector2(200, 200);
NodeEditor.ContextCallback(cb);

[Seneral]

Sorry I missed this issue for some reason.
Afaik touchscreen controls do already work, atleast they do on my tablet. Context click should be triggered through holding the touch for a sec, and then the context menu should pop up as normal...
If that does not work for some reason, what pmhpereira says is correct so you could mimic that functionality with buttons on the side panels.

[pmhpereira]

I tested on my phone and the context menu only shows when touching with two fingers, but you can't select an option.

[Seneral]

Hm, I thought the touch behaves the same on every device. Maybe it's a difference because my tablet has windows 8 not android. I'd first try to emit a right click event on android when the click lasts for more than 0.5 secs or so if that is possible, it wouldn'T require you to rewrite the wholecode much. This would be done in the Update function I guess...

[zardchim]

Hello pmhpereira and Seneral ! Thank you for the advise!
I also tried it on my Android Phone, but with the same situation of only shows the context menu with two fingers touching.

I have used "#If UNITY_ANDROID" inside the InputEvents() with touch phase function, the menu will pop after pressing for over 1 second, please see below. However, I am thinking if in mobile application, the contextMenu should still exist after moving out your finger unless further touching by the user

`#if UNITY_ANDROID
foreach (Touch touch in Input.touches)
{
switch (touch.phase)
{
case TouchPhase.Began:
previous_touch_time = Time.time;
break;

                case TouchPhase.Stationary:
                    curEditorState.dragNode = false;
                    curEditorState.panWindow = false;

                    if (curEditorState.focusedNode != null)
                    { // Clicked a Node
                        if ((Time.time - previous_touch_time) > 1)
                        { // Detect click on a connection knob
                            if (!CanvasGUIToScreenRect(curEditorState.focusedNode.rect).Contains(touch.position))
                            { // Clicked NodeEdge, check Node Inputs and Outputs
                                NodeOutput nodeOutput = curEditorState.focusedNode.GetOutputAtPos(touch.position);
                                if (nodeOutput != null)
                                { // Output clicked -> New Connection drawn from this
                                    curEditorState.connectOutput = nodeOutput;
                                    return;
                                }

                                NodeInput nodeInput = curEditorState.focusedNode.GetInputAtPos(touch.position);
                                if (nodeInput != null && nodeInput.connection != null)
                                { // Input clicked -> Loose and edit Connection
                                  // TODO: Draw input from NodeInput
                                    curEditorState.connectOutput = nodeInput.connection;
                                    nodeInput.RemoveConnection();
                                }
                            }
                        }
                    }
                    if ((Time.time - previous_touch_time) > 1)
                    {
                        // Editor Context Click
                        GenericMenu menu = new GenericMenu();
                        if (curEditorState.connectOutput != null)
                        { // A connection is drawn, so provide a context menu with apropriate nodes to auto-connect
                            foreach (Node node in NodeTypes.nodes.Keys)
                            { // Iterate through all nodes and check for compability
                                foreach (NodeInput input in node.Inputs)
                                {
                                    if (input.type == curEditorState.connectOutput.type)
                                    {
                                        menu.AddItem(new GUIContent("Add " + NodeTypes.nodes[node].adress), false, ContextCallback, new NodeEditorMenuCallback(node.GetID, curNodeCanvas, curEditorState));
                                        break;
                                    }
                                }
                            }
                        }
                        else if (curEditorState.makeTransition != null && curEditorState.makeTransition.AcceptsTranstitions)
                        { // A transition is drawn, so provide a context menu with nodes to auto-connect
                            foreach (Node node in NodeTypes.nodes.Keys)
                            { // Iterate through all nodes and check for compability
                                if (node.AcceptsTranstitions)
                                    menu.AddItem(new GUIContent("Add " + NodeTypes.nodes[node].adress), false, ContextCallback, new NodeEditorMenuCallback(node.GetID, curNodeCanvas, curEditorState));
                            }
                        }
                        else
                        { // Ordinary context click, add all nodes to add
                            foreach (Node node in NodeTypes.nodes.Keys)
                                menu.AddItem(new GUIContent("Add " + NodeTypes.nodes[node].adress), false, ContextCallback, new NodeEditorMenuCallback(node.GetID, curNodeCanvas, curEditorState));
                        }
                        menu.ShowAsContext();
                    }
                    break;

                case TouchPhase.Moved:
                    break;

                case TouchPhase.Ended:
                    break;
            }

#endif
`

[pmhpereira]

I'm not sure if @Seneral ever thought of supporting this plugin in mobile devices (he said it works for Windows tablet, but possibly because it automatically emulates finger touches as mouse clicks, like a native Windows computer).

If he/we were to support this, I think the best approach would be to have a separate class managing the input state (one implementation for computers and one for mobile devices) and have the remaining code querying it.

@Seneral, your thoughts?

[Seneral]

I did not actively wanted to support this but if @zardchim needs this, we can resolve this together. So, it seems only context clicks are not supported so far. I can take a look at how I can emulate it.

[zardchim]

@Seneral Thank you very much for the help!

Currently I am looking at a simpler way to connect node together on smartphone (e.g. by simply touching on the connection knobs consequently, a line will be automatically formed in between).
Besides, I hope to use such node editing feature to perform programming flow (e.g. if / else logical operation with input and output), thus to control an Arduino boad.

I really appreciate if you can help me, but please also advise direction / functions to me so I can work on together :) !

[pmhpereira]

@zardchim, you should start by checking #37. It's currently being implemented and it is a functionality that will allow Node_Editor to call methods from any gameobject in the scene.

Building logical nodes is somewhat easier and you can start doing it yourself. It's just a matter of creating a new BoolType (currently there is a FloatType only) and implement the nodes you need (just check the default ones), which compare two inputs and output the result of the logical operation.

[Seneral]

I've no experience with an Arduino board or similar, but there's some things to consider regarding the conditional operations.
The calculation system is only there to calculate/evaluate objects, it will visit every branch if possible and is not able to apply if/else statements in a sense that you go along a branch, if you understand what I mean. For instance you can create a bool type like @pmhpereira said but it will not help you perform that conditional operations, but only to calculate that bool value.
For this the transitioning system is built, it will allow you to move along paths based on conditions (when it's finished). Another thing I consider to add to the calculation system is something like output blocking so you can apply conditional oeprations without the transitioning system.

Before going any further though, do you have the Arduino API accessable from withing Unity? Then it'll be a first step to expose it to the user through nodes, atleast for the most common ones. It's not adviseable to always rely on the action node as it has alot of overhead compared to a normal node (when it's even implemented).
After this step I can help you with the logical programming flow part with the conditional operations @zardchim:)

[zardchim]

@Seneral Thank you very much for the advise!

For the Arduino API, I used a simple serial interface to communicate and control. The computer will receive information from the arduino regularly, and can control the Arduino by sending serial message in the form of text "A", "B", "C". I will wrap the communication as a function, so the logical programming flow should be like:

I have a input node, that can get value by calling a function.
The value is further operated / determined through logical operation node like "If/else" , "and/or".
The result will be transmitted to a output node, and a output function will be called.

I believe most of the features are being implemented, however in Android, I find the popup menu like "UnityEditor.EditorGUILayout.EnumPopup" in CalcNode cannot be displayed, do you have suggestion on this? I also tried to popup a menu when pressing a button in the sideGUI() of RuntimeNodeEditor, but it shares the same problem .... >.<

[pmhpereira]

@zardchim, you can see which custom UI elements are available in RTEditorGUI.cs. As of now, all the Popup elements are yet to be implemented. The remaining things work.

The problem with displaying the native ones at runtime is that you need to #import UnityEditor, which is impossible to do in a build. The alternative is to have our custom UI elements, which don't need UnityEditor.

[Seneral]

Ok, so this could be done connections which will be blocked, that will be the most convenient and cleanest solution. I'll try to get to that feature tomorrow, if you can't wait it should be hard to implement (add a variable/function fo the blocking state to NodeOutput and let the calculation system (in NodeEditor.cs) handle that) :)
pmhpereira is right about the editor GUI, it's unity's 'fault' ;)

[zardchim]

@pmhpereira @Seneral Thank you very much for the advise :)

I see, so UnityEditor.EditorGUILayout cannot be used in android while GUILayout can.

[Seneral]

Any news on this? Did you implemented an alternative solution? ;)

[zardchim]

@Seneral
I have been working on using the nodes to do basic control to the arduino and it works on PC, e.g. use a variable node to send command to a "LED node" to control the lightness.

However, the interface is not user-friendly on Android ... I need to press stationary on screen to popup the AddMenu and select which node to add. Besides I am thinking alternative way to replace PopUp element.

I am struggling on the scale of gui on different screen, for now different nodes are generated like node.rect = new Rect(pos.x, pos.y, 100, 50), I have replaced some of them to node.rect = new Rect(pos.x, pos.y, (Screen.width*4/5)/5, Screen.height/7) in order to perform adaptive sizing, however is there a clever way to do so?

[Seneral]

Than you'll be delightful to hear that I'm reworking the Input system. This will primarily seperates every control from each other and removes the mess the input function is atm, but this will also help to reassign, change and add new hotkeys and controls:)
For example you could add the ability to show the context with a double-tap instead of just the right-click.

Regarding screen dpi, it might work to use the zooming system. It works in a way that allows to manually modify the zoom and it persists (zooming only adds/substracts from the zoom factor). So you could set the zoom value apropriately on small screens depending on the Screen.dpi (or similar). Tell me if that works! :)

[zardchim]

@Seneral

Sometimes when I connected blocks, an error "GUI Error: You are pushing more GUIClips than you are popping. Make sure they are balanced)" appeared. I discovered the problem arise when I try to connect 2 knobs, after I clicked one knob and pull to somewhere else, if the MouseUp position is inside a Node, the bug appear.

It may be possible related to the code below, and the error is "NullReferenceException: Object reference not set to an instance of an object
NodeEditorFramework.NodeEditor.InputEvents () (at Assets/Script/Plugins/Node_Editor/Framework/NodeEditor.cs:529)
NodeEditorFramework.NodeEditor.DrawSubCanvas (NodeEditorFramework.NodeCanvas nodeCanvas, NodeEditorFramework.NodeEditorState editorState) (at Assets/Script/Plugins/Node_Editor/Framework/NodeEditor.cs:180)
NodeEditorFramework.NodeEditor.DrawCanvas (NodeEditorFramework.NodeCanvas nodeCanvas, NodeEditorFramework.NodeEditorState editorState) (at Assets/Script/Plugins/Node_Editor/Framework/NodeEditor.cs:135)
RuntimeNodeEditor.OnGUI () (at Assets/Script/Plugins/Node_Editor/RuntimeNodeEditor.cs:75)
UnityEngine.Debug:LogException(Exception)
RuntimeNodeEditor:OnGUI() (at Assets/Script/Plugins/Node_Editor/RuntimeNodeEditor.cs:86)
"

` case EventType.MouseUp:

                if (curEditorState.focusedNode != null && curEditorState.connectOutput != null)
                { // Apply Drawn connections on node if theres a clicked input
                    if (!curEditorState.focusedNode.Outputs.Contains(curEditorState.connectOutput))
                    { // An input was clicked, it'll will now be connected
                        NodeInput clickedInput = curEditorState.focusedNode.GetInputAtPos(e.mousePosition);
                        if (clickedInput.CanApplyConnection(curEditorState.connectOutput))
                        { // It can connect (type is equals, it does not cause recursion, ...)
                            clickedInput.ApplyConnection(curEditorState.connectOutput);
                        }
                    }
                    e.Use();
                }`

[Seneral]

I also experienced this sometimes but never was able to reproduce it. I restructured this section a bit though and since did not experierenced that problem again, so I think that fixed it:)

[zardchim]

@Seneral

I have git cloned a clear copy from the github and test again, and am able to reproduce the problem.

The procedure is : left click a knob and pull a wire from it, if the cursor position is inside a node when you mouseUp, the error appear.

[Seneral]

Ah ok, reproduced that too. I already fixed that though (because I though what the heck did I do there lol), and due to some other changes I cannot directly map that fix to the current version. The following should do it though:

if (curEditorState.focusedNode != null && curEditorState.connectOutput != null)
{ // Apply Drawn connections on node if theres a clicked input
    NodeInput clickedInput = curEditorState.focusedNode.GetInputAtPos(e.mousePosition);
    if (clickedInput != null && clickedInput.CanApplyConnection(curEditorState.connectOutput))
    { // It can connect (type is equals, it does not cause recursion, ...)
        clickedInput.ApplyConnection(curEditorState.connectOutput);
    }
    e.Use ();
}

Won't be able to respond for the next hours though:( Hope that does it for you;)

[Seneral]

@zardchim As I said I've been working on a new Input system for the canvas groups #41 (someone on the Unity Forums wants to work in them). It is basically finished and I'll soon create a commit (probably tomorrow). It is WAY cleaner and WAY easier to add new controls to or change existing ones:)

[zardchim]

@Seneral

It looks really awesome !!!!!!!!!!
Thumb up for thousand times (Y) (Y) (Y) !!!!!!!

[Seneral]

Here it is 8baf1fd! Enjoy! ;)
Well it should greatly ease the creation of new controls, especially for Android! You can just add new EventHandler/Hotkey attributes to the existing controls when on android (preprocessor checks?) to provide additional ways to access these controls.
Please report back if that fits your needs, @zardchim !

[Seneral]

@zardchim Hey, any news? I'm very interested if you managed to adapt the controls to touch:)
Additionally, I recently got the chance to work with Arduinos for a weekend, and I immediately thought about your goal. I'd really like to see what you've done so far, even if it just works on the PC right now;)