Flow Guideline Contents
status: work in progress
This documentation shows practices, tips and know-how to effectively develop flows.
Messages that move through a flow are JavaScript msg objects.
Such messages are partially affected by Node-RED and Node.js restrictions and the implementation models, but are highly flexible because you can add any attributes to, and input values of any type into, a message object.
That is, you can modify all properties in a msg object and can add new properties.
However, a message causes dependencies between the node that generates the message and the node that uses the message.
For example, if a node expects the msg.payload of an input message to include temperature information, a node at a stage previous to that node must insert temperature information into msg.payload.
If you write a large-scale flow, the message moves through a lot of nodes. So, not clarifying such dependencies will seriously impair the flow's reusability and flexibility to changes.
Therefore, to increase the reusability and flexibility of a flow, you should consider an appropriate design for messages as well as for the flow structure.
In addition, the system design might change depending on the message design. Before implementing a flow, you need to design the messages.
There is no naming restriction on the names of properties of the msg object. Therefore, there is a risk that multiple nodes will unintentionally use the same property.
In order to suppress this risk, it is better to determine, as a design rule, the parameter that stores the data in the msg object for each data location.
Messages were divided into two types: data that is processed by nodes, and parameters that control the node functionality.
We recommend that you set the former data in the msg.payload property.
This is because nodes provided by Node-RED use properties immediately under msg. for the latter data.
For example, the MQTT output node references the topic property (msg.topic) of the received message, and sets the topic name of a message to be sent to the MQTT broker.
The following figure shows an example in which a bug occurs as a result of using the 'msg.topic' property in Function node proc1 as the storage destination of the data to be processed by the node. Although the topic name is set to "Building/Room1/temperature" at the MQTT sending node, the topic name becomes an unintended value ("hot" or "cool") and is sent.
In particular, care is needed because nodes such as the MQTT node can set control of the node's functionality by the node settings displayed in the sidebar and also by settings specified in the msg object.
In such nodes, the setting specified in the msg object has priority even if the setting is written in the sidebar.
This is why you must clearly separate the parameters used to store these two pieces of information in the msg object.
Even if the design follows this policy, many pieces of data might be processed by a node, and a conflict with a property name might occur between them. Because no method mechanically checks dependencies of properties between nodes, users must understand the properties accessed by each node. The following figure shows an example of a problem that occurs when multiple nodes use the same property name in different processes.
To make this problem easier to handle, design the flow structure so that the number of nodes in dependencies does not exceed the number of tabs. Then, make the flow in a tab as small as possible. Just doing this can make it much easier to understand the dependencies.
In addition, the first node of the flow in a tab verifies that the message provided as input has the expected property structure. Any properties other than those expected to exist can be used freely in the tab. This verification process also represents the boundaries of dependencies between tabs.
In Node-RED, you can create nodes and use them on Node-RED screens.
In such a case, a parameter in msg might control the functionality of the created node.
As a policy, it is better to set a parameter that controls the functionality of a unique node by using the format msg.[node-name]_[parameter-name].
The reason for using the parameters immediately under 'msg.' is that the data to be processed by a flow is set in the msg.payload property or properties under the msg.payload property, as described in the previous section.
The parameters immediately under 'msg.' also use Node-RED standard nodes, so use the format [node-name]_[parameter-name] so as not to collide with the standard nodes.
Although msg objects become redundant, you can also use the format msg.[node-name].[parameter-name].
When a user creates a unique node, develop the node assuming that the parameter that controls the functionality of the node is entered by using the format msg.[node-name]_[parameter-name].
Note that backward compatibility is mandatory because the property name of the parameter that controls the functionality of a node is the interface between the flow design and the unique node. Therefore, when assigning a property name, as much as possible use a name that has backward compatibility.
In a node to which multiple messages with different types are input, add tag information for identification to each message.
A Node-RED node can have multiple output ports, but can have only one input port. Messages moving through flows include messages for representing the control of a node and messages for representing the data to be processed. Therefore, when you need to distinguish input messages with different types for each execution path of a flow, you need to distinguish the input messages according to a property in each message.
More specifically, assign a tag for distinguishing the execution path to a property of each message and set a different tag value for each input path. In node processing, the processing for each message is selected by checking the tag value. To set tags, you can use a template node.
This makes it possible for a node to which multiple types of messages are input to perform processing according to the message type.
Storing data to be processed by nodes in msg.payload was described previously. However, you need to be careful when handling large amounts of data.
Messages are secured in memory. If large amounts of data are included in messages, a heavy load is placed on the processing time and the amount of memory used.
In particular, note that the entire processing of Node-RED will stop when the memory is exhausted.
To avoid such problems, you can store the actual data in persistent storage outside Node-RED, and reference that data in messages. In a flow, configure the nodes so that they receive references and process the data that is on persistent storage.
Because only references to data are passed in a flow, you cannot use nodes that expect the actual data.
In particular, you might be unable to reuse the standard nodes prepared by Node-RED and the nodes and flows created by others for other purposes.
You need to devise a method for inserting the data into Node-RED (for example, finely dividing the data to be referenced and then entering it into msg.payload).
You need to consider applying this guideline while taking these tradeoffs into account.
In addition, validation of incoming messages that nodes expect becomes possible. Also debugging is facilitated if error handling processing is added to handle the arrival of an unexpected message. Error handling is described in "Improving Reliability and Quality" (reliable_high_quality.html).
In processing in which multiple messages are received, the order of arrival of messages is not guaranteed. Therefore, you need to design the processing of a flow so that the processing does not depend on the order of arrival. In other words, the correct result must be obtained even if messages arrive in an order different from the expected order.
To explain this, this section describes Node-RED itself. Node-RED is developed using Node.js, which is an asynchronous event-driven programming language. As a result, in Node-RED, processing of each node is executed in an event-driven manner. Here, an "event" indicates that a message entered a node.
The fact that each node is asynchronously executed in an event-driven manner in Node-RED means that processing is not synchronized between messages when multiple messages move through a flow. By not synchronizing processing, a flow can simultaneously process more messages because processing of a message does not need to wait for the end of processing of another message. On the other hand, there is no guarantee of processing order, and no guarantee that messages that previously entered the flow will be processed first.
If the user needs to perform processing that depends on the order of messages inserted into the flow, the user needs to implement, in the flow, processing that controls the processing order. For example, consider a method for the situation in which information in received messages is accumulated in a node context, and processing is to be performed after all the required messages are received.
// Accumulation of messages
var msgs = context.get('messages') || [];
msgs.push(msg);
if(msgs.length === ...) {
... // Process messages
}
context.set('messages', msgs);