Node-RED is a visual programming platform known for its user-friendly interface and low-code approach widely used for IoT application developement. In IoT applications, efficient data processing is crucial, and operations such as filter, map, sort, and reduce are essential for simplifying data transformations. However, many users struggle to implement these operations while sticking to a low-code approach, often relying on function nodes, which can introduce unnecessary complexity. In this guide, we will show you how to filter, map, sort, and reduce data with Node-RED for efficient data tranformation while sticking to low-code approach.

#  What is Low-Code

Low-code is a software development approach that requires little to no coding to build applications and processes. Instead of using complex programming languages, you use visual interfaces with basic logic and drag-and-drop capabilities.

Low-code is not just about accelerating development; it’s about democratizing it. It’s about giving more people the ability to create solutions to business problems.

— Charles Lamanna, Corporate Vice President of Business Applications & Platforms at Microsoft

For more details refer to the following articles:

• Why Low-Code is Better.
• Why you need a low-code platform.

#  Why do you need to learn to filter, map, sort, and reduce the data?

Filter, map, sort, and reduce are essential functions in data processing because they efficiently transform, extract, organize, and aggregate data, that makes it easier to analyze and derive insights from datasets. For example, consider the scenario where you have an array of sensor data retrieved from an database. The data looks something like this:

[
    {
        "timestamp": "2024-06-17T10:00:00Z",
        "temperature": 298.15
    },
    {
        "timestamp": "2024-06-17T11:00:00Z",
        "temperature": 299.15
    },
    {
        "timestamp": "2024-06-17T10:30:00Z",
        "temperature": 300.15
    },
    {
        "timestamp": "2024-06-17T10:15:00Z",
        "temperature": 301.15
    },
    {
        "timestamp": "2024-06-17T10:45:00Z",
        "temperature": 303.15
    },
    {
        "timestamp": "2024-06-18T09:00:00Z",
        "temperature": 297.15
    },
    {
        "timestamp": "2024-06-18T10:00:00Z",
        "temperature": 300.15
    },
    {
        "timestamp": "2024-06-18T11:00:00Z",
        "temperature": 301.15
    },
    {
        "timestamp": "2024-06-18T12:00:00Z",
        "temperature": 302.15
    },
    {
        "timestamp": "2024-06-19T10:00:00Z",
        "temperature": 298.15
    },
    {
        "timestamp": "2024-06-19T11:00:00Z",
        "temperature": 299.15
    }
]

However, you've noticed that the temperature data is in Kelvin, but you need it in Celsius. Additionally, the data is not correctly ordered by timestamp, and you only need the data of June 17th. Finally, you want to calculate the average temperature for that day. Users who are not familiar with Node-RED basics can use a JavaScript function node to achieve this, as shown below:

Using function nodes isn't wrong, but it adds complexity to your applications, for more information refer to the Drawbacks of using Fuction nodes Article. Since not everyone on the team may be familiar with JavaScript, it can limit who can solve business problems. To keep the application flow simple, using a low-code approach to perform these operations is crucial.

In the following sections, we'll explore how to perform these operations using a low-code approach.

#  Mapping

Mapping often refers to the process of applying a function to each item in a list, array, or other collection to produce a new collection of transformed items. here in our context, we need to covert the temperature data of each object from kelvin to celsius. To perform mapping we will use the Split, Change, and Join nodes.

1. Drag a Split node onto the canvas, the Split node will Split a message into a sequence of messages which will allow us to operate on each message, additioanlly split node bind the metadata to each of the object splitted, this metadata will helps join node to merge the all of message sequence back to an array.
2. Drag a Change node onto the canvas, set the msg.payload.temperature to payload.temperature - 273.15 as JSONata expression.

3. Now drag the Join node onto the canvas and set the Mode to "Automatic". This will automatically join all the messages originating from the Split node into an array.

#  Filtering

Filtering is the process of selecting specific items from an array to create a new array. In Node-RED, filtering is achieved using mapping and condition-based routing. Now we are familiar with mapping and have done it above, so we need to use only one more extra node which is the switch node for condition-based routing.

1. Drag a switch node and place it after the Change node and before the Join node.
2. Set the condition to check whether msg.payload.timestamp includes '2024-06-17' This condition ensures that only messages containing the specified date in their timestamp are sent further.
3. Next, In the switch node checked the option "recreate message sequences" that will repair the msg.parts metadata added by Split node if any messages are dropped by the switch node.

#  Sorting

Sorting, as the name suggests, means arranging items in a specific order. This order can be ascending (smallest to largest), descending (largest to smallest), or based on any defined criteria. In the Node-RED you can sort the numbers, alphabets, arrays, strings, and more. To perform sorting, we have to use the Node-RED Sort Node.

1. Drag the Sort node on the canvas.
2. Set the key to timestamp as the JSONata expression and then set the order to 'ascending'. We set the key to timestamp because we want to sort the data based on the timestamp. You can set it to temperature if you want to sort based on that instead.

#  Reducing

Reducing refers to the process of combining elements of a data structure (such as an array) into a single value. It involves iterating over the elements of the data structure and applying a combining function repeatedly until all elements have been processed.

1. Drag another Split node onto the canvas
2. Drag Join another node onto the canvas.
3. Select the mode to "reduce sequence", set Reduce exp to $A+ payload.temperature, initial value to 0 and the Fix-up exp to $A/$N

In this configuration, the Join node is set to reduce sequence mode. The initial value of the accumulator ($A) is initialized to 0. As each message is processed, the current temperature (payload.temperature) is added to $A. Once all messages have been processed, the accumulated sum $A is divided by the total number of messages ($N) to compute the average temperature.