Batch

#  Where and why do we use the Batch node?

The Batch node is used to group sequences of messages into batches.

#  Modes of operation

#  Group by Number of Messages

The Batch node can be configured to group messages into batches based on the number of messages in a batch.

For example, if you set the batch size to 5, then the first 5 messages will be grouped into a batch before being released. Then the next 5 messages will be grouped into a batch, and so on.

See the below example flow for what this looks like in practice.

Batching messages into 5 groups

Additionally, you can add overlap so that the last message is included in the next batch

Batching messages into 5 groups with overlap

#  Group by Time Interval

The Batch node can be configured to group messages into batches based on a time period.

For example, if you set the batch time to 2 seconds, then all messages received within 2 seconds will be grouped into a batch before being released. Then the next batch will be created for the next 2 seconds, and so on.

See the below example flow for what this looks like in practice.

Batching messages into 2 second groups

#  Concatenate sequences

The Batch node can be configured to concatenate message sequences into a single message.

For example, you could use this to duplicate the sequence of messages in a sequence or order them by topic.

See the below example flow for what this looks like in practice.

Duplicating a sequence of data

Batch filter and concatenate

#  Demo flows used in this article

Node Documentation

Creates sequences of messages based on various rules.

Details

There are three modes for creating message sequences:

Number of messages
groups messages into sequences of a given length. The overlap option specifies how many messages at the end of one sequence should be repeated at the start of the next sequence.
Time interval
groups messages that arrive within the specified interval. If no messages arrive within the interval, the node can optionally send on an empty message.
Concatenate Sequences
creates a message sequence by concatenating incoming sequences. Each message must have a msg.topic property and a msg.parts property identifying its sequence. The node is configured with a list of topic values to identify the order sequences are concatenated.

Storing messages

This node will buffer messages internally in order to work across sequences. The runtime setting nodeMessageBufferMaxLength can be used to limit how many messages nodes will buffer.

If a message is received with the msg.reset property set, the buffered messages are deleted and not sent.