Make Event Batching Configurable

[vikman90]

Parent Issue: #241

Description

Currently, the Wazuh Agent sends batches of messages from its queue to the server every second. We propose making the batching process configurable to allow greater flexibility.

The agent should send a batch of messages under the first condition that occurs:

1. If batch_interval has elapsed since the last batch and there are messages in the queue.
2. If the accumulated size of messages in the queue reaches batch_size.

This design ensures:

• Messages do not stay in the queue for longer than batch_interval seconds.
• Requests do not exceed a specified maximum size.
• The queue drains at a controlled rate, preventing overflow unless external limitations apply.

Proposed Configuration Options

The following parameters can be added to agent.yml:

events:
  batch_interval: "10s"  # Maximum time between batch sends (default: 10 seconds).
  batch_size: "1000"       # Maximum number of messages in a batch (default: 1000).

Parameter Details

• batch_interval:

  • Specifies the maximum time between batch sends.
  • Accepts values with time units: "500ms", "10s", "1m", "1h".
  • Valid range: "1s" to "1h".

• batch_size:

  • Specifies the maximum number of messages in a batch.
  • Valid range: "1000" to "1000000".

[vikman90]

Work update

November 18

• This branch started from Standardize Time Unit in Agent Configuration Parser #286.
• Added default values to wazuh-agent.yml.
• Options parsed and applied in the code.
• The code pases clang-tidy.
• Smoke tests with mock-server: Test Wazuh Agent with Mock Server Using Docker and Imposter #281.
• As the code in the mentioned branch needs some changes in the unit tests, we need to wait for that before implementing the tests for this task.

[jr0me]

Update

Checked out branche and rebased to latest master. Fixed tests related to the message queue get and pop messages functions.

[jr0me]

Update

Picked up the work after having it paused by other tasks.

Rebased again and solved conflicts.

There are new variables for setting the batch size and interval.

The multitype queue already supported having a timeout interval, the mentioned variables will be used as the multitype queue timeout interval. The mechanism to either wait for the batch to be complete or the timeout to have passed is in place.

What is missing now is a way to communicate how many messages have been used from the queue, so they can be popped at a latter staged if they were recived by the manager. Currently the queue doesn't enforce its invariant in any way, so how many messages are retrieved and how many are popped needs to match and this value has to be persisted throughout.

[jr0me]

Update

The number of messages retrieved is now shared between get and pop calls. Needs testing.

Proposed configuration options need to be renamed.

[jr0me]

Update

Created PR for the issue. Waiting review.