Review the fields of the wazuh-monitoring index

[AlexRuiz7]

Description

Parent issue:

• Improve overall perceived performance #4155
• Time-consuming Agents section #3623

We need to review the fields of the documents for the wazuh-monitoring index. This index contains data about the Wazuh environment, and essentially is used to populate the Agents Evolution chart. The objective of this chart is to display a histogram with the agent's status (connected, disconnected, pending, never connected) over time.

For this objective, only the agents' status is required, but the whole response from the /agents endpoint is stored.

{
  "_index": "wazuh-monitoring-2022.32w",
  "_type": "_doc",
  "_id": "j5flfIIBzH8x6PVkpeiR",
  "_version": 1,
  "_score": null,
  "_source": {
    "os": {
      "arch": "x86_64",
      "major": "2",
      "name": "Amazon Linux",
      "platform": "amzn",
      "uname": "Linux redacted |#1 SMP Tue May 7 02:26:40 UTC 2019 |x86_64",
      "version": "2"
    },
    "ip": "10.0.1.68",
    "mergedSum": "d9f3434a15b4ac879334867f1374da14",
    "status": "active",
    "manager": "wazuh-manager-master-0",
    "version": "Wazuh v4.3.6",
    "registerIP": "any",
    "configSum": "8361b0d0c5021ec104b85cf76b3ac0a0",
    "group": [
      "default"
    ],
    "dateAdd": "2022-07-20T09:23:14Z",
    "id": "001",
    "node_name": "master",
    "name": "Amazon",
    "lastKeepAlive": "2022-08-08T09:59:57Z",
    "timestamp": "2022-08-08T10:00:01.167Z",
    "host": "wazuh-manager-master-0",
    "cluster": {
      "name": "wazuh2"
    }
  },
  "fields": {
    "dateAdd": [
      "2022-07-20T09:23:14.000Z"
    ],
    "lastKeepAlive": [
      "2022-08-08T09:59:57.000Z"
    ],
    "timestamp": [
      "2022-08-08T10:00:01.167Z"
    ]
  },
  "sort": [
    1659952801167
  ]
}

There is no doubt that, if the Agents Evolution chart is the only reason to be of this index, we are storing unuseful information, which obviously consumes disk space, and over time, can slow down queries (take into account that by default, a document is created within 15 minutes).

Tasks

• List the uses of the wazuh-monitoring index.
• Optimize the wazuh-monitoring index, based on the results of the previous task. (see Alternatives)
• Discuss if the default job interval makes sense for the uses of the index (default is 15 minutes).

Alternatives

• Fetch the strictly required fields from the /agents endpoint.
• Use the /agents/summary/status endpoint, which already provides the required information for the Agents evolution chart use case.

[gdiazlo]

We've decided to let the fields as they are and revisit this in the future.