External ECS field imports not containing their multi-fields

[ebeahan]

Summary

The expected multi-fields are not imported when external ECS fields are declared. I suspect this is the cause of some integrations not including the anticipated multi-fields in their index templates.

I first experienced on 7.16.3 using the Windows 1.5.0 integration. The logs-windows.sysmon_operational managed index template is missing process.command_line.text.

Example

Declaring an external field with process.command_line:

- name: process.command_line
  external: ecs

produces an index mapping of:

    "process": {
      "properties": {
            "command_line": {
              "type": "wildcard"
            }
          }
        }

In ECS, process.command_line contains the .text multi-field.

The expected field mapping:

    "process": {
      "properties": {
            "command_line": {
              "type": "wildcard",
              "fields": {
                "text": {
                  "type": "match_only_text"
                }
              }
            }
        }

[andrewkroh]

There appears to be a similar problem in that the doc_values and index attributes are not carried through from the ECS definition. event.original is an example.

The integration mapping is:

   "event": {
      "properties": {
        "original": {
          "ignore_above": 1024,
          "type": "keyword"
        },

The expected mapping is:

        "event": {
          "properties": {
            "original": {
              "doc_values": false,
              "index": false,
              "type": "keyword"
            },

https://github.com/elastic/ecs/blob/v1.12.2/generated/ecs/ecs_flat.yml#L2354