Import ECS dynamic templates into transform destinations

[chrisberkhout]

In an integration set (in _dev/build/build.yml) to import ECS mappings, the build process will inject ECS's dynamic templates at elasticsearch.index_template.mappings.dynamic_templates in data_stream/<ds_name>/manifest.yml.

However, for transforms, it does not put them into destination_index_template.mappings.dynamic_templates in elasticsearch/transform/<transform_name>/manifest.yml.

The relevant code is here.

This came up while building an IOC expiry transform for OpenCTI. The workaround was to add an explicit external field reference for each ECS field used.

[andrewkroh]

I think the package-spec also needs to be clarified to state what data streams and indices import_mappings applies to.

https://github.com/elastic/package-spec/blob/a347e4dd88ac1ce1e4c894cfa6e0288460e99eb6/spec/integration/_dev/build/build.spec.yml#L26

[chrisberkhout]

@jen-huang It's now preferred to not import ECS mappings, since elastic/integrations#8542.
However, there's still an issue for transforms. The logs index template applies ecs@mappings to logs-*-*, but only for data streams, so it won't be applied to a transform destination index.

[jen-huang]

cc @kpollich