Case sensitivity for keywords

[Barrybigbuddy]

We are doing some security analysis work and were finding out that applying the ECS standards for security data coming into ES essentially made all of our searches, via Discovery, case sensitive. We were doing a hunt for malicious data and even the search for GET vs get bring up different results since following ECS defaults to keyword for most string data. I was wondering, for the security use case (data using our Beats for different security inputs) if we could recommend and show people how to use normalization and lowercase for many of those security fields? Does it make sense? Or how we can control case sensitivity based on specific use case

[webmat]

Related #570

[webmat]

@Barrybigbuddy Probably not a full solution yet, but may help: #680

[ebeahan]

Since the last activity here, Elasticsearch has introduced the case_insensitive flag for various query types, which will allow for case insensitive matching of a value. A huge plus to this approach is no reindexing of data is required to take advantage of case insensitive. 😄

It falls outside the ECS scope, but a lowercase normalizer now also ships with ES. Custom fields or custom multi-fields could set normalizer: lowercase for particular use cases.