ES slowlog module improvements

[immon]

• change keyword to integer for few fields
• split types

[ruflin]

PR will need a changelog entry

[ycombinator]

Indentation is slightly off.

[ycombinator]

I wonder if we want to index an array of one element, that element being an empty string, over here OR if we want to index an empty array OR not even index this field if no types are present.

Now that Ingest processors support if conditions (elastic/elasticsearch#32398), perhaps we could define the split processor like so:

{
   "split":{
      "field":"elasticsearch.slowlog.types",
      "separator":",",
      "ignore_missing":true,
      "if":"ctx.elasticsearch.slowlog.types != ''"
   }
}

[immon]

@ruflin any opinion on that?
IMHO types field should be stored since it appears in the log line.
Arrays and single values are internally stored the same way since any ES field can be an array. Indeed _source will look different since source document is different, but should we care?

[ruflin]

The log entry that this is coming from looks like this: total_hits[19435], types[], stats[],. If I run the following command in DevTools what I get is "a": []. So I would argue types should be there but with an empty array and not entry.

POST test/_doc
{
  "a": []
}

[immon]

Thanks for review @ycombinator
I removed split processor to get PR merged. I wanted to change data type for few fields but got stuck on the split processor (and things like ES 6.5 installed to run tests)

[ycombinator]

LGTM

[jsoriano]

This will need to be updated with master and make update.

[immon]

@jsoriano this slipped under my radar. I make updated the code + fixed ECS stuff in tests but CI is failing due to lack of workers.

[jsoriano]

@immon sorry for the late reply. Could you try to update again?

[elasticmachine]

Pinging @elastic/stack-monitoring

[jsoriano]

There seems to still be something wrong with the expected json files 🤔

[jsoriano]

It LGTM, I have a doubt on the type for the shard id.

[jsoriano]

jenkins, test this again please

[immon]

jenkins, test this again please

[ruflin]

This seems to have quite a bit of conflicts with master. Let's review again when it's rebased.

[ycombinator]

Hi @immon, could you rebase this PR on master please? That should help towards resolving the conflicts. Thanks!

Alternatively, you could just close this PR and make a new one, whichever is easier!

[immon]

I totally lost track on that effort. It will be easier to create new PR since many things has changed since version 6.5 and 2018. I'm closing this one.