[Alert Summaries] Add summary capabilities to the API and execution logic

[ersin-erdal]

Meta: #143200

Blocked by: #143374 and #143368

The rule API and Task Runner should be modified to provide the new Alert Summaries options and functionality.

If an action is a Summary of alerts per rule run, alerts of the current execution context should be injected in to the action context.

If an action is a Summary of alerts on a custom interval and the action is still throttled the action should be skipped.
If not, alerts that are generated within the given time span, should be injected in to the action context. Alerts should be fetched using the developed provider in #143374.

This issue relates to the following functional specifications (#143200): 1, 3, 4, 5, 6, 7.
This issue relates to the following non-functional specifications (#143200): 1.

[mikecote]

Part of this issue, we should also add the new Alert Summaries options.

[mikecote]

Blocked by #143368 and #143374.

[mikecote]

Merging #143380 into this issue.

[Alert Summaries] Build mustache variables differently for summary actions #143380

[mikecote]

@ymao1 @kobelb

With the upcoming alert summary feature, we will soon provide users access to the full AAD document. This allows users to build mustache templates on any field.

It isn’t too late if we want to limit / filter out fields that would be accessible in rule actions. The current plan is to provide access to everything (including kibana.* fields), should we change that?

[kobelb]

It isn’t too late if we want to limit / filter out fields that would be accessible in rule actions. The current plan is to provide access to everything (including kibana.* fields), should we change that?

Would we be exposing all fields from the existing AAD documents, or only the fields in the forthcoming FAAD schema?

[mikecote]

Would we be exposing all fields from the existing AAD documents, or only the fields in the forthcoming FAAD schema?

In the initial phase, it's hooked up to the rule registry, so all the existing AAD fields / documents.

[mikecote]

We could look into limiting existing AAD documents to only FAAD fields, if we have finalized/limited the list.

[kobelb]

Thanks for the clarification, @mikecote. The safest option is to hold off on exposing the kibana.alert.* fields and wait on the forthcoming FAAD schema. Otherwise, we run the risk of users starting to consume fields that we will end up removing. Do you see this being a problem?

[mikecote]

Thanks for the clarification, @mikecote. The safest option is to hold off on exposing the kibana.alert.* fields and wait on the forthcoming FAAD schema. Otherwise, we run the risk of users starting to consume fields that we will end up removing. Do you see this being a problem?

Shouldn't be a problem from my perspective, it's easier to add later than to remove, so lets filter them out at first.

We will have to migrate the security solution to this feature at a later time, we can see if we need special handing for them, while keeping the fields out elsewhere.

[mikecote]

It might be that we'll need to expose some of the information about the alert if users want to iterate through them and provide a brief description of the alert 🤔 (ex: alert id) but maybe ECS has enough information for those use cases?

[kobelb]

It might be that we'll need to expose some of the information about the alert if users want to iterate through them and provide a brief description of the alert 🤔 (ex: alert id) but maybe ECS has enough information for those use cases?

I don't think ECS will have enough information. If we know we need certain fields exposed and we're fairly confident that they'll be in the FAAD schema, we can allowlist specific fields.

[mikecote]

I don't think ECS will have enough information. If we know we need certain fields exposed and we're fairly confident that they'll be in the FAAD schema, we can allowlist specific fields.

Sounds good, we will do an exercise to allow list specific kibana.* fields and get feedback. I will create an explicit issue to limit the fields for summaries.

[mikecote]

Removing blocked label as #144130 is now merged.