[Discuss] move user-related background task parameters from alerts to task manager itself

[pmuellr]

Currently, task manager provides a way to run a function "in the background", but doesn't provide a way to associate a "user" with the task. For alerting, we use API keys, stored with the alert data, to provide that user-level association, to provide access to user- (and space-) scoped facilities such as ES callCluster.

Longer-term, we're likely to see more task manager clients want the same kind of capability - when their task runs, there is somehow a user-/space-/etc- scoped facility available to them.

So it seems like we should basically move all that functionality in alerting, into task manager. Presumably opt-in, not every task manager client would need this.

Slightly related: issue "research generic way of handling background tasks like requests" - #55648 . If we end up providing this scoping facility in TM, it might be nice to figure out the best way of making this work the same way scoping facilities work with HTTP requests - as methods available on a "request" object. In this case, it would be a "background" request, instead of an "http" request.

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[mikecote]

@kobelb I recall you having some concerns when this was mentioned a while ago. Was the concern about the load of API keys that would get created if we make this happen for every task? or something further?

I chatted with @ppisljar and seemed this may help their use cases for scheduled reports and background search. If we were to explore this, we could look into an opt-in approach for user context.

[kobelb]

If I recall correctly, my concerns were that API Keys are only available when the communication between Kibana and Elasticsearch occurs over TLS. We currently have to essentially block users from using Alerting until this occurs, so I was hesitant to perpetuate this limitation to other consumers of task-manager because I was paranoid that they wouldn't take this into consideration and assume they could always use API Keys.

[mikecote]

@kobelb got it, thanks for the info!

[thomheymann]

Having the ability to associate a user with a background task would be super useful for audit logging.

We currently log authenticated HTTP requests in the audit log which is helpful for auditors to understand what actions users performed.

I would like to do the same for creation and execution of background tasks but it is currently not possible since the fake request objects used by task manager have no user information attached. Having some sort of "authenticated background task" that is scoped to a specific user would enable audit logging.

The core team is having discussions to decouple core services from KibanaRequest objects which is related to this too: #39430

[mikecote]

There is also a request from the Kibana App Services team for such a feature. They would be able to use this for scheduled reports and potentially other services. I talked to them about the requirement of TLS between Elasticsearch and Kibana, and they seem ok with such a limitation. It would also make it consistent for background tasks between alerting and scheduled reports to have the same requirements.

I will put this issue up for triage again.

cc @ppisljar

[mikecote]

Moving from 7.x - Candidates to 8.x - Candidates (Backlog) after the latest 7.x planning session.

[mikecote]

Moving from 7.x - Candidates to 8.x - Candidates (Backlog) after the latest 7.x planning session.

We now cleared enough space in the ~7.14 release to accommodate an issue like this one. I will bring it back to the triage meeting again.

[mikecote]

We could split this issue and have a follow-up to migrate alerts to this at a future time.