Rewrite plugin_status logic limiting usage of Observables (reducing heap size)

[gsoldevila]

Attempt to solve #128061

The idea is to rewrite the src/core/server/status/plugins_status.ts logic in order to simplify the RxJS pipeline. We take into account "input" observables and "output" observables, but we no longer use RxJS for the internal update of plugin statuses.

1. We receive core status updates, and plugin reported status updates through input observables. We debounce these events to avoid unnecessary calculations.
2. The PluginsStatusService class maintains an updated copy of the state, in a sort of Redux store pattern, so to speak. The state includes convenient structures to make updates more efficient. These structures include a tree of reverse dependencies, and a list of plugin names sorted by depth. The root plugins (depth = 1) are those that do not depend on any other plugin. A plugin that depends on a root plugin will have depth = 2, and so on so forth.
3. The state is updated iteratively, going through the ordered list of plugins. This causes less recursive calls and less heap consumption.
4. Once the state has been updated, PluginsStatusService then signals the internal Subjects that are connected to the output observables.

Key changes to reduce heap size and improve performance

• Previous implementation used Observables to recalculate and propagate the status of each plugin through the tree of dependencies. The switchMap and concatMap operators create new Observables with each higher order emission.
• The recursive approach is not optimal in this scenario, because in order to make sure all statuses are updated, we might need to visit each node more than once. For instance, if plugin C depends on plugins A and B we might have to calculate C's status twice (once when we process A's dependencies and another time when we process B's dependencies).

Memory consumption impact

Heap size after Kibana started reduced from 800Mb to 280Mb.
Heap snapshot is taken right after Kibana started.
main branch

the current branch.

[mshustov]

buildkite build this

[mshustov]

buildkite test this

[mshustov]

@gsoldevila, if we confirm the positive impact of the PR on memory consumption, I'd consider porting it back to 8.1 and 7.x.

[jbudz]

This is huge, memory consumption looks great:

#128061 (comment) for comparison.

[mshustov]

Maybe we shouldn't pass the default values?

[gsoldevila]

This was like that in the previous implementation.
The getSummaryStatus is used in a couple of places:

• status_service uses the default "All services are available"
• plugins_status (this class) uses the slightly different "All dependencies are available".

The problem is that getSummaryStatus method does not currently have the knowledge of whether it's being called with plugins statuses only, or with plugins statuses + core services statuses.

But if we agree to use the exact same message for both cases, then I can probably remove that default value.

[mshustov]

question: maybe we should iterate through toCheck set?

[gsoldevila]

Actually, we need to iterate in "depth" order:

• first updating root plugins' statuses, then updating depth=2, and so on so forth.

Otherwise we might try to update a plugin status before updating its dependencies.

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 9db8466
• Storybooks Preview
• Cloud Deployment
• Webpack Bundle Analyzer

Metrics [docs]

✅ unchanged

History

• 💛 Build #33645 was flaky 3d64fed
• 💔 Build #33463 failed c1f15f3
• 💔 Build #33126 failed 6113711
• 💔 Build #32952 failed 6113711
• 💔 Build #32859 failed 4b023f4

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream