ERT client high CPU usage

[markusdregi]

It has been observed high CPU usage on the client side while running forward models on the cluster.

[markusdregi]

Should be tested on an RGS node with a 1000 realizations and 200 jobs per forward model.

[markusdregi]

Talk to the users directly.

[jondequinor]

As @sondreso suggested, will test 1000 realizations with 200 jobs that just sleep—locally.

[jondequinor]

1000 realizations * 200 jobs using LSF did eventually consume, for normal view and detailed view respectively, 40% and 100% of the CPU on which it ran. I saw 4 second response time in the GUI for detailed view, none for normal view.

Using the CLI to run the same simulation consumed 2% CPU.

The visual rendering of the simulation progress is thus the cause of this high CPU usage.

[jondequinor]

We will address this by scaling down the update frequency of the heavier parts of the UI proportionally to the amount of realizations you have. This makes sense when the cost of properly fixing this is compared to the negative effect this will have on the UX. I.e. the proper way requires (weeks of) refactoring—whereas the effect on the UI is that an update may take 30s, which is negligible compared to simulations that take days to complete.

[markusdregi]

Should we have a maximum update frequency of 10 seconds?

[jondequinor]

Maybe we split the GUI into three parts, each with its own frequency?

1. A maximum more-than-10 second update frequency for the detailed view.
2. A maximum of 10 second update frequency for the simple (main progress bar) view.
3. The run time is updated every second—always.

The reasoning here is that the CPU spikes for a couple of seconds each time the detailed view is rendered. And my CPUs were idling, so this may be more than a couple of seconds on less busy machines…