Mention pros & cons against other tools used for measuring application startup times

[eero-t]

There are pre-existing tools for doing same as what this does, albeit not so focused on getting single, and potentially misleading, number for application startup. First application window appearing isn't necessarily any indication that application is ready to interact with the user, alhtough that is as at least as relevant startup metric for the user, as the app window appearance.

XResponse tool was designed to measure X applications startup and other user interaction times: https://www.freedesktop.org/wiki/Software/xresponse/

Maemo version has several improvements on top of that: https://github.com/maemo-tools-old/xresponse

I think there was also some tool(s) that measured startup time user interaction timings using toolkit introspection / accessibility features.

PS. This could have picked a more original name. Just Googling "etrace github" returns many tools called etrace, and at least one of them (written decade ago) was even ptrace based like this is...

PPS. File tool should mention also how many times given file is opened/read. Sometimes apps redundantly parse files many times.

[anonymouse64]

Hi, thanks for the comments.

You're right of course that the primary measurement that we use this for, the time before a window is displayed to the user, is slightly misleading in that the application still is probably not ready to interact with users the way that something like xresponse would measure. This number is still very important to us for a few reasons though:

1. This number is very user visible. If a user clicks on a window and it takes 500ms vs 50ms to respond, the user will probably notice this, but if it takes 10s vs 1s to display anything after the user clicks on their launcher to open an application it is way more visible and they are much more likely to notice this. We see this in bug reports, on the snapcraft forum, on social media, etc. that users think snaps are slow primarily because they are slow to show a window or show some indication that the snap is opening/working/doing something after they go to launch it. It could certainly be the case that after we have made snap launch time faster than we will then run into the problem of snaps not being as responsive, but since we haven't had as many reports about that problem, we haven't put effort into measuring that problem.
2. This number also has very usability concerns in that if somebody is booting up their computer and wants to quickly launch a snap app to do something, taking an additional 5s to launch their application does slow them down, where as an application taking another few hundred milliseconds to respond to the user clicking on something most likely does not slow the user down this much.
3. Our analysis of this number has led us to find performance issues which are probably deeply related to the root cause of other performance issues with snaps. For example, with this measurement I also found that snaps have inefficient dynamic library loading: https://forum.snapcraft.io/t/effects-of-dynamic-library-caching-on-snap-startup-performance/14454 and analyzed the effect of making changes to how dynamic libraries are searched for, and those changes have led to some performance improvements in snaps that will also help non-graphical applications like CLI applications as well. As such, this number is a "good enough" proxy for general snap performance that it was worth the effort we put into analyzing it with this program and continues to pay dividends as we explore new avenues of performance. We may some day run into issues which etrace and the time to display a window do not help us, but we don't want to worry about that until we have to, premature optimization being the root of all evil and such.

Regarding the name, it's a fair criticism the name could be better and there are too many other things named this way, but I rather like how simple it is and naming things is hard and I'd rather work on etrace itself than spend time trying to rename or come up with a better name. 😄