Merge Buckets and prune events older than set interval

[skewballfox]

I think that activity watch could greatly increase it's utility by keeping significantly less data. One of the issues I've been having with using the data in python is the signal to noise ratio. Many of the events generated are noise, and exist because the state of some window changed without any user interaction (I can provide an example collected from my local system if necessary), and as it currently is finding meaning in the data is much more difficult than it should be.

Let's take a moment to think about the purpose and intended user for activity watch. This is a tool meant for self-tracking, i.e. monitoring a single user potentially across multiple systems. As human beings, we can only direct our attention to one thing at a time. Even when multi-tasking, we are really just doing a context switch and single tasking repeatedly. and our devices are designed with this in mind: only one window can have focus/capture input at a given moment.

I think it would be best to prune the data (from the watchers) according to the following heuristics:

1. On a single system, Afk events take priority, followed by window events
2. on multiple systems host-specific priority is determined strictly by time
3. if a higher priority event starts prior to the end of the duration of the current event, then the current event is split into two events
4. if an event has the exact same data as the current event, then the duration of the previous event should be extended
5. if the app associated with a specific event is something which produces events (such as from "aw-watcher-web-chrome"), then that window event is replaced by the events of that watcher(alternative would be to "embed" the app events into the window event)

Note this isn't talking about the data collected directly by watchers, as some of these rules only work if assessing the data after the fact. More that the data should be "pruned" according to a user defined interval of time: every day, once a month, etc. Some situations are not covered intentionally (such as whether to consider watching media as being afk or not), as those should be up for discussion. but even without any changes, this would drastically improve what you could derive from looking at the timeline: you would see how your attention drift from window to window and device to device. you're timeline could now directly, intuitively show you things such as what monitored stimulus generally happened prior to picking up your phone, how your google searches related (or didn't) to the application/file being interacted with, etc. It has an added bonus of drastically reducing the amount of data stored, and thus translates to multiple improvements: longer intervals before intervention, improved query times, and much less noise for people trying to use it with other datasets(such as biometric data).

I'm currently working on an algorithm to prune events according to these rules, and working on a visualization to represent this timeline in a way similar to a horizontal gitflow diagram. I'm trying to do it in plotly so that it can be translated easily to javascript(and thus integrated into the web-ui).

I also think that there needs to some way to extend this algorithm, so that third party data might be integrated without necessarily needing to construct a watcher( ex. terminal history captured via atuin).

appreciate any feedback, and if there is anything I need to alter about the ruleset, please let me know.

[skewballfox]

I've created a very rough implementation of algorithm for merging watchers here. Note, currently this doesn't handle multiple host, and I think the best way to do that, is to handle the host separately, and then merge afterwards. Once I have a working algorithm, I'll port it to rust.

[skewballfox]

an alternative to merging the watcher events into a single dataset, is to simply prune the buckets according to the rules.