Backend saving of aw-webui setting / categories etc.

[xylix]

Is your feature request related to a problem? Please describe.
Issue #342 is related, and also would help the "Note: These settings are only saved in your browser and will not remain if you switch browser. We are working on getting this fixed." warning we have on the webui currently. It would also be a step towards syncing user settings, after implementing sync as a feature.

Describe the solution you'd like
An ability to save aw-webui settings in the backend, probably through a POST with a settings object as the body. Ability to retrieve the settings through a get request.

Describe alternatives you've considered
Postponing this feature? I kind of see it as a mandatory step at some point.

@johan-bjareholt Erik told me you might have ideas / info about implementing this feature on the backend side. Can you fill me in if you've been thinking about a solution / if you have any ideas?

[ErikBjare]

There are a few options here. Either we create a simple key/value table in the SQLite database, or we create a separate key/value datastore.

We could use this key/value store for other things as well like app icons or saving queries in the query editor.

In any case I suggest the keys are simple namespaces, like settings.categories and settings.dayoffset.

---

When it comes to syncing, I'm not sure how that should work. One way could be have the most recently updated value across all hosts always be the canonical value for any given key. That would require the table to also have a column for when the value of a key was last updated.

This would however enable possibly deleterious effects (let's say I saved a set of categories on a remote host, then added it to syncing, which then wiped my carefully edited categories on my main device). So maybe always ask the user when there is a newer set of settings available? Or some other solution that I haven't thought of. Whatever the rule is it needs to consider CRDT (Conflict-Free Replicated Datatype) mechanics.

[xylix]

Yeah there's some things to figure out about the syncing of settings. Personally I'd favour the ask the user approach, maybe with a dialog showing diffs for all key sharing values which are different and asking user which value to keep.

Looking into adding a simple key/value table to the sqlite database, I ended up at here https://github.com/ActivityWatch/aw-core/tree/master/aw_datastore . It seems like the current datastore class is pretty heavily designed to only support buckets (with it redefining getitem to getting buckets and all). Will have to make a decision whether to just add a new datastore class for kv or to extend the existing datastore.py.

@ErikBjare Am I right in thinking that peewee and mongodb are both old implementations, and we only support sqlite now?

[ErikBjare]

with it redefining getitem to getting buckets and all

I don't think the __getitem__ is being used anywhere, it shouldn't be anyway.

Am I right in thinking that peewee and mongodb are both old implementations, and we only support sqlite now?

Actually peewee is the datastore currently used by default, but the idea is to switch over to the sqlite datastore. The only thing keeping us back is fear that the migration could mess up somehow for users, but in my and @johan-bjareholt's experience it's pretty solid. The MongoDB datastore is deprecated and will be dropped. So I guess you could say that we'll only support the sqlite datastore going forward.

[xylix]

Allright if the __getitem__ shouldn't be used anywhere I feel a bit safer adding some create_value and delete_value methods.

Interesting, trying to figure out if I am running pw or mongodb (I remember seeing some infopopup about peewee not being available when I last built ActivityWatch on my laptop) I got a 500 internal server error when trying to GET the /0/logs endpoint. The info endpoint also doesn't contain info about the current database backend.

But considering your info, should I create a task / work on migrating all users to sqlite? Would seem like it would enable me to add db features without needing to duplicate the work on other datastores.

[ErikBjare]

You most definitely are using peewee. I think the /0/logs endpoint has been broken quite a while and isn't tested nor used anywhere anyway (deprecated).

I think the only thing needed to switch to sqlite is to change the default wherever it is set in the code, then hope that the migration goes smoothly for all users. Feel free to submit a PR doing exactly that, and then start working on get/set operations for the key-value store.

[ErikBjare]

The PR that implemented the necessary API in aw-server-rust is here: ActivityWatch/aw-server-rust#87

[treymo]

I'd like to see this feature and am trying to catch up on the status. My current understanding is that: to support a key-value store, the default storage for aw-server needs to be changed from peewee to sqlite.

I saw the discussion in ActivityWatch/aw-server/pull/64 but I'm not really sure what the conclusion was. Will aw-server be moved to sqlite? Is there a way around this sqlite migration to get this feature in the python server? Will this feature have to wait until the new rust server is the default?

[johan-bjareholt]

@treymo The key-value store is already implemented in aw-server-rust, we are only waiting for that to be the new default server which should hopefully happen soon.

[treymo]

Gotcha, so this feature is not going to implemented in the python version of the server. I will hold off and hope this transition to rust as the default happens soon.

[johan-bjareholt]

@treymo Yes, we are only waiting for that to be the new default server which should hopefully happen soon. It saves us time if we only have to implement it once.

[in-in]

I really want to save my custom rules for categorizing events so I don't have to fill those fields again. In case of reinstalling the system or transferring the configuration to another machine. 🙏

[awesomehaze]

If it would be easy, a simple import/export feature would go a long way in making this less painful.

[ThyrixYang]

@johan-bjareholt How to use aw-server-rust to export my config now? Can we just change the server and export then change back?

[johan-bjareholt]

@ThyrixYang Saving in the backend won't land until aw-server-rust lands which little progress has been made the past few months. There is however progress on exporting/importing categories

ActivityWatch/aw-webui#260

[ErikBjare]

I recently made a lot of progress on the frontend-side in this PR: ActivityWatch/aw-webui#327

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tobiasdiez]

This is still not implemented as far as I can see.

[ErikBjare]

I finished this today, see ActivityWatch/aw-webui#493

Might need some battle-testing in beta releases (releasing v0.12.3b11 later today), but seems pretty ready!