[BUG] Clearing browser cache loses all users' data and app config with no warning

[TimeScience]

Description

When a user clears their browser cache all data in big-agi is lost without warning and is not recoverable. There doesn't appear to be any text alerting a user to this when they start using big-agi (or at least not one that is obvious enough given the potential consequences of losing all of one's data).

This issue might be considered a feature request not a bug (ie requesting persistent non-browser cache storage), but from a UI/UX perspective, if a user can lose all their past chats and all API configurations just by clearing the cache on their browser and there is no warning in the big-agi app that this might happen, the outcome is users can easily lose all their data and config without warning, which is a pretty serious bug, even if the solution to it might be considered a new feature.

This is particularly an issue because big-agi runs in docker and also can be configured to run as a stand-alone app (I know this is really a browser feature, but from the user perspective, it behaves like a stand-alone app), so from a user perspective (particularly if one is not really a developer), big-agi behaves as if it were a desktop application, so a user might reasonably assume that their data is persistent and will not disappear if they happen to clear their browser cache... Like imagine if every file one created in Office just stopped existing if you cleared your browser cache. That's basically what happens with big-agi at the moment.

Suggested solutions:

• In the short term, as a minimum, it would be helpful to at least warn users on startup that this is the case and provide them with instructions on how to export/import their data
• Add recurring auto-export of all chats feature (which is essentially adding persistent storage since the user data is then stored persistently)
• Include a setting to export AI model config... assuming that API key backup is harder due to security issues, it would be nice if at least the model selections and other config are saved and would re-appear once the user put back in their API keys.

The longer term way to fix this would be to either set up secure export of all settings to desktop (or just have the app behave like normal desktop software and save this somewhere on disk)

Device and browser

Windows 11

Screenshots and more

No response

Willingness to Contribute

• 🙋‍♂️ Yes, I would like to contribute a fix.

[enricoros]

A cloud backup also will be great to survive wipes. Leaving this open as there are good ideas.

[TimeScience]

yeah definitely. I had been thinking that cloud sync would require admin on the side of big-agi to manage accounts, etc but it occurs to me that Cloud Sync using a user's credentials with one or a few cloud storage providers would also solve the problem and not require any business infrastructure on the part of big-AGI, so that's a nice potential solution as well.

[enricoros]

@TimeScience I'm implelenting it, at least for the users of big-agi.com to start with. I have ~2,000 chats including on mobile that wait to be synchronized.

[rodgermoore]

I am such a big(AGI) fan of this project! So many great features. THIS, local browser cache storage, is the only part of Big-Agi that is not user friendly at all. For the best user experience you just want server side syncing and not worrying about losing all your chats. If cloud-sync is implemented it would be great all devices can pick up this sync so all chats are synced on all devices a user uses. Keep up the great work.

[enricoros]

Thanks @rodgermoore - being implemented right now. Big roadblock for everyone. It will introduce server dependencies, but we'll get it out quickly and then clean it up. Expect updates in weeks timeframe.

[TimeScience]

Thanks for working on this so quickly!

I just came across the documentation for environment variables... just so I understand how this is all works... would it be correct to say that the issue of api keys, etc not being retained, could be addressed by using environment variables? But as currently implemented, there isn't an environment variable that would let big-agi know where to reload all chats from? (noting that much of this will presumably be sorted out in the background once cloudsync is implemented)

[enricoros]

@TimeScience the API keys are for managed installations, where you want the backend to have the keys so every new device can already use the latest models.

The issue with the chats is that they're saved in the browser Storage.

Working on this (even in this precise moment) to have cloud Sync, so that chats and user (not server Env Vars) keys and likely personas and more will be persisted.

[dagelf]

Just adding an "All" option to the existing JSON import and export should be trivial... but I'm not (yet) familiar with the codebase at all...... will have a look nonetheless.