My thoughts (en gross) about what I expect in such project

[donangel]

A year or so ago, when was the big social network political banning (Trump, etc.), I had ideas similar to what's PVP project is. I was frustrated with how platforms such as Patreon or GAB were fully dependent on BigTech, which could at any point take them down, together with their data. This made me think, "What about my life online?" I am a heavy online user with a lot of content out there. I'm using GitHub to store all my writing, and I'm using OneDrive to store all my (and my family's) data, etc. I own Synology NAS (old one, 214+), which backs up my cloud stuff, but still... it's critically dependent on this infrastructure.

Now, I'm not planning on doing something illegal. However, I am amazed about how the political agenda took over BigTech, so having the possibility to truly own my data and be independent of them inspires me.

I see the ideal Personal Vault Project solution for me like that:

1. Mirror of all my "social life": the posts I do with (ideally) the comments' threads - whichever is public, of course. I'm not sure if Facebook would allow that (they probably would claim that's their property), but having the ability to export the data and take it with me makes me think that it might work. Imagine a "nightly" backup of all my social network activity: Facebook, Twitter, Mastodon, etc.;
2. A mirror of all my cloud-based content. Most often, that would be documents and images, but it might also be something else;
3. A mirror of all my activities in the browser. A "mirror-client," which could store my full web history (respecting the "In Private" mode).
4. The activities on my machine. Imagine RescueTime-like log. That, combined with more detailed specific processing of my software, would make an excellent work log.
5. Contacts backup. Building up something like a 'personal CRM' system, where any data item, which could be associated with a contact, will be (see below)
6. E-mail, Calls, and Chat history and mirror: Viber, Whatsapp, Telegram, Messenger, Signal, SMS. Everywhere, where possible, the chat should be referenced to my Contacts. Here I would add the possibility of exempting an account from being backed up on PVP (to filter out work-related stuff, which should not be stored in my personal vault)
7. Protection of my data: as was described in the initial article about PVP, I should be the only one to be able to open and read the data set. It would be nice to grant someone the right to open the data set in case something happens with me, but that could be arranged by arranging a simple password/authentication transfer to this person.

I would be very happy if the system had a rich client (which would allow caching and better functionality) and web-based access. In addition, the storage could be built in a way so that I provide the storage (if I want) where my data is stored. The best solution should be able to run out of a Docker container at home. Of course, having a centrally hosted solution would also be nice, but it kind of breaks the "Big Tech independence" paradigm.

I see the system built as a centralized storage service, which will provide APIs for a few basic entities. At this point, I can think of the following:

• Contact (important since it's part of the "personal CRM" concept)
• Text Data (indexable/searchable entity)
• BLOB
• Image (yes, it's a BLOB, but since it's such a basic need, I think it may need a specific type)
• Flow (of entities or flows)

Each entity can have searchable metadata assigned to it, for example:

• Timestamp
• Contact
• custom meta tag with data to it, e.g. [Location] => [ [42.602079] [23.476818]]

Some of the metadata could be "fixed" (for example, the timestamp and the contact above), recognized, and processed by the system in a more special way.

All entities should be "chainable" (e.g., imagine what Whatsapp chat looks like) to form a more logical flow.

PVP would be best built with a plugin architecture, so it's easily extendable with specific functionalities. The base system will provide the basic APIs (storage and entity management). Everything else could be serviced on top of it.

I can continue much more than that, but it will become impossible to read. I can provide my MVP later since now everything looks like a "first priority" to me.

[dlucian]

You make a very good point Doncho.

What I was thinking initially is that we need a storage system that has both an object storage mechanism (like S3-compliant) which allows to store meta-data for an object (like you mentioned location or others), but also should be something accessible from your computer, like a network drive, where you can copy whatever files you want to copy. And then have the system attach meta data to the copied files as well.

But this gets complicated fast.

So we're going back to the original, basic requirements: (1) we want to easily dump data into this vault, and (2) we want to easily retrieve any kind of stored data.

With this concepts in mind, the simplest thing would be to go for a file-system approach, where one can dump anything, especially Google Takeout or Facebook Archive (ones entire archive of data that Google/Facebook has on them), and then the system would index that in a way that's easily searchable. For the meta-data I'm thinking hidden directories, like MacOS has the .DS_Store or Linux the .dot-files.

About mirroring the social life, some will be easier than other. For example some services have APIs, but for others, it would probably mean manually downloading an archive every week, month or quarter. Which means obviously a delay of the backed-up/searchable information.