Skip to content
This repository has been archived by the owner on Dec 22, 2021. It is now read-only.

Db refactor #24

Merged
merged 26 commits into from
Oct 14, 2021
Merged

Db refactor #24

merged 26 commits into from
Oct 14, 2021

Conversation

jagerman
Copy link
Member

@jagerman jagerman commented Sep 14, 2021

Completely overhauls the underlying database design (see the added src/schema.sql):

  • referential integrity
  • data normalization
  • using triggers for various updates to reduce the amount of data logic in code.
  • indexes
  • one single database rather than 1 + 1/room
  • various structural improvements for useful features such as:
    • individually and default-assignable read/write/upload
    • time-based permission expiries (e.g. restrict this permission for x amount of time)
    • distinguishing between admins (who can add/remove moderators) and moderators (who can only moderate but not control the moderator list)
    • global server moderators/admins
    • global server bans
    • single VIEW for simple querying of a user's effective permissions (i.e. with database-side coalescing of global/local permissions).
    • pinned message via foreign key
    • room icon as a regular uploaded file
    • per-file file expiries (so that admins/mods can upload non-expiry files, such as the room icon, or pinned downloads).
    • message history so that when messages get edited or deleted a log is kept of the old value (the idea being that mods would be able to look this up).
    • movable messages (e.g. so there could be a "bad posts" room that only moderators have access to).

Still a bunch TODO on this:

  • existing queries need to be rewritten to query the new structure
  • migration from the multi table -> single table
  • exposing endpoints for potential new functionality will come in a later PR.

src/handlers.rs Outdated Show resolved Hide resolved
src/storage.rs Outdated Show resolved Hide resolved
src/models.rs Outdated Show resolved Hide resolved
src/rpc.rs Outdated Show resolved Hide resolved
src/handlers.rs Outdated Show resolved Hide resolved
@jagerman jagerman force-pushed the db-refactor branch 3 times, most recently from ff11c1a to 9de5be7 Compare October 4, 2021 19:02
Completely overhauls the underlying database design:

- referential integrity
- data normalization
- using triggers for various updates to reduce the amount of data logic
  in code.
- indexes
- one single database rather than 1 + 1/room
- various structural improvements for useful features such as:
    - individually and default-assignable read/write/upload
    - time-based permission expiries (e.g. restrict this permission for
      x amount of time)
    - distinguishing between admins (who can add/remove moderators) and
      moderators (who can only moderate but not control the moderator
      list)
    - global server moderators/admins
    - global server bans
    - single VIEW for simple querying of a user's effective permissions
      (i.e. with database-side coalescing of global/local permissions).
    - pinned message via foreign key
    - room icon as a regular uploaded file
    - per-file file expiries (so that admins/mods can upload non-expiry
      files, such as the room icon, or pinned downloads).
    - message history so that when messages get edited or deleted a log
      is kept of the old value (the idea being that mods would be able
      to look this up).
    - movable messages (e.g. so there could be a "bad posts" room that
      only moderators have access to).
"due to" works as an adjective that follows a noun (not verbs/verb
phrases) and is being used incorrectly in all the error messages fixed
here.
- "Deleted" messages should be set to null, not actually deleted
- Add automatic updating of "edited" to the message update trigger
- Accomodate janky deprecated Session compact_poll logic that expects
  deletion lists even if it doesn't ask for them.
- Add a useful view for (admin direct sqlite) message querying.
- Fix message_details VIEW query
Uploaded room images shouldn't expire; this makes the expiry column
nullable and sets their expiry to null, and adds a trigger to expire
replaced room images.

Also fixes a panic in file deletion that made it not actually delete
files.
Trim padding when storing and repad on the fly when retrieving.
They have to be defined after the tables on which they run, of course.
Also fixed some syntax errors in them and made them work.
The old Room response included the room token as "id", which is quite
awkward, and not something we want to propagate into the future API, so
we can't just start returning the extra details if we don't want to
maintain that misnaming.

This changes the API so that existing (deprecated) endpoints get the
deprecated field, and the new Room deserialization is free to break
backwards compat.
- Rename `info_updated` -> `info_updates` to be consistent with `updates`.
- Relax triggers to only update `info_updates` instead of both updates
  and info_updates, and fix some trigger statements that were updating
  the wrong value.
- Add table for tracking (multiple) pinned messages.
- Trigger info_updates update when pinning/unpinning.
@jagerman jagerman marked this pull request as ready for review October 14, 2021 13:13
@jagerman jagerman changed the base branch from main to dev October 14, 2021 13:13
@jagerman jagerman merged commit e229a69 into oxen-io:dev Oct 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants