TODO.txt

Notes / Outstanding questions:
    Consistency Model and context compression:
        Note: An index is a tree of Entities. Other non-index-related Entities may form a graph. Removing entity head X from the context
            manager upon its descendent(or identical) reference by N known paths is sufficient to preserve the consistency invariant
            *only* if there are N total possible paths to Entity X. If it's possible that there are N+1 total paths, we have the
            potential for an invariant violation, as the N+1th path may not reference a sufficiently recent head. As such, we must either
            abstain from removal of Entity head X from the query context until all possible referents can be assured to posess
            a descendant or equal relation head (easy to determine for trees, IE: the root index), OR we must supplement the relationship
            travesal with a root_index traversal.
        
        1. Does it make sense to create different classes of Entities, such that the tree/graph differentiation is codified?
           At the cost of increased potential for conflicts, this would improve safety by preventing illegal writes, and remove the burdon
           of the decision from the querying party. (What if A -> B then somebody else writes X -> B? is this a real risk?)
        2. Does the context compression scheme still make sense as designed, given that consistency model enforcement requires a root_index lookup for
           every (multipath capable) relationship traversal?
        3. Is it possible that the realities of incremental index compression + probablistic merging might help to improve average relationship traversal
           efficiency of the above (specifically the root_index traversal) by way of a tendency to have root index fragments still resident in the context?
           (root_index traversal skipping when leaves are resident?)



Steps remaining prior to topic/topo-compression3 merge:
 [ ] Consider edge / relation nomenclature
 [ ] Think about relation concurrencies, real use cases. LWW seems like pretty weak sauce here
 [ ] Think about edge concurrencies. Balanced-tree seems like a good exercise
 [~] Edge/Relation bifurcation
 [ ] The thing compiles
 [X] Entity type differentiation and detection
 [ ] Entity subscriptions
 [ ] Context compaction
 [ ] Observer interface
 [ ] Channels for Slab -> Context -> Entity
 [ ] Clean up Factorization diagram 





Steps Believed necessary for the passage of the first self-consistency test case:

[X] Arc-ify SlabRef ( necessary because of all the memorefs that will be pointing to it for peering management )
[X] Enable MemoRef management by the Slab
  [X] For the memo itself
  [X] For memo -> parent memos
[X] Finish renaming Record -> Entity
[X] Finish basic entity subscription plumbing
[X] Implement rudimentary simulator
[X] stop endless memo emission / bouncing
[~] Entity indexes
    [X] Dummy entity head index
    [X] All received memos get mindlessly jammed into entity head index
    [X] entity head index is actually pruned to leave only the head memos
    [X] entity head index is itself built from memos
    [X] bootstrap entity index which requires the entity index (turns out to be simple graph traversal. The index doesn't use index, duh)
    [ ] local subscriptions should be no different from nonlocal subscriptions§
[X] memo body
[X] memo peering
[X] Memo -> relation memos
[X] memo retrieval ( slab.localize_memo )
   [X] Return found Memo
   [X] Memo not found response (Participating/NonParticipating)
[ ] basic memo durability scoring
[~] Basic state projection logic
    [X] Extremely basic Causal LWW hash keys
    [X] Extremely basic LWW relationship-supersession
[~] Peering Memos
  [X] inter-slab peering memos
  [ ] Consolidate multiple peering changes into a single peering memo
  [ ] Emit Goodbye memos for slabs that are shutting down
  [X] Local Peering Memos?
  [ ] Determine if Entity-Peering is necessary for indexes (peering which is silently subsumed by newer memos)
[ ] Hash-based Memo ID generation
  [ ] Lazy hash generation
      Memos which are never transmitted to another slab should never have MemoIds assigned.
      Hard linkage precludes the need for MemoId generation. Only necessary when transmitting.
  [ ] Commutatively merge-able Entities:
      hash of: type (edit/keyframe?), values, tier/context, *fully materialized references*
      How do we prefer a keyframe memo vs the head of a chain of edit memos?
      They'll have identical checksums.

      Think about: is this still a Merkle tree?
      (context root -> child -> child -> leaf: yes)

      Barring a cryptographically secure commutative
      hash, the necessary commutative property is defeated by inclusion of the precursors,
      which will vary dramatically.
  [ ] Standard Entities: hash of values, references, parent
[ ] Very basic testing
  [ ] basic asymmetric communication scenarios ( memo conveyed from SlabA->SlabB but not C, etc )
  [ ] basic data loss scenarios
[ ] Cleanup
  [ ] Memo / MemoRef serialization across slabs ( Memorefs may get leaked at present )
  [ ] Differentiable data types
  [ ] durability scoring buckets (start with fixed number of buckets, later make it variable)
  [ ] Remove unnecessary memo relations arguments
  [ ] Update SlabRef to accept different kinds of channels, not just simulator channels
  [ ] Move XYZPoint out of network::Sender into slab
  [ ] Determine how to make second order effects deterministic
  [ ] Revise test suite with more robust spatial scenarios
  [?] Slab arc vs SlabRef arc - eliminate unnecessary double-Arc and switch all Slab handles to SlabRef handles?

 _________________________________________________
| SLAB                                            |
|  MemoRef A (peers) -> Memo A                    |
|                       |                         |
|                    MemoRef B (peers) -> Memo B  |
|_________________________________________________|


Durability Buckets:

Bucket 1 (min,max) [
    MemoRef (peers,score),
    ...
]

MemoPeerRef (peer,score) ?