Distributed systems study

Ideas.

• CRDT's
• Kademlia, DHT's.
  • Sub-Second Lookups on a Large-Scale Kademlia-Based Overlay
• P2P systems:
  • IPFS
  • Incentives Build Robustness in BitTorrent
• Theory:
  • FLP impossibility.
  • Byzantine (Albanian) Generals problem.
  • Safety, liveness, and consistency
• Paxos.
• Raft.
  • In Search of an Understandable Consensus Algorithm
• Google systems:
  • Bigtable
  • Chubby
  • GFS
  • GFS evolution https://queue.acm.org/detail.cfm?id=1594206
  • Building Large-Scale Internet Services
  • Spanner, Megastore
  • Storage Architecture and Challenges
  • Mapreduce
• Facebook systems
  • Blob storage https://distributed-computing-musings.com/2023/12/paper-notes-f4-facebooks-warm-blob-storage-system/
• Amazon systems
  • Cassandra / DHT
• Two-Phase Commit
• Two-Phase Lock

Weird ones:

• https://web.archive.org/web/20200630133111/http://rystsov.info/2020/06/27/pacified-consensus.html

Code.

• TLA+ specifications of Raft.
• Minimal Python implementations of Raft/Paxos.

Engineering.

• The Tail at Scale
• Microservices and the First Law of Distributed Objects
• Fallacies of distributed computing
• gRPC principles
• SSTable, Memtable, B-Tree, LSM Tree

Academic Papers / reading list.

• https://muratbuffalo.blogspot.com/2021/02/foundational-distributed-systems-papers.html

Blogs / writers.

• https://distributed-computing-musings.com/page/7/ - great overview
• https://decentralizedthoughts.github.io/videos/
• https://muratbuffalo.blogspot.com/2020/06/learning-about-distributed-systems.html

Software.

• LevelDB
• RocksDB.
• SQLite
• RQLite - distributed sqlite.
• FoundationDB - bought and open-sourced by Apple in 2015

Scraps

Google Spanner runs two-phase commit over Paxos

https://stackoverflow.com/questions/59243408/why-does-google-spanner-use-both-two-phase-commit-and-paxos

To understand partitions, we need to know a little bit more about how Spanner works. As with most ACID databases, Spanner uses two-phase commit (2PC) and strict two-phase locking to ensure isolation and strong consistency. 2PC has been called the “anti-availability” protocol [Hel16] because all members must be up for it to work. Spanner mitigates this by having each member be a Paxos group, thus ensuring each 2PC “member” is highly available even if some of its Paxos participants are down. Data is divided into groups that form the basic unit of placement and replication.