Skip to content

Latest commit

 

History

History
45 lines (28 loc) · 1.9 KB

README.md

File metadata and controls

45 lines (28 loc) · 1.9 KB
Raw

Collabs Many-User Rich-Text Editing Benchmarks

Many-user rich-text editing benchmarks for Collabs, Yjs, Automerge, ShareDB, and Google Docs. As described in the Collabs paper's Section 7.1.

Usage

Dependencies:

  • Linux
  • Node v18+

Install:

npm ci --legacy-peer-deps

Build:

npm run build

(This just runs npm run build in the server/ folder.)

See README.md in each folder for specific usage instructions.

Organization

  • server/: Web server that serves each framework's rich-text editor and orchestrates the experiment.
  • client/: Node.js program that uses Puppeteer to run a client, i.e., open the server's page in Chromium and simulate user activity. Also contains script to run a full experiment locally.
  • analysis/: Analyzes the output of clients and the server to generates metrics for the paper.
  • aws/: Helpers in case you choose to run the experiment clients on AWS ECS.
  • metadata_overhead/: Microbenchmarks to measure metadata overhead, as described in the paper's Section 7.2.

Data

  • analysis/charts/ contains LibreOffice Calc spreadsheets with the summary data used in the paper.
  • Analysis script output (including CPU profiles and saved states - needed to run metadata_overhead benchmarks) and raw data can be downloaded separately from https://cmu.box.com/s/ina8bc35c2wyu22q0tydfyaeb2c4744c.

Acknowledgments

Extensively modified from the OWebSync paper's eDesigners benchmark's code, provided by Kristof Jannes.

Simulated user activity is based on Martin Kleppmann's automerge-perf typing trace.