Phase 3: Sync Engine and Real-Time Collaboration

[youknowriad]

This is the overview issue for the collaborative editing project as defined on this post.

The goals are multiple here:

• Allow multiple users connected to the same WP-Admin to collaborate together on the same content (multiple posts/templates...) live.
• Enabling offline editing and synchronization of data.
• Offer a great DevX where developers are freed from thinking about the collaborative editing needs. Data is collaborative and synced by default and abstracted away from the UI developer.

Tasks:

• Review previews explorations (Collaborative editing using WebRTC #1930 and asblocks).
• Bootstrap the sync engine architecture and package (being experimental flag) Bootstrap a minimal sync package  #52681
• Allow persisting and loading data from a local database. (browser, offline-first data)
• Build a transport communication layer between multiple peers working on the same documents. (WebRTC based on long polling or poling?) [Try] HTTP based PHP signaling server for colaborative editing #53922
• Extensibility: Allow the transport layer to be replaced ?
• Design and represent the “presence” of connected peers in the interface. Integrate selection state with other useful parts of the interface, like the block list sidebar.
• Implement a security layer on top of the transport layer to prevent private data from being shared between peers without the right permissions.
• Lift concurrent editing restrictions. Right now WordPress locks a post to the current user that is editing it. Another user can take over, but it doesn’t allow two users working on the same post at the same time.
• Review undo / redo stacks and resolution.
• Implement the syncing behavior for all the entities of WordPress Data (CRDT representation and config).
• Employ peer caret and selection primitives that work across block types and design their semantic and visual representation.
• Explore the possibility of “following a user” as they work through a document.
• Assess performance and memory usage and impact on non-collaborative editing sessions.
• Support multi-site (both multi WP and single WP installs) local databases (document ids) Bootstrap a minimal sync package  #52681 (comment)
• Support syncing for global styles edits Bootstrap a minimal sync package  #52681 (review)
• Check that all entities are synced properly.
• Implement e2e tests for collaborative editing and offline syncing.
• Support inline comments
• Allow peers to make edits as suggestions to later be accepted/rejected.

[prajapatisagar]

Hello Team,

As a first task of this issue, I have added my findings on #23129 PR.

Thanks

[youknowriad]

Hi @prajapatisagar Thanks for the review there. Actually, the exploration has moved now to this PR #52681 That was a good reminder though, I'm going to close that old PR now.

[youknowriad]

I merged the initial bootstrap behind an experimental flag and added a number of todo list items to the current issue. You all feel free to own one of these tasks and iterate on it.

[annezazu]

Out of the list of subtasks for real time collaboration, I've created an issue for inline block commenting so there's a more dedicated space to discuss and explore: #59445

[creativecoder]

I found this presentation very informative, there may be some useful information for the collaborative sync engine:

"New algorithms for collaborative text editing" by Martin Kleppmann (Strange Loop 2023)

Specific points to highlight:

Automerge is a CRDT (conflict-free replicated data type) engine with a variety of API libraries available (including JS!). Of particular interest it the Peritext merge algorithm which is designed to handle issues specific to merging richtext/markup. I could see this being very useful for creating more elegant revisions for block content.

Screen.Recording.2024-05-01.at.16.42.29.mov

As an example using that technology, there's a prototype collaborative text editor called Upwelling, which is kind of like a combination of Google Docs and git. There's likely some design and technical inspiration there, particularly for asynchronous collaborative editing.

[annezazu]

For anyone following this work, I wanted to connect the dots to this discussion issue from @dmonad. Kevin is sponsored by Automattic to move this work forward and happens to be the author of Yjs! Two weeks ago he shared a lengthy update complete with video demos of some of what he's been working on.