Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minimal yjs based collaboration #2971

Merged
merged 60 commits into from
Jan 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
8a6c5ca
wip: use sync service as provider for y.js
max-nextcloud Aug 4, 2022
92f91a5
fix: do not track state in the syncService
max-nextcloud Aug 8, 2022
8bfbbec
fix: no need to start syncService from WebSocketPolyfill
max-nextcloud Aug 8, 2022
5a0092d
fix: add default for onUpdate in createEditor
max-nextcloud Aug 8, 2022
18c5bc6
fix: handle connection errors via events
max-nextcloud Aug 8, 2022
5512f8a
fix: keep track of handlers in a way that allows to unregister
max-nextcloud Aug 8, 2022
888a754
WIP: Comment out version check in `DocumentService->addStep()`
mejo- Aug 8, 2022
bd0cc58
debug: print version and data
max-nextcloud Aug 9, 2022
827ff8c
Invoke WebSocket `onclose()` in `close()` of polyfill
mejo- Aug 9, 2022
0c5a85e
WIP: Use version from steps instead of tracking it in the document
mejo- Aug 9, 2022
fd9ebcf
Get rid of document push lock
mejo- Aug 9, 2022
ad3e377
fix: send multiple steps at a time if they are available
max-nextcloud Aug 10, 2022
8a3fefb
fix: reduce load on the server during fast typing
max-nextcloud Aug 10, 2022
3da13bb
wip: use base64 encoding
max-nextcloud Sep 19, 2022
04d51de
update: use logger api to log associated data
max-nextcloud Sep 20, 2022
d30e506
refactor: deconstruct response object in _handleResponse
max-nextcloud Sep 20, 2022
88b63ca
refactor: extract SessionApi from PollingBackend
max-nextcloud Sep 20, 2022
950ac18
cleanup: sendSteps always has a function param
max-nextcloud Sep 20, 2022
7b94433
cleanup: unused default options for sync service
max-nextcloud Sep 20, 2022
b76b66d
wip: move all api requests to SessionApi
max-nextcloud Sep 21, 2022
d4e7eeb
refactor: move sendSteps into sync service
max-nextcloud Sep 21, 2022
b0e0a0b
fix: server error handling when pushing steps
max-nextcloud Sep 21, 2022
c040bc7
refactor: move endpointUrl helper into SessionApi and tweak it
max-nextcloud Sep 21, 2022
b184b75
cleanup: old Collaboration plugin
max-nextcloud Sep 21, 2022
2ecc23a
analyse: recordings of editing session
max-nextcloud Oct 11, 2022
fc4aeff
fix: start from last saved version
max-nextcloud Oct 11, 2022
8e00942
test: yjs responses when leaving out queries
max-nextcloud Oct 17, 2022
eb67714
enh: reply to yjs sync step 1 as the server
max-nextcloud Oct 17, 2022
d3113b3
doc: add some comments from coworking with mejo
max-nextcloud Nov 22, 2022
07a9dbd
fix: and test basic session api functions
max-nextcloud Dec 27, 2022
f0c6287
enh: store yjs document state on save
max-nextcloud Jan 9, 2023
bd03735
enh: emit `opened` and `loaded` with `documentState`
max-nextcloud Jan 18, 2023
93f58a5
minor: rename sessionApi.spec.js to SessionApi.spec.js
max-nextcloud Jan 18, 2023
32b7dac
actually load documentState on start
max-nextcloud Jan 19, 2023
9abd7b6
Apply suggestions from code review
max-nextcloud Jan 19, 2023
2f33818
enh: add user colors
max-nextcloud Jan 24, 2023
b2186da
fix(y.js): Use unique identifier for websocket provider
juliusknorr Jan 25, 2023
e493862
feat(api): Return user display name on session creation
juliusknorr Jan 25, 2023
a3ea174
style(php): Fix php-cs warnings
juliusknorr Jan 25, 2023
9909cbf
fix(api): guest name for session may be null
max-nextcloud Jan 25, 2023
f755268
fix(tests): keep session alive so it can be joined
max-nextcloud Jan 25, 2023
e859d9e
fix(api): return false in hasUnsavedChanges for empty steps
max-nextcloud Jan 25, 2023
cc2cf1a
chore: Fix a few psalm errors and add typings
juliusknorr Jan 26, 2023
f1d390b
fix: Do not try to save if unavailable
juliusknorr Jan 26, 2023
c2a2e5a
fix: Also pass initial session to websocket polyfill
juliusknorr Jan 26, 2023
003ac50
style(eslint): Fix eslint errors
juliusknorr Jan 26, 2023
de6d718
tests: Fix cypress tests
juliusknorr Jan 26, 2023
483fa32
fix: Use real api name for inserting files
juliusknorr Jan 26, 2023
2cfe3d2
perf(sync): Increase step count to speed up initial document loading
juliusknorr Jan 27, 2023
b1bec3f
draft(sync): Try to avoid sending back all steps
juliusknorr Jan 27, 2023
07779ca
refactor(PollingBackend): Try to simplify polling backend a bit
juliusknorr Jan 27, 2023
6bc730f
fix(ConflictDialog): Reconnect the polling backend on resolving a con…
juliusknorr Jan 27, 2023
e644116
fix(Cursors): Show user cursor position
juliusknorr Jan 27, 2023
f20fe1f
draft: We still have no proper autosave behaviour especially when closng
juliusknorr Jan 27, 2023
7512ef2
feat(conflict): Allow conflict resultion without restarting the session
juliusknorr Jan 28, 2023
c586ea6
fix: lint errors
max-nextcloud Jan 30, 2023
ecb26c5
fix: filter out awareness updates in query responses
max-nextcloud Jan 30, 2023
b31a18e
enh(PollingBackend): only autosave every 30 sec.
max-nextcloud Jan 30, 2023
23c9e2b
fix(cypress): use same preparation to get save requests
max-nextcloud Jan 30, 2023
b49ade4
doc(DocumentService): comment on magic value "AAE"
max-nextcloud Jan 31, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions appinfo/routes.php
Original file line number Diff line number Diff line change
Expand Up @@ -35,15 +35,13 @@
['name' => 'Attachment#getMediaFileMetadata', 'url' => '/mediaMetadata', 'verb' => 'GET'],

['name' => 'Session#create', 'url' => '/session/create', 'verb' => 'PUT'],
['name' => 'Session#fetch', 'url' => '/session/fetch', 'verb' => 'POST'],
['name' => 'Session#sync', 'url' => '/session/sync', 'verb' => 'POST'],
['name' => 'Session#push', 'url' => '/session/push', 'verb' => 'POST'],
['name' => 'Session#close', 'url' => '/session/close', 'verb' => 'POST'],
['name' => 'Session#mention', 'url' => '/session/mention', 'verb' => 'PUT'],

['name' => 'PublicSession#create', 'url' => '/public/session/create', 'verb' => 'PUT'],
['name' => 'PublicSession#updateSession', 'url' => '/public/session', 'verb' => 'POST'],
['name' => 'PublicSession#fetch', 'url' => '/public/session/fetch', 'verb' => 'POST'],
['name' => 'PublicSession#sync', 'url' => '/public/session/sync', 'verb' => 'POST'],
['name' => 'PublicSession#push', 'url' => '/public/session/push', 'verb' => 'POST'],

Expand Down
5 changes: 4 additions & 1 deletion css/prosemirror.scss
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ div.ProseMirror {
&[contenteditable=false],
[contenteditable=true],
[contenteditable=false] {
border: none !important;
width: 100%;
background-color: transparent;
color: var(--color-main-text);
Expand All @@ -31,6 +30,10 @@ div.ProseMirror {
user-select: text;
font-size: 14px;

&:not(.collaboration-cursor__caret) {
border: none !important;
}

&:focus, &:focus-visible {
box-shadow: none !important;
}
Expand Down
278 changes: 278 additions & 0 deletions cypress/e2e/SessionApi.spec.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,278 @@
/*
* @copyright Copyright (c) 2022 Max <max@nextcloud.com>
*
* @author Max <max@nextcloud.com>
*
* @license AGPL-3.0-or-later
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
*/

import { randUser } from '../utils/index.js'

const user = randUser()
const messages = {
awareness: 'AQoBw9LM0gwAAnt9',
update: 'AAIKAAHYidydCwEEAQ==',
query: 'AAABAA==',
response: 'AAECAAA=',
}

describe('The session Api', function() {

before(function() {
cy.createUser(user)
window.OC = {
config: { modRewriteWorking: false },
webroot: '',
}
})

beforeEach(function() {
cy.login(user)
cy.prepareSessionApi()
})

describe('open the session', function() {
let fileId
let filePath

beforeEach(function() {
cy.uploadTestFile('test.md')
.then(id => {
fileId = id
})
cy.testName().then(name => {
filePath = `/${name}.md`
})
})

it('returns connection', function() {
cy.createTextSession(fileId).then(connection => {
cy.wrap(connection)
.its('document.id')
.should('equal', fileId)
connection.close()
})
})

it('provides initial content', function() {
cy.createTextSession(fileId, { filePath }).then(connection => {
cy.wrap(connection)
.its('state.documentSource')
.should('eql', '## Hello world\n')
connection.close()
})
})

it('handles invalid file id', function() {
cy.failToCreateTextSession(123)
.its('status')
.should('equal', 404)
})

it('handles missing file id', function() {
cy.failToCreateTextSession()
.its('status')
.should('equal', 412)
})

})

describe('step types', function() {
let connection

beforeEach(function() {
cy.uploadTestFile()
.then(cy.createTextSession)
.then(con => {
connection = con
})
})

afterEach(function() {
connection.close()
})

// Echoes all message types but queries
Object.entries(messages)
.filter(([key, _value]) => key !== 'query')
.forEach(([type, sample]) => {
it(`echos ${type} messages`, function() {
const steps = [sample]
const version = 0
cy.pushSteps({ connection, steps, version })
.its('version')
.should('eql', 1)
cy.syncSteps(connection)
.its('steps[0].data')
.should('eql', steps)
})
})

it('responds to queries', function() {
const version = 0
Object.entries(messages)
.forEach(([type, sample]) => {
cy.pushSteps({ connection, steps: [sample], version })
})
cy.pushSteps({ connection, steps: [messages.query], version })
.then(response => {
cy.wrap(response)
.its('version')
.should('eql', 0)
cy.wrap(response)
.its('steps.length')
.should('eql', 0)
})
})
})

describe('sync', function() {
const version = 0
let connection
let fileId
let filePath
let joining

beforeEach(function() {
cy.testName().then(name => {
filePath = `/${name}.md`
})
cy.uploadTestFile()
.then(id => {
fileId = id
return cy.createTextSession(fileId, { filePath })
})
.then(con => {
connection = con
})
})

it('starts empty', function() {
cy.syncSteps(connection)
.its('steps')
.should('eql', [])
})

it('saves', function() {
cy.pushSteps({ connection, steps: [messages.update], version })
.its('version')
.should('eql', 1)
cy.syncSteps(connection, { version: 1, autosaveContent: '# Heading 1', manualSave: true })
cy.downloadFile(filePath)
.its('data')
.should('eql', '# Heading 1')
})

it('saves yjs document state', function() {
const documentState = 'Base64 encoded string'
cy.pushSteps({ connection, steps: [messages.update], version })
.its('version')
.should('eql', 1)
cy.syncSteps(connection, {
version: 1,
autosaveContent: '# Heading 1',
documentState,
manualSave: true,
})
cy.createTextSession(fileId, { filePath })
.then(con => {
joining = con
return joining
})
.its('state.documentState')
.should('eql', documentState)
.then(() => joining.close())
})

afterEach(function() {
connection.close()
})
})

describe('public sync', function() {
const version = 0
let connection
let filePath
let shareToken
let joining

beforeEach(function() {
cy.testName().then(name => {
filePath = `/${name}.md`
})
cy.uploadTestFile()
.then(_id => {
return cy.shareFile(filePath, { edit: true })
})
.then(token => {
shareToken = token
})
.then(() => cy.logout())
.then(() => {
cy.prepareSessionApi()
return cy.createTextSession(undefined, { filePath: '', shareToken })
.then(con => {
connection = con
})
})
})

it('starts empty public', function() {
cy.syncSteps(connection)
.its('steps')
.should('eql', [])
})

it('saves public', function() {
cy.pushSteps({ connection, steps: [messages.update], version })
.its('version')
.should('eql', 1)
cy.syncSteps(connection, { version: 1, autosaveContent: '# Heading 1', manualSave: true })
cy.login(user)
cy.prepareSessionApi()
cy.downloadFile('saves.md')
.its('data')
.should('eql', '# Heading 1')
})

it('saves yjs document state public', function() {
const documentState = 'Base64 encoded string'
cy.pushSteps({ connection, steps: [messages.update], version })
.its('version')
.should('eql', 1)
cy.syncSteps(connection, {
version: 1,
autosaveContent: '# Heading 1',
documentState,
manualSave: true,
})
cy.createTextSession(undefined, { filePath: '', shareToken })
.then(con => {
joining = con
return con
})
.its('state.documentState')
.should('eql', documentState)
.then(() => joining.close())
})

afterEach(function() {
connection.close()
})
})

})
2 changes: 1 addition & 1 deletion cypress/e2e/sections.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ describe('Content Sections', () => {
})
cy.then(() => {
cy.getContent()
.find('h1 [data-node-view-content] span')
.find('h1 [data-node-view-content]')
.click({ force: true, position: 'center' })
.then(() => {
cy.getActionEntry('headings')
Expand Down
75 changes: 75 additions & 0 deletions cypress/e2e/sync.spec.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
/**
* @copyright Copyright (c) 2023 Max <max@nextcloud.com>
*
* @author Max <max@nextcloud.com>
*
* @license AGPL-3.0-or-later
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
*/

import { randUser } from '../utils/index.js'

const user = randUser()

describe('yjs document state', () => {
before(() => {
cy.createUser(user)
})

beforeEach(() => {
cy.login(user)
cy.uploadTestFile('test.md')
cy.visit('/apps/files')
cy.intercept({ method: 'POST', url: '**/apps/text/session/sync' }).as('sync')
cy.openTestFile()
cy.getContent().find('h2').should('contain', 'Hello world')
cy.wait('@sync', { timeout: 7000 })
cy.getContent().type('* Saving the doc saves the doc state{enter}')
cy.wait('@sync', { timeout: 7000 })
})

it('saves the actual file and document state', () => {
cy.getContent().type('{ctrl+s}')
cy.wait('@sync').its('request.body')
.should('have.property', 'autosaveContent')
.should('not.be.empty')
cy.closeFile()
cy.testName()
.then(name => cy.downloadFile(`/${name}.md`))
.its('data')
.should('include', 'saves the doc state')
})

it('passes the doc state from one session to the next', () => {
cy.getContent().type('{ctrl+s}')
cy.wait('@sync').its('request.body')
.should('have.property', 'documentState')
.should('not.be.empty')
cy.closeFile()
cy.intercept({ method: 'PUT', url: '**/apps/text/session/create' })
.as('create')
cy.openTestFile()
cy.wait('@create', { timeout: 10000 })
.its('response.body')
.should('have.property', 'documentState')
.should('not.be.empty')
cy.wait('@sync').its('request.body').should('have.property', 'version')
cy.getContent().find('h2').should('contain', 'Hello world')
cy.getContent().find('li').should('contain', 'Saving the doc saves the doc state')
cy.getContent().type('recovered')
cy.closeFile()
})
})
Loading