Story Data Missing, Journal Data Preserved

[michaelarthurcaulfield]

I was working on a page, when that page was reloaded it had quite a few paragraphs missing and some paragraphs rearranged. No fork to local, just a normal page all messed up. This is similar to behavior reported by my students.

By a fluke I happened to have the original open when the new one opened, so I snapped a side by side shot:

The one on the right is the messed up one. Down at the bottom you'll notice they have very different histories, with the "correct" one showing a series of adds and edits, but the mangled one showing a series of adds with no edits.

If you look at the JSON from the mangled page, you'll see in the journal that the original edits are there in the journal, and there are no deletions. There are, however, a lot of weird nothing edits, some of which seem (maybe) malformed:

http://journal14.hapgood.net:3000/view/idea-mining

I've saved the JSON in the earlier well formed one here (much cleaner)

https://docs.google.com/document/d/1_NodiwwlU74wZedes1OMjhLnQqxhJNNae8opTuj7gwg/edit?usp=sharing

And just in case it was important, I dumped a series of screenshots from the browser console log into a Google spreadsheet:

https://docs.google.com/document/d/1oTbdQ3cjd5sAfJIw5y7WVc3xFZRF-HavqcAVTaoH3nY/edit?usp=sharing

I was using Chrome on a Chromebook (I know, I know), latest version. But there don't seem to be any errors in the log, and the data obviously made it into the journal.

Hoping this helps track this down.

[WardCunningham]

Thank you for documenting this case in such detail. Last week we had a European contributor judge wiki unusable. I'm suspecting a case where a sequence of edit actions are fired in rapid sequence.

Most notable is the RETURN handling in the TextEditor where a 500 msec pause was added as a temporary fix to continue with exploratory development.

https://github.com/fedwiki/wiki-client/blob/master/lib/editor.coffee#L12-L29

Of course my experience editing fed.wiki.org would be very different from someone editing the same code form Eastern Europe. Someone editing against a small share of an amazon server would have an experience somewhere in between.

Two years ago I was hoping that by now the reliable way forward would be obvious. That is not yet the case. One strategy is to make the edits more synchronous and put a queue in the loop. Another would be to batch whole pages and submit them in ne group that can be retried until complete. There are more choices.

With 76 followers here there are surely a few opinions. Detailed how-tos are even more welcome.

[WardCunningham]

I should mention that one approach to increased reliability is to bring the storage half of the editor closer to the interface. This is what I do when I write in a server running on my laptop and then rsync the result to the public site. See http://code.fed.wiki.org/view/welcome-visitors/view/exploring-federated-wiki

[michaelarthurcaulfield]

An interesting thing to me is that the JSON from the working one looks perfectly correct. It's journal is cleaner. And it has LESS entries which means it can't be dropped requests.
In my limited understanding, it almost feels like entries in the journal are being generated by actions the browser does not recognize as significant. Server gets sent a weird malformed edit, browser ignores it. Is that possible? I'm out of my depth here, obvi.

[michaelarthurcaulfield]

OK, weirder and weirder. I go to the affected page Idea Mining. I create a Idea Mining Scratch page. i drag and drop the journal from Idea Mining onto Idea Mining scratch to do a journal merge, and BAM! the full page I wrote appears, missing paragraphs back, things out of order put in order.
Even with the extra gunk in the journal, running the journal still reproduces the page. I hope that's a hopeful sign. It also gives us a laborious but possible way to get back lost work.

[michaelarthurcaulfield]

New instance. My Welcome Visitors page is getting log heavy. I create a clean one in a separate site by dragging elements onto a fresh one. Page looks like this:

I drag it to my other site, page redraws like so:

Some data missing, all of the story data out of order, with newer elements I've added on top.

Dragging the journal onto a scratch page in this instance does not replace lost data (as it did last time) but does restore the correct order of items on the page. Adding to this issue because falls in broad category of "Journal is right page is wrong" errors.