Detect changes to markdown syntax and open in code editing mode

[max-nextcloud]

Splitting out an idea laid out in #593 (comment) as suggested by @PhrozenByte in #593 (comment)

Background

We want to enable the best possible editing experience to users while making sure that markdown syntax is preserved for those who care about it by default. There's a long discussion about this in #593.

A lot of work is going into preserving the markdown syntax already but the architecture of texts wysiwyg editing and limitations of the underlying libraries make it close to impossible to preserve all syntax out there.

Proposal

(updated to match the current state of discussion)

• When opening a .md file... check if it would be serialized to the same code.
• If serializing the document would change the syntax open the document in code editing mode.
• Have an info, why there's no wysiwyg functionality.

Possible Additions

• It would be good to allow switching to wysiwyg - but that would not be required for a start from my point of view.
• Make the syntax match the source as close as possible. If we manage to match the syntax we can have wysiwyg more often.

Alternatives

Previous proposal

So for those documents that would be altered when saved by text I propose the following default behavior:

• detect if a file would be serialized differently and ask the user how they want to proceed:
  • open read only
  • open as code
  • open as rich text and accept the changes

The desired behavior could be customized by the user ("Do not ask me again") or per instance to avoid asking the user all together.

[dumblob]

Before deciding anything, please read something about "undo" and UX. E.g. https://en.wikipedia.org/wiki/Mode_(user_interface) , https://www.uxmatters.com/mt/archives/2020/03/are-you-sure-versus-undo-design-and-technology.php etc.

[PhrozenByte]

Thank you @max-nextcloud ❤️

Just a few additional notes:

• This approach is fully backwards compatible with previous Text versions: If a file was previously saved with Text (no matter whether it was originally created by Text, or edited later), we won't see any syntax changes and users can proceed as usual.

• This approach might yield some false positives when a user edits a Markdown file using another editor. However, something I like about this idea is that the number of false positives is tangled with the number of Text incompatibilities: we can reduce the number of false positives by improving Text's compatibility with other Markdown software.

• I'm not sure about the best UX for this... Personally I don't like "How do you want to proceed?" or "Are you sure?" dialogs. I'd rather vote for a small warning banner right above the WYSIWYG buttons and to just disable auto saving. But I'm no UX expert. Further input is needed here, maybe from our design team? @dumblob: I can't think of an "undo" approach here, can you please elaborate?

• It should be possible to disable this protection. Further discussion is needed about on which basis (globally via occ, globally via admin settings, per user, per file, per user and file, …) we need this setting. This also highly depends on the UX approach we choose.

• Whether we should enable this protection by default needs some discussion. I guess this question also depends on the number of false positives we might see...

[susnux]

So for those documents that would be altered when saved by text I propose the following default behavior:

There should be no saving without altering.

But the problem I see here is:
When is the user asked? Every time a step is send to the server?
Only when the last user leaves the session?

I do not think this is easily implemented for cooperative editing.

[dumblob]

I do not think this is easily implemented for cooperative editing.

If .md can not be made cooperative, then make .md editable non-cooperatively (which as I understand it can be made safe by default) and let the user explicitly switch to .mdc ("markdown cooperative") by clicking a button which would then ask whether the file be renamed to .mdc with the traditional "Convert? It may lose current formatting and metadata." message dialogue.

@PhrozenByte by "undo" I meant anything which does not ask any additional dialogues and allows only safe operations by default. The above "switch to collaborative markdown" approach would satisfy this.

[susnux]

Additionally: Maybe it would be at least a good first step to provide a warning on first opening a MD file.
Saying something like that the text app is not universal markdown editor but a collaborative WYSIYG editor and can not ensure that markdown files created with external editor will keep their syntax.

So users are informed and may choose another app for editing markdown or as soon as we have implemented per-file source editing of markdown files choose that option

[nimishavijay]

This seems like an interesting question for a design review call! We can discuss this in the upcoming call on our Design team public talk channel on Tuesday 29 Nov at 14:00 CET. Cc @max-nextcloud @jancborchardt who will be there and anyone else who is interested :)

[janklostermann]

If TipTap as the Markdown processing foundation of >Text< provides the information that it detects markdown or other stuff it cannot handle, a very user-friendly path would open up.

With "cannot handle" I mean: convert the original data into TipTaps internal format and save it back into its original format (here: markdown) later will result in data-loss at save (=conversion from internal format to markdown).
If able to handle it, it would create the identical file after a round-trip (just load and save)), as is probably true for a file created/edited only by >Text<.

If TipTap doesn't provide such compatibility info, maybe this is a valuable and easy to implement extension/new feature for it?

With this compatibility info from the TipTap importer available I would suggest the following options in the error message for fast risk-free usage when trying to open a not fully-compatible mark-down file as the offered choices:

• if there is a Nextcloud app installed to edit in "code editing mode": Open the file with that should be the default button (selectable by just pressing Enter) - if not, then this button would be hidden
• opening readonly or
• cancel (not open at all)

When the user chooses to open the file readonly a button could be available in the editor to switch to editing (which would convert into ">Text< markdown" and save with data-loss). This switch from readonly to editing should only be done after confirmation of a message box (default: cancel) explaining the consequences of potential data-loss that might be suffered (incl. any available details provided by TipTap).

I think, this way could avoid all the problems of the other approaches (file extension #3660, or metadata #3636). Also no performance issues with huge files due to potential additional file reading for comparisions would occur (achieving the same result by using the reading of the file that is done anyhow by >Text</TipTap). Or do I miss something?