Add metadata to files indicating they were created / edited by text

[max-nextcloud]

Make NC Text "tag" each .md file it edited by adding some header string to the top of the content of the .md file.

When opening .md NC Text would first look up this tag and if it were not there, NC Text would open it read-only. The user would then be presented with a button "reformat & edit" button telling the user it would lose certain formatting after clicking. Then the file would be tagged. Any newly created .md file would be automatically tagged to keep it user-friendly as it is now.

Basically the same as #3477 but with explicit tagging as part of the content of the .md file.

Originally posted by @dumblob in #3630 (comment)

[bronson]

This sounds like a great solution.

When the file is first displayed read-only, maybe it could present two options:

• Reformat and Edit
• Edit as Code

As long as the tag is unobtrusive, this should make just about everyone happy. (is that right? the only people I know of who would still be unhappy are those who expect simple text files to show without delay, which is unrelated to this issue)

Frontmatter seems like a good place to put the tag.

[bronson]

The tag will probably need a "PREVENT REFORMAT" or "ALWAYS EDIT AS CODE" setting for when you have .md files that are incompatible with the Text app, and you want to prevent other users from accidentally losing the formatting/data.

[PhrozenByte]

I'm afraid it doesn't really make any difference whether we store this information inside the file, or within the file extension (e.g. .nc.md, see #3630): Since other editors don't know about this metadata, one can still edit files created by Text using arbitrary (both Markdown or plain text, both web-based and desktop) editors and cause the exact same compatibility issues as before. See #3630 (comment)

[bronson]

I'm not following. The scenario that is being discussed is:

• create .md file elsewhere
• open file in the Text app, Text app automatically reformats file
• discover that file has suffered data loss

If the Text app makes sure to only reformat files that have the tag, or explicitly by permission, then how will the Text app cause data loss?

[bronson]

Ah, I think I understand the scenario you're worried about:

• create .md file elsewhere
• open file in Text app. Text app asks permission to reformat. You say yes.
• file is saved reformatted and tagged
• someone else opens the file in a different editor and makes incompatible changes WITHOUT removing the tag
• finally, someone opens the file again in the Text app. Because the file is tagged, the Text app reformats it. Data is lost.

That's true. But at least that's much MUCH better than the situation now.

And hopefully the tag can read something like "NC-TEXT: AUTOMATICALLY REFORMAT THIS FILE" so it's obvious that, if you're hand-editing it, you'll want to remove the tag.

[PhrozenByte]

This basically happens every time someone chooses to edit a Markdown file with another editor after it was touched by Text. Relying on the user to understand and remove the tag manually is very problematic from an UX perspective. I can't really imagine how this is supposed to work for the overwhelming majority of Nextcloud's users, not being aware of what this tag is even about. Since we're talking about users outside of Nextcloud's scope, users might not even know that Nextcloud is involved, or what Nextcloud Text even is. But I'm no UX expert and maybe the design team has some input on this.

Another major issue is that this "compatibility tag" breaks other Markdown software. At best, other Markdown software will show an odd "NC-TEXT: AUTOMATICALLY REFORMAT THIS FILE" paragraph at the beginning of the file, but worse, the tag might break the usage completely, e.g. software expecting a YAML Front Matter at the first line of the file. As an example, these files will break Pico CMS for Nextcloud. In a nutshell, we add yet another non-standard resp. standard-violating characteristic to otherwise valid Markdown files, causing another string of compatibility issues.

[bronson]

The problem that this issue is trying to solve is "I copied a plain .md file into Nextcloud, used the default Text app with default settings, and now it's ruined." This proposal does solve that.

If the issue becomes "Someone was editing a file in the Text app and I went out of my way to edit it in a custom editor and I ignored the docs and added some incompatible syntax and neglected to remove the tag and NOW it's ruined" then that's actually a big improvement, isn't it?

It sounds like you're insisting on a perfect solution. The only 100% lossless solution that I can think of would be for the Text app to stop using Markdown as its storage format... which is probably not what you're advocating? I doubt the various WYSIWYG Markdown editors will ever be able to interoperate with zero chance of conflict.

[bronson]

As for the paragraph you added by edit...

I don't understand what you mean, "this tag breaks other software"? Nobody's actually made a proposal beyond "using frontmatter seems like a good idea" have they? If you know something more about this proposal will be implemented, it would be nice if you would describe what you have in mind.

[PhrozenByte]

I'm just pointing out the disadvantages of this suggestion. This idea only provides improvement for a very limited use case. That's a good thing at first. Since it feels like that this use case happens to be yours, I can totally understand that you want to push this idea, but please understand that we must keep other users in mind, too. Thus we must also consider the negative effects of such a change - and then trade off the advantages against its disadvantages. If you feel personally attacked please excuse me, that's not my intention.

In a nutshell, I've pointed out that I see two major flaws with adding such a "compatibility tag" to Markdown files:

• It only works for files that were either never touched by Text, or only touched by Text; any user trying to use Text together with other Markdown editors, will still have a bad time, or even a worse time (see next bullet) if this is implemented; this happens to be the same issue as with changing the file extension (see #3630)
• The "compatibility tag" will, howsoever it might be implemented (in the absence of any specific suggestions we might discuss specifically), cause incompatibilities with other Markdown software, as it is a non-standard resp. standard-violating characteristic of otherwise valid Markdown files; specifically, other Markdown software won't understand this compatibility tag and thus either show it to users being totally unaware of the situation, or even break the file's usage completely (depending on the specific implementation and the other Markdown software involved)

Whether there are better ideas to solve the superordinate issue doesn't really matter at first: the advantages of an suggested change must outweigh its disadvantages. But it happens to be that we indeed have multiple better solutions for the superordinate issue. The first being #3630, specifically using .nc.md as file extension, as it isn't prone to the disadvantages outlined in the second bullet above. The second being #3477, suffering none of the above.

By the way: No idea what paragraph you're talking about I supposedly added by edit?

[bronson]

any user trying to use Text together with other Markdown editors will still have a bad time

Agreed, and that's an inherent propperty of using Markdown as a save format for WYSIWYG editors. I don't think this issue can change that, positive or negative.

This is just meant to reduce the chance of "I copied a Markdown file into Nextcloud and Nextcloud ruined it."

The "compatibility tag" will, howsoever it might be implemented (in the absence of any specific suggestions we might discuss specifically), cause incompatibilities with other Markdown software

Maybe? Until we have a concrete proposal to discuss, I don't think it's possible to determine the relative merits of implementing it or not.

That said, I'm certain it's not as bad as you claim, and the incompatibilities that crop up probably won't include data loss. I don't know how you can be worried the tag can "break a file's usage completely." If corruption does happen, that would almost certainly be due to a bug in that particular 3rd party editor, wouldn't it? Markdown editors are expected to work reliably and without loss regardless of how screwed up the input is. Again, that's just Markdown.

I'm sorry for assuming you added the paragraph by edit. Your second paragraph appeared after I hit the "Comment" button. Now I see that's clearly that's not in your post's edit history so... I don't know what happened. Thanks for pointing that out; good lesson on double checking for me.

[blizzz]

Commonly Front Matter is used to prefix documents and enrich with meta data. E.g. used in pandoc for convertion, but also by blog engines. Cf. #3571 I am aware of any common compatibility "header" or "tag", but it could be an initiative to bring it in, and editors could adapt to it. That would make it potentially not a Nextcloud-only solution, even if it starts as one.

Apart of this, I am afraid this might be too complicated for unaware users, sometimes maybe unnecessary (simple, compatible documents). Would a different, or additional option be inspection? Trying to see whether only compatible formats are present, or known problematic structures are detected, and use this into consideration for warning users?

[PhrozenByte]

Trying to see whether only compatible formats are present, or known problematic structures are detected, and use this into consideration for warning users?

Just for the record: This suggestion is tracked in #3477

[suntorytimed]

I used Mindforger as a note-taking app for quite some time, which utilises a markdown flavour itself. It uses metadata tagging, which it interprets in software, but can be omitted in other markdown editors. Maybe this would work for NC Text as well? It looks like this:
## Headline <!-- Metadata: type: Note; tags: meeting-minutes; created: 2023-05-30 13:38:46; reads: 14; read: 2023-06-06 16:19:36; revision: 5; modified: 2023-06-06 16:19:36; -->

As an example rendered in HedgeDoc:

Nextcloud Text is one of the few markdown renderers, which can't interpret it, but it also doesn't break the syntax: