Add support for editing and rendering multiple documents at once

[johnblommers]

So Scrivener has this feature and I think it's wonderful. You select several scene documents in the document tree, the contents appear in the proper order in the editor pane, each separated with a tasteful horizontal line, and you can edit what you see.

This would be very helpful when you have several scenes in different chapters that all occur at the same place. And you want to make sure that your descriptions are consistent.

At the same time it would be wonderful if the right hand pane contains the realtime rendered version of the Markdown text in the scenes without the need to press Ctrlr.

[vkbo]

Hmm, I'm not sure I will add this. It deviates a bit from the principle of being a simple one-at-a-time document editor. I will add it as a potential future feature for now.

I've split the view to a separate issue #103, as this has also been requested by @Number042:

The view pane is not really intended for seeing the rendered content as Markdown is itself meant to be human readable. It is mainly intended for having references and related documents open while writing.

I propose adding an option to enable real-time rendering of the view pane document if the two editors are showing the same file.

The main reason this wasn't added in the past is that it would require the rendering of the html code to be done by a separate thread as it may interrupt the text editing in the edit pane if the document is large.

[johnblommers]

A corollary to this enhancement request would be that if the user clicks on a chapter heading in the tree pane then all of the contained scenes be displayed in the editor pane. And when the user clicks on the novel heading then the entire novel is displayed in the editor pane.

[vkbo]

Hmm, not sure what you mean by "chapter heading" here? Since the tree doesn't display headings, just file labels. If you mean folders, they are not structurally chapters. You can use them to organise chapters of course, but novelWriter doesn't know that. Personally, I use them to separate the stages of the story progression, which matches how I've sectioned up the plot description.

But I have thought about making the view panel behave just like what you suggest. I doubt multi-doc editing will be implemented, as it diverts from the simplicity idea, but I will consider it again when the core functionality is in.

More important in the short term is the merge and split document features. They will be hard merges and splits, but can in principle be used to do what you suggest.

[johnblommers]

You have caught my meaning. My approach is that a folder is a chapter and a document/file contains a scene. It's how I think about novel structure. That allows me to rearrange scenes and chapters directly in the tree pane. To me that's huge.

This explains why I asked for the synopsis feature because, when each scene has a synopsis, it makes fare more sense to rearrange the scenes based on a synopsis rather than based on a title.

I've come to appreciate that it's the H1 and H2 and H3 et. al. headings actually define the novel structure: title, chapters and scenes. Indeed I initially used the import feature to pull an old NaNoWriMo manuscript written in Markdown into novelWriter. But I stopped there because I was disinclined be splitting the material into folders and documents.

Let me completely agree that:

More important in the short term is the merge and split document features. They will be hard merges and splits, but can in principle be used to do what you suggest.

[vkbo]

You have caught my meaning. My approach is that a folder is a chapter and a document/file contains a scene. It's how I think about novel structure. That allows me to rearrange scenes and chapters directly in the tree pane. To me that's huge.

Oh, sure. That is the entire reason why I made novelWriter in the first place. Otherwise, I was perfectly happy to use for instance GhostWriter, which is an excellent markdown editor, and significantly more pleasant to work with than office-type document editors. But its single-document focus made it tricky to use with my workflow.

This explains why I asked for the synopsis feature because, when each scene has a synopsis, it makes fare more sense to rearrange the scenes based on a synopsis rather than based on a title.

I'm working on a main window outline view, which displays the novel as a tree, following the H1 to H4 headers rather than folders. This view can have any meta data column visible, including synopsis. It's similar to what yWrite does.

I was not planning to make it drag-n-drop enabled though. That vastly complicates things as a section isn't necessarily a file. Let me complete the basic outline view first, and then we can look into ways to improve it to support various work flows.

[johnblommers]

I think that a main outline view that follows the H1 to H4 headers would make an excellent navigation pane. Click on a heading and it appears in the editor pane.

Assuming that the document tree view is visible concurrently with the navigation pane, one could drag and drop documents and folders in tree view and the navigation pane would ideally update itself automatically.

This would be huge.

[vkbo]

Yes, that is essentially what I'm working on. They will both be visible. The outline will be on a tab occupying the main work area, and the editor and viewer will be on another tab. Having that much space available, I can essentially make any meta data in the index available, and the user can select what they want to see.

Exactly how I handle updates depends in how sluggish it will be if I just regenerate the view on each change, or if I need to write some clever diffing implementation. A normal novel project shouldn't contain more than up to maybe 200 structure elements, so I'll do some performance tests.

[vkbo]

This more or less overlaps with my latest comment in #185. I'm also working on a test case for #541, which may or may not be included in the 1.1 release. That feature would allow an alternate tree view where one can browse and open files by the chapter/scene header like in the Outline tab, but without having to move to that tab first.

[dadoprom]

Hi, this feature was the best thing of Scrivener, and still there is no text editor with such function available in linux... so I really hope it will happen in your app :) thanx for your hard work :)

[vkbo]

With the way novelWriter is written, this is difficult to do in the editor. The editor is a single unit that can only exist once, and it contains a single editable area.

In a separate piece of code I've tested if I can stack multiple text editors end to end, in order to open multiple documents for editing. It sort of works, but as soon as you start stacking editor windows within a layout, it no longer grows properly as you add text. The size of the edit area will instead have to be resized programatically.

I think this will be easier to achieve in QML, as it is much much more flexible with respect to how GUI pieces flow and scale.

As for allowing rendering multiple documents in the view panel, this is fairly easy. It is essentially what the Build Novel Project tool does anyway. Being able to display multiple scenes on the side, while editing a single scene in the editor, will bring you half way to the goal I think.

[dadoprom]

Well, I am not a programmer, so I do not know how to do it... but they somehow managed to do it in Scrivener...

But still this is a good start... thank you :)

As for allowing rendering multiple documents in the view panel, this is fairly easy. It is essentially what the Build Novel Project tool does anyway. Being able to display multiple scenes on the side, while editing a single scene in the editor, will bring you half way to the goal I think.

[vkbo]

Well, I am not a programmer, so I do not know how to do it... but they somehow managed to do it in Scrivener...

Since I never responded to that: novelWriter is limited by the framework that is used to build the user interface. The editor in that framework is very limited, and making a new one isn't really an option. I don't have the time and resources of a commercial software company. This app is only possible because frameworks like Qt exist in the first place, but that's also a limitation. Other apps with advanced features write their own editors from scratch.

Anyway ...

I've been looking at this problem again. One way to make it possible to edit multiple documents is to simply concatenate them when opening in the editor, and leave some magic line of text that separates them. However, the risk here is that if the user removed that divider, or otherwise messes it up, the information is lost and the text cannot safely be saved back into its respective documents.

I've looked at ways to make a line read only (uneditable), which works to a point, but I have to block every possible way a line can be edited, and if I miss any, there is still a chance of messing it up. I'm not sure this is an ideal approach either.

[BlayW]

This is a feature I would also love to see. I know it's been a while since this comment, but I think your suggested implementation could work. I appreciate the issue you bring up about the read-only marker in the text, this could (and probably would) be messed up by the user somehow.

One thing I'd consider is having metadata that is tracked while multiple documents are open. This metadata would track what line number each document begins on and updates as the user adds lines anywhere in the combined document before this line.

I imagine an invisible tracker like this wouldn't be very costly and would prevent accidental tampering by the user.

A visual marker would probably still be useful, but not be depended on.

[vkbo]

One thing I'd consider is having metadata that is tracked while multiple documents are open. This metadata would track what line number each document begins on and updates as the user adds lines anywhere in the combined document before this line.

I imagine an invisible tracker like this wouldn't be very costly and would prevent accidental tampering by the user.

This runs into the same problem as the marker though. It has to be maintained for any possible way of editing, including deleting or pasting large blocks of text. The additional downside is that it's also invisible, so the user won't even know they've messed it up. It's also hard for the editor to know if something was a mistake or intentional.

The editable marker is actually better in that it is very obvious, and the user can even move it to change the target document for the following text. Adding a check when saving that all document can be written to would help avoid some mistakes.

At the end of the day, I think the better solution is to add a way to allow more than one document to be open. It avoids all these problems. With my own writing, the main use case is that I need to edit a note in parallel with writing a scene. If the editor and viewer panels weren't fixed, and instead one could open up to, say, four documents in either edit or view mode in different panes, it would solve the main use cases.

Multi-document viewer is coming. That part is a lot simpler to achieve. Also, from 2.6, it will be possible to drag and drop documents from the project tree to the current editor and viewer panes, and that would also make it easier to allow multiple free panes to exist.

[peter88213]

Indeed I initially used the import feature to pull an old NaNoWriMo manuscript written in Markdown into novelWriter. But I stopped there because I was disinclined be splitting the material into folders and documents.

@johnblommers: Is that not possible? I'm not sure, but I figured there was a conversion function for this. If not, I once wrote a Python script that splits up a Markdown file and creates a novelWriter project with one file for each scene. Maybe you want to try it: https://github.com/peter88213/md2nw/.

[vkbo]

It is indeed possible. However, with the Novel View, as an alternative to the Project View with the project tree as documents, you can navigate the content of larger documents by its headings, so there isn't really a need to. John's comment is quite old, and made before that view was added.

A design choice of novelWriter is that there are no restrictions on how you split your documents. It's a matter of preference. I like to work in individual scenes, which lets me use status labels to indicate progress and rewrite/draft stage, but I know of authors who put multiple (shorter) novels in a single project, and use one document for each. Myself, I tend to start with four documents for a three act story (two documents for act two, split at the story midpoint) and use those for outlining the main parts before splitting it up further when I start writing scenes.