add support for continuous hard wrap

[vlmutolo]

Continuous hard wrap

#2128 deals with hard-wrapping selected text, but ideally we'd have a way to hard-wrap as the user types. This makes for a much better experience when editing files with comments, hard-wrapped plaintext documentation, etc. All the same use cases we listed for the reflow command.

In terms of implementation, I think we could actually just basically do "reflow" on every key press (while in input mode with this feature enabled). It probably shouldn't be exactly the same, since the reflow command optimizes the paragraph globally to produce consistent line lengths close to the maximum, and it would be kind of a weird experience to have the lines reflow back and forth while you're typing. So we'd want to use the "greedy" mode from textwrap for the wrapping instead of their "optimal" mode.

In terms of performance, we'd only be reflowing one paragraph at a time, and that really shouldn't be too expensive. I'd propose as a first cut doing reflow (almost) exactly as we do it now, but on every key press. Then, as I mentioned in a comment on 136, if we wanted more performance we could try to send a patch upstream to the textwrap crate for an "incremental" wrap command that would work off of a delta and produce maybe an iterator of Cow<str> and regions that were changed.

[David-Else]

I have never used an editor with hard wrap as you type, I would have thought it would be the job of the editor to soft wrap as you type and hard wrap on save (if wanted). The hard wrapping should be handled by the LSP formatter as it would be different for every language?

[the-mikedavis]

There's the textwidth configuration from vim that does similar: https://vim.fandom.com/wiki/Automatic_word_wrapping

Automatically breaking a line (by putting the word that exceeds the line width on a new line) is the most minimal and straightforward approach to this IMO. Rebalancing lines seems treacherous: it may not be easy to tell what can or can't be broken. For paragraphs it's straightforward but not for programming languages in general. Plus it's easy to go select the last paragraph [p and reflow it :reflow if you want to have it be perfectly balanced.

I tend to agree with @David-Else that hard wrapping should be something you explicitly ask the editor to do for you (via a command or by saving). (Although I suppose if you found a way to robustly reflow all the time, that would be quite impressive and I might be convinced to change my mind.)

[vlmutolo]

@David-Else

The hard wrapping should be handled by the LSP formatter as it would be different for every language?

I was mostly thinking about prose, such as Markdown or git commit messages. I wouldn't want this applied to code. And it would definitely have to be enabled by the user.

@the-mikedavis

(Although I suppose if you found a way to robustly reflow all the time, that would be quite impressive and I might be convinced to change my mind.)

There are vim plugins that do this. I believe the one I'm thinking of is vim-pencil. The behavior was a little quirky, but I bet with the help of a robust library like textwrap we could do better.

vim-pencil handles the explicit opt-in auto-wrapping with something like :hardwrap 80 or :hardwrap toggle. It's just a mode that you can turn on or off.

[the-mikedavis]

Ah interesting, vim-pencil looks quite cool! I suppose it makes me wonder where it falls between plugin and core. On the one hand it seems pretty handy, on the other, it's a bit fancy/nuanced and I could see it being its own project. What do others think?

[sudormrfbin]

Hard-wrap as you type is wonderful for commit messages and markdown and general prose as @vlmutolo mentioned, and i think it won't be too hard to implement -- on every space character check the length of the line and if it's greater than the configured hard-wrap length, insert a newline.

[vlmutolo]

on every space character check the length of the line and if it's greater than the configured hard-wrap length, insert a newline.

I think this is also what @the-mikedavis suggested, and it's at the very least a good first-cut approach. We'd basically get to the level of what vanilla Vim does with git commit messages.

But the vim-pencil plugin takes it a step further and allows you to actually delete text and have the text after your delete reflow to fit the line. It does this just like a WYSIWYG editor would, and it's a significantly better experience, especially if you're writing a lot of prose.

Most people are probably more interested in use cases for programming, but it would be awesome if Helix also had these sorts of first-class features for regular plaintext prose. Dynamic soft and hard wrap are probably the two biggest missing pieces at the moment.

[mgeisler]

It probably shouldn't be exactly the same, since the reflow command optimizes the paragraph globally to produce consistent line lengths close to the maximum, and it would be kind of a weird experience to have the lines reflow back and forth while you're typing. So we'd want to use the "greedy" mode from textwrap for the wrapping instead of their "optimal" mode.

Yes, on-the-fly wrapping where old wrapping decisions are affected by new text is indeed a strange experience! 😄

In terms of performance, we'd only be reflowing one paragraph at a time, and that really shouldn't be too expensive.

If you haven't already seen it, you can clone https://github.com/mgeisler/textwrap/ and do

% cargo run --example interactive --release

to start a primitive interactive demo editor. It will show the time needed for every redraw and it re-wraps everything on every keystroke.

This will let you try out the weirdness of the optimal-fit wrapping mode. It looks like this in action:

[adsick]

I agree with @vlmutolo

Most people are probably more interested in use cases for programming, but it would be awesome if Helix also had these sorts of first-class features for regular plaintext prose. Dynamic soft and hard wrap are probably the two biggest missing pieces at the moment.

[pascalkuthe]

Once #5420 lands the softwrap logic there can be reused in apply_impl to hardwrap text instead of softwrap it

[filipdutescu]

I really miss this when writing commits, would be great to have it done, after @pascalkuthe's PR lands

[gamma-delta]

Hello, how is this going?

[pedrolucasp]

Gentle ping on this one.

[kirawi]

The relevant file for the soft-wrap logic is https://github.com/helix-editor/helix/blob/9df1266376323b3dae07e48bd1e64463d3aec1dd/helix-core/src/doc_formatter.rs

[matta]

I'm interested in working on this.

In terms of implementation, I think the simplistic approach @the-mikedavis suggested up thread, is the place to start. It is basically what both vim and Emacs does, and I've always found the approach to be helpful and not annoying.

I would like help on how to expose the user-facing option(s) that control the behavior. Have any design thoughts been sketched?

Emacs, which I've used for years and am quite happy with in terms of how it handles this problem, works a lot like vim. Basically, it triggers when you hit space or enter on a line. If the current line at that instant exceeds the requested text width, that line alone is "reflowed" to become one or more lines. The contents of that line, and that line alone, is wrapped. Then the space or enter key takes effect. There are edge cases to handle (e.g. when hitting Enter only the characters to the left of the cursor is reflowed and the stuff after is not), but that is the basic idea.

See:

• https://www.gnu.org/software/emacs/manual/html_node/emacs/Auto-Fill.html
• https://www.gnu.org/software/emacs/manual/html_node/elisp/Auto-Filling.html
• https://www.gnu.org/software/emacs/manual/html_node/elisp/Adaptive-Fill.html

Reflowing the entire current paragraph while typing is a neat idea, but I've not seen prior art for doing this with hard line endings in a text editor. I can see the utility while editing Markdown, etc., but in both Emacs and vim I found the "reflow this paragraph now" commands to be fairly usable, so I don't personally need the feature. It is probably a feature that should be developed as a separate option, since it is much more invasive to the buffer contents. For example, today :reflow breaks URIs. It would suck if typing anywhere in a paragraph broke any URI in the paragraph.

In comments only

Emacs can be configured to auto wrap only in comments. Does vim have a similar feature?

Is it useful for Helix? (I think yes, and this is my primary use case)

Does Helix know when the current file type has comments?

Additional argument for the simplistic approach

When writing code, it is common to write comments like this:

// This code performs a function that may be useful to you. It takes a widget
// and passes it through the frobinicator.
// Be aware that it might explode.
// TODO: eliminate the cases where it might explode.

It doesn't make sense to reflow the above, but presumably it is useful to, optionally, auto-hard-wrap after "widget" to the next line while typing.

[pascalkuthe]

A continously hardwrap implementation should use the softwrap infrastructure which already perfomantly conputes wrapping on every keypress anyway.

Reflow has many oddities (non-local wrapping which is very disorienting in an editor) and doesn't compose with other features we want to ads. It is also very inefficient to run kn every keypress. We plan to replace it with the internal softwrap infrastructure eventually. We only want to have a single wrapping implementation.

[matta]

A continously hardwrap implementation should use the softwrap infrastructure which already perfomantly conputes wrapping on every keypress anyway.

Reflow has many oddities (non-local wrapping which is very disorienting in an editor) and doesn't compose with other features we want to ads. It is also very inefficient to run kn every keypress. We plan to replace it with the internal softwrap infrastructure eventually. We only want to have a single wrapping implementation.

I take this to mean that you would like new wrapping features to use the code behind the editor.soft-wrap feature, which was written for Helix specifically, and not :reflow, which uses the textwrap crate. This sounds good to me.

Focusing instead on the user-level experience, is there a consensus that "continuous hard-wrap" should work similar to Vim/NeoVim and Emacs in the following senses:

Vim's https://vimhelp.org/insert.txt.html#ins-textwidth which says:

Long lines are broken if you enter a non-white character after the margin.

Similarly, Emacs' https://www.gnu.org/software/emacs/manual/html_node/emacs/Auto-Fill.html#Auto-Fill says:

Auto Fill mode breaks lines automatically at the appropriate places whenever lines get longer than the desired width. This line breaking occurs only when you type SPC or RET.

Same basic idea.

The soft-wrap algorithm can be used for this, but it would be good to agree on the user-level experience first.

I like the idea of vim-pencil-like behavior, but that is probably too much to begin with.

[matta]

I could use some implementation guidance. I'm stuck trying to figure out the Helix text mutation model, especially the newer parts (virtual text is pretty complex, and I'm not quite sure how Selection and Transaction relate and especially which operations are idiomatic and/or efficient).

To make it more concrete, I'm looking at insert_char and I am wondering how best to implement the logic I described in the previous post.

Let's consider the simplest possible semantics, which appear to be Vim/NeoVim when tw is set non-zero. Vim will wrap only if a non whitespace character is entered and the cursor is on a column exceeding tw.

For the moment, let's assume we want those exact semantics in Helix. How are they best implemented?

Looking at insert_char, it sometimes inserts 1 character, sometimes 2 (in the auto-pairs case), and in full generality each of those chars might take up one or two physical columns. And of course it'll do it for every Range in the current Selection.

Is there a way to take the transaction insert_char currently generates and then map every selection Range to determine if its Range.cursor has exceeded the desired physical wrap column?

And then, how do I apply the "soft wrap infrastructure" @pascalkuthe mentions? That code looks oriented toward display, and not generating changes and/or a Transaction against a doc, but I might be missing something.

(Update: Yeah, I find the logic in PRs like #10996 easy to follow. It is dealing with the same kind of checks ("does the document's text now look like X?") taking into account a pending Transaction that is confusing me)

[pascalkuthe]

This month I don't have the bandwidth to support something like this. This is not a trivial feature.

Generally the idea is to use the document formatter to transverse the line with softwrap enabled and textwidth set to something equivalent and insert a newline character document char idx of the first wrapped grapheme.

Right now softwrap infrastructure is not 100% suited for that since it always counts newline characters towards display width which is the right thing when wrapping at the edge of the screen but not for a use specified width. The changes in #6417 fix that

[matta]

This is not a trivial feature.

Yes, I figured this out quickly!

The changes in #6417 fix that.

Thanks, it is good to understand that some work is pending that should make this kind of feature easier. For the near term, I will put my attention elsewhere.