More flexible text system integration

[mattmassicotte]

One of the core goals of this library is to make it text-system agnostic. But it still has a few limitations in that area. Specifically:

• It relies on Apple-built text view behaviors in a few areas
• The TextInterface type is a protocol and that brings along a number of problems (like overlapping method/var names for conforming types)
• Support for multiple cursors is tricky

Not directly related to this, but it would also be cool to break the dependency on TextStory.

I've begun addressing these on a branch. But I quickly discovered that getting rid of TextStory is hard. This because a number of filters and the indentation system rely on some non-trivial string query capabilities with that package.

That core functionality has actually been useful in many of other places. So, as a first step, I'm considering further decomposing that library. This kinda makes sense, since I'd like TextStory to remain focused on NSTextStorage anyways.

But that would mean TextFormation would still have at least one dependency. And it isn't yet clear how painful it will be to not have access to Rearrange. So, from a strictly "less dependencies is better" point of view, it could be a lateral move.

@thecoolwinter, what do you think?

[thecoolwinter]

I like the idea of splitting from TextStory, but I'm not sure how necessary it is as you'd still need to have some string mutation protocol anyways.

I do think however that making a more 'functional' API might be helpful in solving some of the issues with relying on Apple text implementation, and make it more flexible for scenarios like multiple cursors.

What I mean is the filter API takes two parameters each time it's run:

• Text storage (or at least some reference type string)
• Mutation to apply

And it returns:

• Whether or not to apply the mutation (same enum as before)
• Array of mutations to apply (in order)
• Suggested cursor range

Which gets rid of any state and leaves the filters ready to act on whatever is needed without worrying about cursor management or any other storage specific stuff. That would allow the implementer to update the cursor location and work with multiple cursors by running the filters once per cursor and updating accordingly.

[mattmassicotte]

Yeah, you're right. Separating them could incur a little duplication.

I have to look, as I'm not yet sure if all the mutations a filter can make can be described in an independent way. But, I really like the suggestion!

[mattmassicotte]

Just wanted to provide an update, since it's been a while. Still going to do this.

[mattmassicotte]

holy hell it's been more than a year. But I'm finally at a place where I can return to this and put some real work in.

[mattmassicotte]

Ok I have actually begun the real work. I have a prototype of an idea that supports a way more flexible system. Unfortunately that does come with some trade-offs, in that there's more work required for integration. But I think this is worth it.

Along the way, I'm also planning on breaking the TextStory dependency.

I'd love to get your thoughts on this @thecoolwinter. You can see the general idea here:

TextFormation/Sources/TextFormation/Filter.swift

Line 73 in 95df87b

func processMutation<System: TextSystem>(_ range: System.TextRange, string: String, in system: System) -> System.Output?

[thecoolwinter]

More work integrating wouldn't be too big a trade off. I like the new direction, are you planning to have the new filters still apply their changes in the handler? I'm wondering if it would be better to have it return a replacement string along side the range and delta to allow the text system to apply it outside of the filter.

[mattmassicotte]

Yes, changes can still be directly applied. But how a change is applied is totally under the control of the TextSystem thingy. This makes implementing some filters easier, but I don't (think) it is absolutely necessary. What were you thinking about that prompted that idea?

[thecoolwinter]

I was prompted by a problem we have right now where it's hard to tell the undo manager which changes are for which cursor. But, looking again I think that problem is still solvable with the new system. Correct me if I'm wrong but this would no longer require using the (shouldChange -> Bool) delegate method to discard edits? If that's the case we can move filter application to after edits are applied, which would be perfect.

[mattmassicotte]

Ahh yes! I think it's probably possible to still use that delegate method as a hook. But fundamentally I don't think that is flexible enough.

However, I don't think you can apply filters after edits in the general case. The way I am starting to use this is by apply filters to each individual cursor, and then using the output deltas to fix up cursor positions that are further along within the storage. But I'm still playing around with those ideas, as well as lazy cursor mutations, so I don't have any final ideas there.