Wrapping

[VadimOsovsky]

Hey @znjameswu ! Awesome library! Thanks for making such amount of work publicly available!

Currently the whole equation is rendered on a single line, are you planning (or is there already) a built-in support for wrapping?

Thanks! Have a great day!

[znjameswu]

Thank you for your interest in this library!

Yes, wrapping (Breakable equations) was originally planned and even prototyped in this repository in very early stages (but later removed). It will be added in future, probably very soon.

The major blocking issue is how to design those render contracts. I currently can only think of completely rewriting custom versions of RenderBox and BoxConstraint, but that leads to very bad code quality. If you have any thoughts on this, or if you know any exisiting Flutter library that handled this correctly, please let me know. It will greatly accelerate the development of this feature.

[VadimOsovsky]

@znjameswu What I'm going to do about that as a workaround is just split my katex into chunks, render them separately and put it in the Wrap widget. I also split normal text into each word and put each word into it's own Text widget, so I could insert katex inline. That's the best idea I can think of.

[znjameswu]

@znjameswu What I'm going to do about that as a workaround is just split my katex into chunks, render them separately and put it in the Wrap widget. I also split normal text into each word and put each word into it's own Text widget, so I could insert katex inline. That's the best idea I can think of.

That is how KaTeX renders equation, but is very hard to achieve with current Flutter Math's design.

For example, the height of left/right delimiters depends on the highest element within the delimiters. In Flutter Math, I decided to leave the height/width calculation to Flutter's layout pipeline rather than precalculating them from AST. So I have to wrap the entire thing in a RenderObject to share layout result, which causes problem for line breaking.

Also, wrap the entire node within one RenderObject is very helpful for future editing implementations. So currently my plan is still to render them as one RenderObject and then develop contracts for line breaking.

[MidhunrajXYZ]

Using a Wrap widget is a workaround. Spit the Tex equation into multiple smaller ones and make them as children of Wrap. Probably not efficient but worked for me.

        Wrap(
            direction: Axis.horizontal,
            runSpacing: 5,
            spacing: 4,
            crossAxisAlignment: WrapCrossAlignment.center,
            children: [
              Text('What are the'),
              Text('solutions of'),
              Text('the quadratic '),
              Text('equation: '),
              Math.tex(
                r'x^2',
                textStyle: TextStyle(fontSize: 25, color: Colors.black),
              ),
              Math.tex(
                r'+6x',
                textStyle: TextStyle(fontSize: 25, color: Colors.black),
              ),
              Math.tex(
                r'+8',
                textStyle: TextStyle(fontSize: 25, color: Colors.black),
              ),
              Math.tex(
                r'=0',
                textStyle: TextStyle(fontSize: 25, color: Colors.black),
              ),
              Text('? '),
              Text('(find both'),
              Text('solutions)'),
            ],
          )

Need to split the rest of the text too. Otherwise inline wrapping will not be perfect.

[znjameswu]

Wrapping will probably be on top of the queue after NNBD migration. However I'm no expert on line breakings. I'll talk about my planned implementation. It is probably hugely flawed, so if you find any mistake or know this better, be sure to let me know.

Background

According to my knowledge, the most common solution for line breaking is to first introduce a centralized layout manager, TextPainter for example. The manager processes all the content and break them into smaller unbreakable "text run"s. Then, it lays out individual text runs one by one. This, in principle, is similar to the Wrap widget usage mentioned by previous comments.

It is achievable, however it is problematic with Flutter. If we do this during widget building, we have to calculate everything out to know the size of some interacting widgets, then we are reinventing Flutter's layout algorithm. If we do this in the painting stage, then the widget in question will become a monolithic black box just like current RichText, which is not very healthy for a high-level package.

Some Pitfalls

1. We cannot line-break everywhere. We should only be allowing line breaks at:
   1. BEFORE plain binary operators
   2. AFTER breakable spacing characters
   3. (probably there is still some cases left)
2. We need to be able to break INSIDE some nested nodes, such as EquationRowNode, LeftRightNode, FunctionNode, and NaryOperatorNode. As long as their ancestors are all such node types, no matter how deep they live in the tree, they should be able to line-break.
3. Before placing a node, we need to know if its right joint is unbreakable. If it is, then it means the first-run width of the next one node (or in some cases, the next few nodes) WILL affect the layout of the current node (appendedWidth mentioned below). In fact, unbreakable joints are the common scenario, and breakable ones are not.

A Fractal Algorithm

The planned algorithm is a fractal design. Every Line widget manages the line breaking of its children, while itself still can be children under parent Lines and get managed. (Again: I don't know much about this field. the whole fractal idea may be entirely unsound. If you know it better, be sure to let me know.)

Constraints down

Instead of BoxConstraint, a new BreakableConstraint mocking (extending) BoxConstraint will be used. The core properties will be

1. firstLineMaxWidth
2. fullLineMaxWidth
3. lastLineMaxWidth

Meanwhile, RenderLine will have a method computeFirstRunMinWidth. As indicated by its name, it returns the min width of the first run.

Their relation is as below.

Sizes up

After layout, RenderLine.size will be an instance of BreakableSize (which is also mocking Size). Its core properties will be height and lastLineWidth. RenderLine will also have a method called computeDistanceToLastBaseline. The relation is as below.

How it works

Each RenderLine only lays out its direct children.

First it calculates the appendedWidth for all its children. This width exists because this child may have a unbreakble joint at the right side and thus is appended with one or multiple nodes. We skip the detail of such calculation.

Then it lays out every children.

1. For an unbreakable child, if "remaining line width" - appendedWidth - size.width < 0 and its left joint is breakable, start a new line and place the child there. Otherwise append it to current line.
2. For a breakable child.
   1. If it has only one run, treat it the same as unbreakable one.
   2. If it has multiple runs. If "remaining line width" - this.firstRunMinWidth < 0 and its left joint is breakable, start a new line. Otherwise append to the current line with lastLineMaxWidth set to fullLineMaxWidth - appendedWidth. (If this happens to be the last child containing a breakable point, you have to use lastLineMaxWidth - appendedWidth instead.) After layout returned, update layout progress accordingly (how much the baseline moved & how much width the last line occupied.

There are two restrictions during layout:

1. A child can never decide to break before the first run, i.e., the first line must contain something.
2. A child can never return with an empty last line, i.e. the last line must contain something.

There is one additional case to cover to complete the algorithm:

1. When handling the children containing the last breakable point, we must special-case it.
   1. If it is a breakbale child with multiple runs, use lastLineMaxWidth - appendedWidth.
   2. It it is not, use "remaining line width" - (fullLineMaxWidth - lastLineMaxWidth) - appendedWidth - size.width < 0 as the criteria.

Special point

Since BreakableConstraint, BreakableSize and RenderBreakable are all mocking vanilla RenderBox's counterpart, they can interact seamlessly with other widgets. If you place them under a ordinary RenderBox, they will still render correctly, but wrapping will be disabled.

Of course, the widgets in this package will place equations inside an adapter widget to enable wrapping.

An Centralized Algorithm

Instead of the fractal algorithm, we place a centralized RenderObject manager at the top of the tree.

The performLayout of children will be called twice.

In the first performLayout, the wrapping is ignored. During this layout, every RenderLine will collect the sizes of the text runs. Sizes returned by children will be translated and concatenated according to joint types. (If wrapping is disabled, then this pass is enough.)

Before the second performLayout, the centralized manager will calculate and issue a list of offsets for every text run. In the subsequent layout, each RenderLine will translate the offsets it received, to the actual offset for ordinary RenderBox child, or to a list of offsets for RenderLine child. RenderLine will strictly places children according to the offsets it received.

[znjameswu]

@creativecreatorormaybenot Again I may need your help😄. I'm not so sure about the correct way to do the wrapping. Also I quite doubt the idea of mocking Size, Constraints etc (it seems evil).

Your team definitely has more experiences on Flutter and design patterns. This is a long plan though, it would be great if you can spare some time in the next few weeks to give it a check.

[Skyost]

@znjameswu It would be incredible from you, this is definitely the only thing that is missing in this library ! 😄

[creativecreatorormaybenot]

@znjameswu To be honest, wrapping is probably not what you want to support first.

KaTeX supported wrapping line breaks late into their development, see KaTeX/KaTeX#1287
And for display mode, it can be ignored anyway.

I think the main reason for this is that horizontal scrolling is preferrable or you do not intend your equation to exceed the horizontal constraint anyway.

It makes a lot more sense with text mode and inline mode, but I would still argue that it is not what you are interested in tackling first.

---

Instead and I think that you threw it in with automatic wrapping, I would suggest that only manual line breaks should be tackled first (\\ et al.).
This is what everyone will expect - it is widely used and probably required for displaying most TeX.

The upside of only supporting manual line breaks initially is that this is well documented and of course already fully implemented in KaTeX for example.
But then again - it is obviously closely related as you implied and the rules are for complete line breaking (also wrapping) are fully explained e.g. in The TeXbook chapter 14.

---

As for your proposal - I think that it is very interesting.

I fully understand your concerns for making unsound design decisions.
Personally, I think I can contribute by adding a few comments that might steer the design into a better direction.

1. Regarding the TextPainter idea - I think you should look at Paragraph instead because TextPainter might be a misleading abstraction if you do not look deeper and this also matches TeX more closely (because it is more low-level). The Dart entrypoint can be found here - keep in mind it is written in cpp but it might be useful in some way anyway.

If you place them under a ordinary RenderBox, they will still render correctly, but wrapping will be disabled.

2. You do not need to worry about this; just add an assertion that throws if something you depend on is not found. For example, how you can only place a Positioned inside of a Stack as a direct render object child (there can be proxy widgets between but no render objects) because it needs to have the correct parent data that only Stack will set.

Since BreakableConstraint, BreakableSize and RenderBreakable are all mocking vanilla RenderBox's counterpart, they can interact seamlessly with other widgets. If you place them under a ordinary RenderBox, they will still render correctly, but wrapping will be disabled.

3. Do you mean that you want to have some delegate logic in the way CustomPainter does it or do you just want to extend RenderObject / RenderBox to implement custom logic. If it is the latter, this is for sure the intended way of using RenderObjects actually because it is what the framework does all the time. RenderSliver e.g. does not even use RenderBox - it implements its own API. RenderBox and RenderSliver basically use separate APIs. For RenderBoxes, there is ContainerBoxParentData etc.

The whole intrinsic size concept is only a RenderBox thing. So we can actually just create our own concept for handling TeX. I am not saying that it needs to be handled differently - there might just be a way to use the RenderBox concepts for this since rendering text also uses it.

4. Did you take a look at how KaTeX does it? I think reinventing the wheel here might not be beneficial because it might be more efficient, but then again it will also take a lot more effort to get away with it and that is something difficult to pull off when you are not working on a framework being paid for it 🙃

5. The way you presented "BreakableConstraint, BreakableSize and RenderBreakable" does seem logical to me and it also seems like an approach that will work. But then again, I am not sure if it required.

6. It seems you want to do a last to first layouting approach, right? Because you need to computeFirstRunMinWidth of the next breakable in order to get the lastLineMaxWidth constraint. So as a consequence, you need to compute that for the last breakable first (and then for the previous one etc.) in order to get the constraint for the first one.

7. What exactly is appendedWidth? I could not figure it out from what you wrote exactly. As in you mentioned it a bunch, however, I was missing an introduction.

---

Overall, amazing efforts from your side ✨
I am not that deep into it yet (not into text rendering in general), but I am really looking forward to this taking off the ground 🚀
And sorry if I am not being that helpful with it - thought I might note down my thoughts anyway.

[poisonpwn]

@znjameswu To be honest, wrapping is probably not what you want to support first.

KaTeX supported wrapping line breaks late into their development, see KaTeX/KaTeX#1287
And for display mode, it can be ignored anyway.

I think the main reason for this is that horizontal scrolling is preferrable or you do not intend your equation to exceed the horizontal constraint anyway.

It makes a lot more sense with text mode and inline mode, but I would still argue that it is not what you are interested in tackling first.

Instead and I think that you threw it in with automatic wrapping, I would suggest that only manual line breaks should be tackled first (\\ et al.).
This is what everyone will expect - it is widely used and probably required for displaying most TeX.

The upside of only supporting manual line breaks initially is that this is well documented and of course already fully implemented in KaTeX for example.
But then again - it is obviously closely related as you implied and the rules are for complete line breaking (also wrapping) are fully explained e.g. in The TeXbook chapter 14.

As for your proposal - I think that it is very interesting.

I fully understand your concerns for making unsound design decisions.
Personally, I think I can contribute by adding a few comments that might steer the design into a better direction.

1. Regarding the TextPainter idea - I think you should look at Paragraph instead because TextPainter might be a misleading abstraction if you do not look deeper and this also matches TeX more closely (because it is more low-level). The Dart entrypoint can be found here - keep in mind it is written in cpp but it might be useful in some way anyway.

If you place them under a ordinary RenderBox, they will still render correctly, but wrapping will be disabled.

1. You do not need to worry about this; just add an assertion that throws if something you depend on is not found. For example, how you can only place a Positioned inside of a Stack as a direct render object child (there can be proxy widgets between but no render objects) because it needs to have the correct parent data that only Stack will set.

Since BreakableConstraint, BreakableSize and RenderBreakable are all mocking vanilla RenderBox's counterpart, they can interact seamlessly with other widgets. If you place them under a ordinary RenderBox, they will still render correctly, but wrapping will be disabled.

1. Do you mean that you want to have some delegate logic in the way CustomPainter does it or do you just want to extend RenderObject / RenderBox to implement custom logic. If it is the latter, this is for sure the intended way of using RenderObjects actually because it is what the framework does all the time. RenderSliver e.g. does not even use RenderBox - it implements its own API. RenderBox and RenderSliver basically use separate APIs. For RenderBoxes, there is ContainerBoxParentData etc.

The whole intrinsic size concept is only a RenderBox thing. So we can actually just create our own concept for handling TeX. I am not saying that it needs to be handled differently - there might just be a way to use the RenderBox concepts for this since rendering text also uses it.

1. Did you take a look at how KaTeX does it? I think reinventing the wheel here might not be beneficial because it might be more efficient, but then again it will also take a lot more effort to get away with it and that is something difficult to pull off when you are not working on a framework being paid for it 🙃
2. The way you presented "BreakableConstraint, BreakableSize and RenderBreakable" does seem logical to me and it also seems like an approach that will work. But then again, I am not sure if it required.
3. It seems you want to do a last to first layouting approach, right? Because you need to computeFirstRunMinWidth of the next breakable in order to get the lastLineMaxWidth constraint. So as a consequence, you need to compute that for the last breakable first (and then for the previous one etc.) in order to get the constraint for the first one.
4. What exactly is appendedWidth? I could not figure it out from what you wrote exactly. As in you mentioned it a bunch, however, I was missing an introduction.

Overall, amazing efforts from your side ✨
I am not that deep into it yet (not into text rendering in general), but I am really looking forward to this taking off the ground 🚀
And sorry if I am not being that helpful with it - thought I might note down my thoughts anyway.

Yes, I think so too, currently large equations with many expressions are rendered in one line, without ‘//‘ it is a long line, adding support for ‘//‘ will be awesome!.

[znjameswu]

@creativecreatorormaybenot

Yeah, you are right. Respecting manual line breaks should be a higher priority. It can be done with a modification to the parser and ast.

Thank you for pointing out. I just went to check TeXBook Ch.14. Well it seems TeXBook doesn't fancy the idea of line breaking inside a nested node ("only outer-level"). That arguably does make things a lot easier. Originally I am trying to reproduce the behavior of MS Word equations, which allows line breaks inside nested nodes (as long as all of its ancestors are of certain types). It seems that the design goal is different and makes the single most difference in implementation.

The reason I'm stuck with RenderBox rather than making whole new RenderObject classes is that, as a result of MS Word style of line breaking, a node itself cannot know if it is allowed to line break (you can line break deep in the trees) -- it all dependes on the type of its ancestors. So the way I come up is mocking RenderBox's contract. If there is a ancestor that prohibits line breaking (e.g. subscript node), the ancestor RenderBox consume the mocked constraints and generate ordinary constraints. That way the descendent will know it cannot line-break. To sum up, in MS Word style, we have to often switch between breakable & unbreakable, mocking might be easier.

Anyways, I think from the information it may be worth discussing which line-breaking scheme we should be after, TeX or MS Word. Personally, I prefer MS Word. I think vanilla TeX's line breaking is so conservative that I frequently ignore that feature. But as you have shown, it is better documented and understood.

[creativecreatorormaybenot]

@znjameswu Makes sense!

From the perspective of content creation, I think that 1. manual line breaks is most important because that is what people really want to do and 2. if there are automatic line breaks, TeX line breaking might be sufficient because it is not crucial to line break in nested nodes (people probably might not really notice the difference).

So judging from that "different perspective" than what we might have, I think that an MVP for line breaking would only need to support the easiest ways. I guess it also depends on how ambitious you want to be.

[olof-dev]

@znjameswu Great package and a very impressive effort! Just to second @creativecreatorormaybenot's comments: I would advocate only starting with explicit line breaking, with \\ and perhaps some of the TeX environments that deal with common use cases, like multline (from amsmath).

For dealing with variable screen/widget widths, perhaps something like an \optionalLineBreak TeX command could be added, where this gets turned into a \\ depending on the widget's constraints. (Only at the topmost level.) It could even be \optionalLineBreak{200} or something to say "when the constraints are w < 200, turn this into a line break, otherwise ignore". And maybe the unit should be scaled according to font size, etc, but you get the idea.

I haven't tried the following myself, but somewhere down the line you might be interested in looking at the package breqn. I suspect this would be way more involved than it's worth, for now.

[znjameswu]

It turns out we can support TeX-style line-breaking quite easily. Since TeX only allows line-breaks at the outmost level, the problem I explained earlier is a non-problem. At the outmost level, no layout elements are interacting. Thus the simple solution proposed in #8 (comment) actually works: Render them in chunks and even expose those chunks to let users decide how to wrap them.

(Of course, SelectableMath won't be able to do this until Flutter comes up with a better selection system. But Math definitely can.)

This is such a simple plan. I think I'll go for this for now. As for advanced MS-Word style line-breaking, they seem to be a strict superset of TeX line-breaking, and I can work on it later in a different issue. @creativecreatorormaybenot What do you think?

[creativecreatorormaybenot]

@znjameswu Well, frankly we do this already anyway :D

I think the simple Wrap approach is probably not healthy for the parser, so I would instead integrate it into the regular rendering pipeline, but it seems good to me 🙂
Would just want to make sure that intrinsic sizing and such do not break.

[znjameswu]

@znjameswu Well, frankly we do this already anyway :D

I think the simple Wrap approach is probably not healthy for the parser, so I would instead integrate it into the regular rendering pipeline, but it seems good to me 🙂
Would just want to make sure that intrinsic sizing and such do not break.

We can introduce a new static function to Math, say

static List<Widget> Math.texToChunks(String content, .....)

Which makes no modification to the parser nor the ast. It does the normal parsing, but unwraps the outermost row node, and put each piece into a Math widget to render them.

[creativecreatorormaybenot]

@znjameswu Yeah, that is what we do at the moment :D

I think it is a really crude solution, but it does work of course.

[znjameswu]

I have implemented the easier TeX-style line breaking. It is currently on the master branch (i.e. null safety enabled). I'm not sure if it is badly needed to be backported to pre-null_safety versions. Welcome to checkout the design.

I'll close this issue and open a new one for the remaining tasks.

[MidhunrajXYZ]

Hi, I can't access the Math.texBreak() method. I am using the null-safety version (0.3.0-nullsafety.1). Is this functionality yet to be added?

My code looks something like this:

final tex = Math.tex(r'2+2');
final tex2 = tex.texBreak(); // this line is erroneous saying texBreak() is not defined.

[Skyost]

Is this functionality yet to be added?

Not in 0.3.0-nullsafety.1. Try this package instead.

[MidhunrajXYZ]

Try this package instead.

This seems to be working. Will this repo eventually add that feature or do I have to continue using the forked one?

[MidhunrajXYZ]

I can't mix text and maths together and wrap them.

final teXt = Math.tex(
      // context.select<QuizState, String>((state) => state.questionStr!),
      r'\text {This string is not going to be wrapped. Is there a way? because I want to mix text and math} \frac {3} {4.4}',
).texBreak();

// ...

Wrap(children: teXt.parts), //output: everything is in a single line. overflow error. 

[Skyost]

This seems to be working. Will this repo eventually add that feature or do I have to continue using the forked one?

I think you'll have to stay with the forked one for now.

[MidhunrajXYZ]

I can't mix text and maths together and wrap them.

final teXt = Math.tex(
      // context.select<QuizState, String>((state) => state.questionStr!),
      r'\text {This string is not going to be wrapped. Is there a way? because I want to mix text and math} \frac {3} {4.4}',
).texBreak();

// ...

Wrap(children: teXt.parts), //output: everything is in a single line. overflow error. 

Is this issue solvable? Here the \text block is not breaking up. Is the logic hard to implement? I think it just needed to separate each words.

[Skyost]

Is this issue solvable?

I have no clue sadly. You may have to directly edit the source code of this library if it doesn't work for you.