Render to HTML5 canvas #251
Conversation
This only handles enough to convert the default example, and doesn't get all the spacings right yet. See http://localhost:7936/canvas.html for example.
Since katex.less starts with font: normal 1.21em KaTeX_Main we have to scale the font by 1.21 to obtain equivalent behavior.
We do this for any node pair where the current style is text style, not only if the immediate parent is explicitely of class textstyle. Not sure whether this is accurate in all situations.
|
This is very exciting, I am at a conference this weekend but I should be able to take a closer look next week. I am happy to help out in any way I can. For some context about the Bokeh use-case, we'd like to be able to render math text into an existing canvas at a specified locations (similar to |
The placement of the horizontal lines looks better now, although the change is pretty ad-hoc and might require verification.
In particular, we now allow for a global halign option which allows printing the formula centered or right-aligned at the reference position, instead of left-aligned only.
The previous list was done by hand, and initially ordered by space size. This new list is automatically extracted from the style sheet. It relies on rather strict syntax conventions there.
|
This can now reproduce most of the formulas used for the snapshot tests. If anyone has experience with the CSS font loading module and/or the Web Font Loader project, I'd be glad if anyone would have a look at hooking up our fonts so that we don't have to guess and hope when they'll be available. It would be best if we could introduce two more options. One to control whether the formula should be rendered with incorrect fonts if the correct ones aren't available, as opposed to not rendering at all, and the other to register a callback which gets called if all (or perhaps even if more) fonts have become available, to trigger a redraw of the canvas. |
|
It's awesome to see how quickly this coming along. I checked out the branch and gave it try. The rendering seems blurry compared to the HTML. It might have something to with being on a retina device. Also, there was some clipping going on, but I think adjust the size of the canvas to match the bounding box of the output should fix that. Canvas: HTML: |
|
Yes, the canvas so far is simply fixed size and pixel density 1. For the latter, this tutorial by Paul Lewis points the way towards a solution. Have done that in another project alreary, works like a charm. Haven't integrated it with the former, i.e. with auto-resizing canvas, yet, but it's certainly doable. |
|
I was just about to link that, we used the same article to develop Bokeh's support for HiDPI modes. @gagern do you think that the capability to render at a specified location onto an already existing, user-supplied canvas is feasible for this work? If not, it should be easy enough for us to maintain small private canvases for annotations, that can be quickly blitted onto our main canvas, but I thought it would be worth asking about. |
|
For retina support, I write this last night: https://gist.github.com/spicyj/1d9145ab46190782260f |
|
@bryevdv: Sure you can render to an existing canvas. Have a look at these lines for an example application. There I compute the box up front, then draw something else (in this case the bounding box and its base line) and then draw the box I computed before. If all you care about is render at a given position, you can call renderToCanvas instead of canvasBox. It will prepare a box and immediately render it at the given position. You can still control horizontal alignment using the halign property of the options object. |
This will keep the resulting box simpler, so it will render more quickly.
This makes the math fit in better with surrounding text.
|
@gagern that's great. We have a new release coming up in a few weeks but immediately after that we will start looking into integrating this as a high priority! |
|
Out of curiosity, what's blocking progress on this? Is it a lack of interest? Or prioritization? I'd noticed, for example, that some diacritics weren't rendering properly with this renderer even though they are supported by the standard renderer. |
|
@marcianx there are a couple of things that I'm concerned about with this and don't have good answers for at the moment:
|
This uses let or const instead of var, resolving conflicts due to this change: katex.js src/buildTree.js
Conflicts: server.js
Knowledge of the hackish nature of this approach, and lack of time for a proper solution.
I expect there are several things which the main renderer can handle but this code can not. Essentially what this code does is read some HTML-like representation, then use the CSS classes from that and turn that into its own metrics information. Essentially a bit like a very lightweight CSS engine, but not based off the KaTeX css file but with parts of that CSS duplicated in JavaScript program logic. So this will miss classes I forgot, or nesting constucts I forgot, or I got wrong. It will also miss classes added to the CSS since I started this branch. I haven't revisited that file in quite a while. And it will miss positioning expressed through means other than these classes, e.g. new ways of per-node positioning, or box borders, or combining characters. All of this is the reason why we consider this more of a hack and proof of concep implementation than a fully supported feature: with the current approach, keeping things in sync is rather hard. We'd prefer some intermediate representation which makes positioning explicit in a way that can be handled cleanly both by CSS and canvas.
You are right, underline isn't supported as it wasn't in core when I started this. It should be possible to add it, if someone (might include me) finds the time. In the meantime, I've resolved some conflicts so this branch merges with master again. That way, people can use master with all its current features to see which of these are supported by the canvas renderer. |
|
Out of curiousity, have you thought about going straight from the parse tree to HTML canvas, via a |
|
Right now, buildHTML handles both metrics/styling decisions (for example, the supsub function that handles superscript/subscript vertical positioning) as well as how to encode those in HTML with CSS classes. I suppose the ideal is something akin to another intermediate representation (at least conceptually) which has sufficient positioning information and can be used to output to either HTML or Canvas. |
|
I agree. Writing a canvas-specific version of |
|
Firstly, thank you very much for rebasing this PR! |
* Added button press to premium, Assessments center current bubble and profile screen changes, Assesments center bubble
|
@gagern Thank you for this PR. It's really exciting to see Latex rendering being brought to the canvas! With this PR, is it possible to be able to get the appropriate width/height and translate each individual character? |
|
@rrandallcainc the PR uses the Canvas API to measure the glyphs to get the width of each as it lays them out. Another approach which is may be performant is to grab widths for glyphs beforehand. Running |
|
FWIW, I am working on modernizing this solution, and already have most of the features implemented. I reached out to @gagern but received no response :(. I'd like to open-source the solution in a separate package to allow others to benefit. This would still leverage KaTeX's parsing/buildTree. If anyone is interested in seeing progress, lmk. I can see what I can do. |
|
@rrandallcainc cool. I'm looking forward to seeing what you come up with. |
|
I'm going to close this PR since whenever we get to supporting canvas it will require substantial rework. It's linked to in a number of other issues. |
@ry-randall – @gagern did this canvas rendering in order to use KaTeX with CindyJS, but meanwhile, he left the CindyJS project, and we are a bit lost, as we are using an ancient version of KaTeX now (see CindyJS/CindyJS#829). Is there any way I could help? I could try to contact @gagern through other channels, but this only makes sense if this would help you… |
|
Sorry for lack of response to @ry-randall in #251 (comment). I find I've trouble keeping up with many comms channels. @kortenkamp I'll provide you with a phone number just in case. If there is any plan to get render-to-canvas support into KaTeX I'd be interested to contribute, but I only have limited time available besides day job. Alternatives that I can think of:
Random thought: It might make sense to also have SVG in mind when doing all of this. On the one hand, SVG should have all the primitives needed for rendering, so translating between the two should be really simple. On the other hand, that would allow using resulting SVG files as test cases. Processing those in automated tests should be a lot easier than processing screenshots, so one could build a really exhaustive collection of test cases and ensure they keep rendering to the same SVG files over time. Of course, the internal structure of those SVG might occasionally change with no real impact on the rendering, but if we avoid deeply nested grouping constructs in SVG, and instead render each primitive symbol on its own, then there shouldn't be a lot of variation there. |
Since my announcement of canvas rendering was met with considerable interest (by KaTeX developers as well as for bokeh/bokeh#647), I decided to share my work early on. At the time I'm opening this request, this branch is in a very early stage of development. But others are welcome to contribute, be it by commenting, by suggesting ideas, or by providing code in the form of pull requests against this branch.
After launching the local development server, see http://localhost:7936/canvas.html for an example of rendering to canvas.