Text rendering

[Maximkaaa]

To support labels on the map, we need a mechanism to render shaped text onto the map. This includes:

• Gliph rendering
  • Raster fonts decoding and building glyph map textures
  • Vector fonts tessellating
• Text shaping - at this point we only need single-line shaping
• Text measuring

Rendering a text along a path would be desirable, but we can do it separately later.

At the first glance, https://github.com/grovesNL/glyphon library can cover all our needs. iced uses it for text shaping, so it must be mature enough to support our use cases.

We would also need test cases for complex cases:

• RTL, vertical (?)
• Arabic, hindu scripts
• Devanagari scripts (see Unicode font do not display properly in map. maplibre/maplibre-gl-js#50 )

Must have options for initial implemenation:

• text size
• text style (bold, italic, underline)
• vertical and horizontal alignment
• offset

Tracking

The basic research stage is complete, and test implementation is merged into main. But it is still just a first step towards full support text labels on the map. So not to forget what is still to be done, I'll be using this issue as a tracker issue for this feature.

Here is the list (not full) of things yet to be done:

General loading and rendering

• Font selection and fallback
• Text alignment (especially critical for correct display of RTL scripts)
• Integration with application-level font engines
• Render glyphs to bitmap map using hinting

Web

• Rendering labels using background canvas in web workers

Feature layers

• Example of using labels for features

Vector tiles

• Better mechanism to configure labels in vector tile style
• Do not show overlapping labels
• Fade-in/fade-out of labels

[Maximkaaa]

@maxammann I know you did a lot of research about this for maplibre-rs. Do you have any input on how to approach this?

[quietlychris]

I believe that some other projects like Bevy are looking at using cosmic-text for this, per bevyengine/bevy#7616

[Maximkaaa]

Yes, glyphon also uses cosmic-text for text shaping, but it also includes glyph texture atlases.

[maxammann]

You can find some of my notes here: https://maplibre.org/maplibre-rs/book/development-documents/font-rendering.html

Text rendering can have varying challenges based on what you want achieve. MapLibre is able to place text in the "game world". So you can actually walk around text and have text "printed onto the map". If you want to have the same then the problem becomes very hard as you have to come up with a technique to achieve that (SDFs, Bezier rendering etc).

I think that goal is maybe too ambitious and not needed in 99% of the cases. If you want to render text in the screen space then you probably can get away with using something like glyphon.

Note though, if you want perfect text rendering for all languages you kind of still have to use harfbuzz and not rustybuzz :/

[Maximkaaa]

I've spent some time trying things out and researching and here is what I've come with: https://maximkaaa.github.io/galileo/blog/posts/text_rendering_design/

[grovesNL]

In case it's helpful, I'm definitely interested in adding transformations into glyphon. We just need to decide on the right approach to expose it. iced currently uses glyphon for regular text but uses the vector representation for transforms (e.g., iced-rs/iced#2204) that doesn't go through the glyph cache, so it would be really cool to upstream something similar to glyphon itself.

[Maximkaaa]

So far I've decided to go to vector only approach first, as it is more general and covers all uses cases. So I don't use glyphon at the moment. But my experiments I already see less than ideal results in some cases due to lack of hinting. So I will definitely be looking into glyphon later.

That being said, even for vector representation of glyphs, caching tessellated glyphs would be very useful. I believe that swash does cache path commands for glyphs similar to how it caches rasterizations. But the most computationally heavy operation for these is tessellation, so if glyphon would take care of caching that, that would be great.

As for transformations themselves, in my case I would have to line up glyphs along a path, which means I would have to apply different set of rotations and translates to each glyph, not to the whole glyph run. This is something SVG can do, but I'm not sure if this is in scope of glyphon. If not, just convenience functions to rotate/translate vector glyphs would be helpful but not essential.

[grovesNL]

Yeah for sure, I'd like to cache tessellated glyphs if we went to vectors for transformations.

The path use case is interesting. I don't know how SVG renderers handle this but it seems like you'd need deep integration with shaping/layout to do per-character transformations (vs. projecting the entire run along the path somehow).

[maxammann]

Great blog post!

So far I've decided to go to vector only approach first, as it is more general and covers all uses cases.

Do you go with the maplibre signed distance fields or plainly tessellating the curves in the font files?

If it is the latter then you likely will hit performance limitations.

[maxammann]

btw https://github.com/wipfli/about-text-rendering-in-maplibre

[Maximkaaa]

Thanks for the link, @maxammann , it is very well written. And as follows from the article, using SDF has some issues also. So far in my tests I haven't faced any performance issues, but I didn't test for performance in any depth. Anyway, my main concern at the moment is to create a solution that will work correctly for all inputs and on all platforms, and MapLibre approach doesn't allow both of these (shaping problem and using server-side rasterizing are big no-no for me). After that is done, performance can be improved by caching, multithreading etc., but that should be done only after testing to see, which part of the code would be a bottleneck.

[kylerchin]

How long will it take for this to be released?

[Maximkaaa]

I don't have much time to work on Galileo at the moment, but hope to roll out first iteration of labels late next week. Implementing everything that is planned including optimizations for web, advanced rasterization and caching will take a while.

[shujaat-github]

@Maximkaaa The following repository might be helpful for font rendering:
https://github.com/mooman219/fontdue

[Maximkaaa]

So, with #90 I've merged basic implementation of the labels. This required a lot of changes in the underlying architecture, especially for vector tiles, but it seems to be working. Some things to note:

• I've tried using cosmic_text, but this library is designed in a way that it's impossible to make it work efficiently in multiple threads. So I've reverted back to using raw rustybuzz. This means there's no built in font fallback etc. But we can probably take some of font loading and fallback from cosmic_text.
• The current implementation is very self-reliant, doing all the font loading logic by itself. In real world applications we would want to use font engine provided by the GUI framework the application built on. This means that we would probably want to return Cosmic text provider, and add other providers that would allow integrate with other GUI frameworks. This needs further research though.