Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support color fonts #487

Closed
RazrFalcon opened this issue Nov 30, 2021 · 33 comments
Closed

Support color fonts #487

RazrFalcon opened this issue Nov 30, 2021 · 33 comments

Comments

@RazrFalcon
Copy link
Collaborator

Emojis can be stored in multiple ways, but we should at least support bitmap one.

@LaurenzV
Copy link
Contributor

I will try to look into this. Although I'm not sure what I'm getting myself into. xD

@RazrFalcon
Copy link
Collaborator Author

Although I'm not sure what I'm getting myself into.

Very bad time 😄

I've mentioned main issues with text handling here recently: linebender/tiny-skia#1 (comment)

Also, just a heads up, but "bitmap fonts" are a bit of a mess of a term. What we need is Emojis support, which in TrueType terms means:

  • sbix (Apple, basically a PNG)
  • bloc+bdat (legacy 1bit, parsed by ttf-parser, but not decoded)
  • EBLC+EBDT (legacy 8bit, parsed by ttf-parser, but not decoded)
  • CBLC+CBDT (Google, basically a PNG)
  • COLR (kinda a mini SVG, v0 is supported, v1 is in progress, had to be rendered somehow)
  • SVG (aka SVG, but technically a subset/superset, we need a custom parser for it)

As you can guess, the first step would be to write a TrueType glyph rasterization library. Well, unless you plan to stick with sbix/CBLC+CBDT.

Overall, I would say the step number 1 would be to untie the current implementation from operation strictly on path outlines. Meaning we have to do layout purely on glyph ids and then fetch/render/cache required glyphs.

I have a lot of ideas, but barely any time. For me, this is the next big resvg update. But it's a beefy one and I have no time.
Therefore feel free to ask any questions.

PS: oh yeah, embedded SVG fonts + WOFF support is also kinda part of it. After all, SVG is mostly about text and nothing else.

@LaurenzV
Copy link
Contributor

I looked a bit into it. I agree that in the long term, it would probably be best to have text just contain a number of glyphs as well as their position, and then just render them correspondingly. Eventually, I plan to implement text selection for svg2pdf, and for that it would make things a lot easier if I just have to deal with positioned glyphs, instead of having to copy paste basically the whole layout logic. But this would probably take a bit of timeto implement.

What do you think if we try this is a "stopgap" approach until we have something better: Right now, an OutlineCluster only contains a tiny skia path. Instead, we could change it so that the OutlineCluster contains a group with some children. These children would be:

  • Path nodes for outlined glyphs
  • Image nodes for raster image glyphs
  • Group nodes for SVG glyphs

And then we basically just render the group as is. I'm not sure if I'm missing something. But I've managed to change the current implementation so that it works with Path nodes and it seems to work fine, so next I could look into whether it actually works with bitmap emojis. If you think that this approach is worth pursuing for now.

@LaurenzV
Copy link
Contributor

LaurenzV commented Mar 26, 2024

Regarding SVG, when you refer to having to write a custom parser I presume that you are referring to the fact that some elements must be ignored according to the specification (e.g. text, stylesheets, etc.)? But I'm really not sure if it's worth it to write a custom parser for that... Maybe we can instead add options to usvg to ignore such elements? Not sure. But for now, it should be fine to just use the existing usvg implementation, no?

EDIT: Nevermind, I should've kept on reading... There is more stuff that we need to support.

@LaurenzV
Copy link
Contributor

But the more I think about it, maybe it's indeed best to first rewrite the current code to contain positioned glyphs which are then subsequently turned into paths, and then worry about how to best deal with emojis... Will try to look into it.

@RazrFalcon
Copy link
Collaborator Author

If I recall correctly, the two main issues with TrueType SVG are: CSS 3/4 features which we do not support (namely computed values) and an ability to reference other glyph (aka <use xlink:href="#glyph14"/>, which are obviously impossible right now). Meaning we have to improve CSS support and add a custom xlink resolver (like we do with images) first.

As for the "next step" - it doesn't really matter to me. The reason I would suggest starting with separating layout and outlines is because outlines can be freely scaled, but bitmap fonts can't. Meaning we have to either get the bigger bitmap from the font and downscale it or somehow dynamically choose which bitmap size to render.
PS: yes, fonts with the sbix table (aka Apple Emojis) contain multiple bitmap sizes for each glyph.

@LaurenzV
Copy link
Contributor

I'll keep you posted! I have some ideas I will try out.

@LaurenzV
Copy link
Contributor

LaurenzV commented Apr 4, 2024

So we have glyph layouts now, so once I find the time I will try to work on this (for now just bitmap emojis) in the next few days/weeks.

But my question is: Do you know any fonts that I can use to test all of the legacy bitmap formats? I can use Apple Color Emoji to test sbix, and I think older version of Noto Color Emoji use CBLC + CBDT. But any idea for where I can fonts for the older ones?

Basically, I'd like to have one font for each of the formats that ttf-parser can return. Any idea where to search for those?

@LaurenzV
Copy link
Contributor

LaurenzV commented Apr 4, 2024

Or would you accept a PR that implements this just for PNG bitmaps for now?

@RazrFalcon
Copy link
Collaborator Author

Well, that the trick - there are none. The only sbix font I know is Apple Color Emoji. Some versions of Noto Color Emoji do use CBLC + CBDT. And that's about it.
Here are some EBDT+EBLC fonts: harfbuzz/ttf-parser#121

Let's stick with PNG for now.

@LaurenzV
Copy link
Contributor

LaurenzV commented Apr 9, 2024

So I'm currently stuck with the following:

Let's say we have the following SVG:

<svg id="svg1" viewBox="0 0 200 200" xmlns="http://www.w3.org/2000/svg"
     font-family="Apple Color Emoji" font-size="32">
    <title>Emojis</title>

    <path id="crosshair" d="M 20 100 L 180 100 M 100 20 L 100 180"
          stroke="gray" stroke-width="0.5"/>

    <text id="text1" x="100" y="100">😀😁😂🤣</text>

    <!-- image frame -->
    <rect id="frame" x="1" y="1" width="198" height="198" fill="none" stroke="black"/>
</svg>

my current implementation renders it like this:
test

Basically, I'm setting "the cursor" at the position (100, 100), and then I'm aligning this with the bottom of the picture and I render the image at font size. However, in Chrome it looks like this:

image

So for some reason the emoji reaches a bit below the vertical axis. But I'm not really sure where this shift is coming from and how to calculate it... Any ideas?

@RazrFalcon
Copy link
Collaborator Author

ttf_parser::RasterGlyphImage has x/y positions. Probably them.

@LaurenzV
Copy link
Contributor

LaurenzV commented Apr 9, 2024

Nope, in this case all glyph raster images have x and y set to zero...

@LaurenzV
Copy link
Contributor

LaurenzV commented Apr 9, 2024

My best guess would be that it is somehow aligned to the descender of the font? But I don't see this being described anywhere...

@RazrFalcon
Copy link
Collaborator Author

Hmm, Apple emojis are rendered "below" the baseline in Firefox and Safari as well, so we definitely have to account for something, but I have no idea for what.

rustybuzz seems to be producing the correct output as well:

> cargo run --example shape -- '/System/Library/Fonts/Apple Color Emoji.ttc' '😀'
u1F600=0+800
> hb-shape '/System/Library/Fonts/Apple Color Emoji.ttc' '😀'
[u1F600=0+800]

@LaurenzV
Copy link
Contributor

LaurenzV commented Apr 9, 2024

Yeah I really have no idea... The width seems to be just fine, there really is just some shift on the y-axis for some reason.

@LaurenzV
Copy link
Contributor

LaurenzV commented Apr 9, 2024

Would it be okay for you if for now I just try to find a magic value that at least works with Apple Color Emoji and Noto Color Emoji? I will also add some tests to make sure it also works for different font sizes. I'm not sure what else I could do.

@RazrFalcon
Copy link
Collaborator Author

Not ideal, but good enough.

@LaurenzV
Copy link
Contributor

Welp... using a magic value that works for Apple Color Emoji does not work for Noto Color Emoji. 🥲 So yeah... I guess we're stuck for now.

@RazrFalcon
Copy link
Collaborator Author

I was expecting you hardcode an offset per font family.
I might look into it next weekend.

@LaurenzV
Copy link
Contributor

Ah... Yeah I guess this could be a last resort option.

@RazrFalcon
Copy link
Collaborator Author

Can you also publish your changes so I could test them locally.

@LaurenzV
Copy link
Contributor

Yes, sorry, I'll try to clean it up and upload it within the next few days.

@LaurenzV
Copy link
Contributor

See #735.

@yisibl
Copy link
Contributor

yisibl commented Apr 12, 2024

Also, just a heads up, but "bitmap fonts" are a bit of a mess of a term.

I suggest changing the title of this issue to Support for Color fonts (including Emoji), this site does a good job of explaining what color fonts are.

Because color fonts are not just emoji, it is a broad font technology, emoji is just one of its usage scenarios, other scenarios including COLR v0 are also widely used in font icons.

Color font is also a term used in CSS: https://drafts.csswg.org/css-fonts-4/#color-font-support

<color-font-tech>= [color-COLRv0 | color-COLRv1 | color-SVG | color-sbix | color-CBDT ]

@yisibl
Copy link
Contributor

yisibl commented Apr 12, 2024

SVG fonts are pretty much an abandoned format in browsers (Chrome has never supported it and doesn't intend to), and COLRv1 is an alternative technology. I would suggest that consideration could be given to lowering the priority of this format implementation in resvg.

@Offirmo
Copy link

Offirmo commented Apr 12, 2024

Hi mates, amazing that you're working on it!
How about focusing at having NotoColorEmoji work? (Open Font license)

@LaurenzV
Copy link
Contributor

LaurenzV commented Apr 12, 2024

SVG fonts are pretty much an abandoned format in browsers (Chrome has never supported it and doesn't intend to)

Are you sure? At least on my Computer Chrome can handle emojis from Twitter Color Emoji, which as far as I know is an SVG font... But maybe it depends on the OS. But anyway, at least in my PR it wasn't much effort to support that, so it seems like the more "obscure" features of SVG fonts that deviate from standard SVG might not be used as commonly. So we might as well just add it.

How about focusing at having NotoColorEmoji work? (Open Font license)

AFAIK the new vesions for Noto Color Emoji use COLRv1 which currently isn't supported yet upstream. I don't know if there are any particular difficulties in implementing this or whether someone just needs to find some time to finish it, maybe @RazrFalcon can say more about that.

But the older versions of Noto Color Emoji use bitmaps, which should work with that PR.

@yisibl
Copy link
Contributor

yisibl commented Apr 12, 2024

Are you sure? At least on my Computer Chrome can handle emojis from Twitter Color Emoji, which as far as I know is an SVG font... But maybe it depends on the OS.

See MDN SVG fonts:

The functionality was removed from Chrome 38 (and Opera 25) and Firefox postponed its implementation indefinitely to concentrate on WOFF.

You can try ttx -l some_font.ttf to see what tables are available for the Twitter Color Emoji font.

@LaurenzV
Copy link
Contributor

Oh. This is talking about the font element in SVG. Yes, this is deprecated, but I don't think there are any plans to implement this, anyway. I was talking about the SVG table in OpenType fonts, which use SVGs as glyph descriptions. And I think those are supported by web browsers.

@RazrFalcon
Copy link
Collaborator Author

RazrFalcon commented Apr 12, 2024

@yisibl The SVG Font element and TrueType's svg table are two different things. No one supports the first one and it was deprecated in SVG 2. But TrueType SVG fonts are pretty common, sadly.

COLRv0 will be supported soon, thanks to @LaurenzV. COLRv1 is in progress by me harfbuzz/ttf-parser#133 (for a while now...).

@RazrFalcon RazrFalcon changed the title Support bitmap emojis Support color fonts Apr 12, 2024
@yisibl
Copy link
Contributor

yisibl commented Apr 12, 2024

Sorry, I misremembered. Please ignore the MDN link and description.

It's true that Chrome doesn't implement OT-SVG(See https://issues.chromium.org/issues/40336440), though, and it doesn't look like there are any plans to do so at the moment, since COLRv1 replaces OT-SVG with some extent.

I don't have much of a problem with implementing OT-SVG in resvg, if it's not complicated and doesn't have a significant performance disadvantage, otherwise, I don't see much point in it.

OT-SVG Test Page: https://pixelambacht.nl/lapislegit/

@RazrFalcon
Copy link
Collaborator Author

Fixed by #735

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants