Feature: Allow symbol scaling in nonmono (down to 2 'widths')

[Finii]

Description

For non-single fonts (i.e. with no --mono) added symbol glyphs will be scaled down to be maximum 2 'slots' wide.

Usually the added Glyphs are about 1.6.. 1.8 the width of the normal 'Letters' we have in the target fonts. But there are tiny/tall fonts, where the added glyphs just look out of place.

Requirements / Checklist

• Read the Contributing Guidelines
• Verified the license of any newly added font, glyph, or glyph set

What does this Pull Request (PR) do?

The first commit refactor the code with no effective change in result. This is needed in preparation to the actual intended change that would look rather complicated otherwise.

The actual change in the 2nd commit allows downscaling added glyphs in non-mono fonts, but still observes the ScaleGlyph data.

How should this be manually tested?

Fonts patched with --mono should come out unchanged.
Fonts patched without --mono but higher EM value (i.e. where the added symbols are smaller than 2 times the normal advance width) should come out unchanged.
Fonts patched without --mono but small EM shall have symbol glyphs not wider than two normal advance widths.

Any background context you can provide?

What are the relevant tickets (if any)?

• Size issue in ProggyCleanTT* #718
• [Feature Request] Shrink very wide icons to fit inside 2 wide space #747
• Can't find the Kali-Linux icon #888

Screenshots (if appropriate or helpful)

[Finii]

There is still an issue, but I promised to create the PR this year ;-)

Please ignore until this is fixed.

It is regarding use of EM vs height for scaling. There is even this comment in the code:

        # font_dim['height'] represents total line height, keep our symbols sized based upon font's em
        # NOTE: is this comment correct? font_dim['height'] isn't used here

Unfortunately there is no explanation given in the commit that adds that. Before the commit height has been used.
Need some time to research, and now I have to celebrate ;-)

Happy New Year 🎉

 Fini

[Finii]

Note to self: I believe height is correct, as is done again afterwards in the copy_glyphs() code... Strangeness.

[Finii]

Pulled my problem out into a new PR #749. I think it is important enough to have its own discussion etc pp

[Finii]

If #749 is pulled, this would result in this change here:

-# Use the font_dim['height'] only for explicit 'y' scaling (not 'pa')
-target_height = self.sourceFont.em if stretch == 'pa' else self.font_dim['height']
+target_height = self.font_dim['height']

Which I think should be done.

[Finii]

Ready for merge, but I would change, as detailed above:

-# Use the font_dim['height'] only for explicit 'y' scaling (not 'pa')
-target_height = self.sourceFont.em if stretch == 'pa' else self.font_dim['height']
+target_height = self.font_dim['height']

[Finii]

Just a thought... Maybe the scaling should not be fixed to '2 widths', but take into account the unicode slot expected width like Windows Terminal here:

https://github.com/microsoft/terminal/blob/main/src/types/CodepointWidthDetector.cpp

But well, this PR is at least a beginning to make sure the glyphs are maximum 2 widths wide and not like ... 3.
Some should be even smaller (i.e. only 1) according to this.

But that would leave us with yet another 'font flavor', "Nerd Font UnicodeMono" ;-D

[jirutka]

@Finii, can you please rebase this PR on top of the current master?

[Finii]

Rebase on master, force push.

This was a bit harder, because the scale glyph mechanics changed in master to be more powerful. On the other hand this code explicitly does not set out to handle scale glyph stuff.

I did only few tests on the code / patch result, because I'm in a hurry (vacation starts today evening and I'm packing in parallel). But I still believe it is correctly rebased. Take with a grain of salt and report back :-)

[Finii]

Would fix #888

Well, the bottom part which is not directly the issue itself.

[Finii]

Rebase on master (trivial), force push

[lethefrost]

Hi @Finii , I am wondering if exact multi-spaced widths would be possible. For example, if one spacing unit is 1024, then make all narrower glyphs have width of 1024; and all wider ones have 2048, 3072, or 4096, by rounding up the width to the closest multiple of the unit width, and centering the glyphs without scaling them? It would be very helpful if we want to patch some different fonts or icons together but still want to keep the codes aligned in editor using the patched font. I have scant knowledge about fonts. I would really appreciate if you could make this work or point out any tool/library/document that could help to me. Thank you so much!🥺🥺

[Finii]

I'm not sure if you seek help for your own project or want to propose a Nerd Font change.

Of course it would be possible to have (in your example) 3072 or 4096 glyphs in a font that has a basic width of 1024.
Typically Nerd Fonts are used in a 'monospaced' environment like terminal emulators. Some widely used terminal emulators do not allow glyphs to be wider than '2 cells' - that is the reason for the PR. Those terminal emulators scale the glyphs down to fit into 2 cells, and usually not very nicely.

The second thing: 'Typically' wider than one cell glyphs are left-aligned. You propose center aligned. I guess people would not like that, as their setup will look different.

Maybe you can tell me more what you have in mind, I'm just fishing in the dark. Sorry.

[Finii]

Rebase on master, force push

[lethefrost]

I'm not sure if you seek help for your own project or want to propose a Nerd Font change.

Of course it would be possible to have (in your example) 3072 or 4096 glyphs in a font that has a basic width of 1024.
Typically Nerd Fonts are used in a 'monospaced' environment like terminal emulators. Some widely used terminal emulators do not allow glyphs to be wider than '2 cells' - that is the reason for the PR. Those terminal emulators scale the glyphs down to fit into 2 cells, and usually not very nicely.

The second thing: 'Typically' wider than one cell glyphs are left-aligned. You propose center aligned. I guess people would not like that, as their setup will look different.

Maybe you can tell me more what you have in mind, I'm just fishing in the dark. Sorry.

Hi @Finii , first I want to thank you soooo much for replying to me that quickly! It's very nice of you to help me out here with my stupid questions. Sorry for not checking notifications these days🥺

Thank you for the explanation! It helps to clarify some of my confusion. I was not so familiar with font designing. I previously thought most common glyphs are center aligned, because I had seen many of this type, like many symbols and characters in latin or cjk charsets. Thank you for pointing it out, letting me have the chance to correct my impression. I now notice that there are also many, maybe more of them left aligned.

What I was trying to do was just to completely 'monospacify' a font, by not allowing any glyph width to be not an integer multiple of the basic unit, say, a cell, or in this case 1024. To set every glyph to have width as exact multiple of 1024, I wanted to round it up. For example, make some glyph of original width 1000 to 1024, 1800 to 2048. Since the width is changed without scaling, I was worried about the aligning problem. That's why I was talking about center aligned (but I had no idea of the effect).

You are absolutely right! I indeed found that my terminal (iTerm2) does not allow characters to occupy wider than 2 cells spaces. It's weird, because even I cannot remember exactly, I sometimes do find fonts containing wider glyphs... just like the ones in issue #747 . But anyway, the 3072 and 7096 I mentioned before are just meant to form a series of integer multiples of 1024, which has no practical significance, supposing there are no glyphs actually taking more than 2 cells.

I asked this question based on personal needs. I guess it seems that no one else has such a need at present, so it may not be a good idea to request for this feature in this project. And you probably won't be interested or have the time to do this, so my original intention was to ask you how to do with a font and implement it myself later.

Thank you again for your help! In the past few days of groping, I seem to have come to understand a little bit of what I can do with a font using python fontforge library.. Please let me know if you have more suggestions and comments on the irrationality of my goal. What tools do you recommend or do you usually use to modify and design a font?

[Finii]

What tools do you recommend or do you usually use to modify and design a font?

Well, I do not design fonts. Creating a good design is very very hard, and takes a lot of time. And there are so many good fonts out there already. I'm not shy actually paying for good fonts, as I know how much work it is. My go to source for buying is hide-advertisement

For modifying fonts, there are several alternatives. I use primarily https://fontforge.org, but it also has its quirks and bugs 😬 . Small tools are the pair showttf and ttfdump. For automation of modification one can use the fontforge Python interface https://fontforge.org/docs/scripting/python.html (but some features are missing there, even more than in the GUI version), the excellent https://fonttools.readthedocs.io have usually all the rest you need.

Turning to your goal. monospaced usually means that all glyphs have the same width, not some integer multiple of a given number. So just 1024 in your example. If there are glyphs 2048 wide, the font is NOT monospaced anymore.

This is usually overcome by letting the glyph 'stick out' to one side. Like the small-t in your name. It sticks out into previous 'cells' (which are not cells, but anyhow):

Here you see the glyph in detail

Ooops, thats the longer version. Anyhow, you see the 'cell'-like rectangular box. You can extend in all directions as much as you like (but some renderers might fail if you extend to much). You even might notice that it also extent into the next glyph (to the right) just a tiny bit.

So, to have a monospaced font, you need an 'advance width' that is the same for each glyph (the center box above), and then you can place what is displayed anywhere. (Well, if the renderer showing the font is capable of showing it).

What terminal emulators are able to render or choose to render is different all over. What you already have seen is that some do not allow glyphs to extend more then 1 advance width more to the right (i.e. '2 cells').

What I want to say is, you need to carefully set your goal, and check if that goal would be supported by any renderer.

The technical side is very simple, on the other hand. This is some python code that opens a font, and modifies the font

import psMat
import fontforge

font = fontforge.open(filename, 1)

for glyph in range(0x21, 0x17f):
    if not glyph in font:
        continue
    bb = font[glyph].boundingBox
    font[glyph].transform(psMat.scale(x_scale, y_scale))
    font[glyph].transform(psMat.translate(x_shift, y_shift))
    font[glyph].width = 1024
    font[glyph].right_side_bearing = ...
    font[glyph].left_side_bearing = ...

font.generate()

The Bounding Box is the total extend in absolute coordinates, you might base your scale on that.
Scale and shift as you like.
Keep in mind, that boundingBox[2] - boundingBox[0] is the total width.
The left, right, and width properties must equal the total width.
Ah, and don't forget to fetch the bb again after scale. There is some rounding in effect.

Hope that helps and gets you going for experiments :-)

Edit: Maybe this is also helpful: #731 (comment) It shows the correct terminology, that I did not use above

[Finii]

What I was trying to do was just to completely 'monospacify' a font

Ah, and wanted to add ... it is almost impossible to take a font designed to be proportionally spaced and press it into monospace. With proportional space fonts the letters like 'W' are often very very wide, while 'i' is slender. When the advance width is the same for both letters, it must be the wider one, probably, the negative space (free white area) between consecutive small letters looks extremely ugly. Imagine 'Wii' looks like Wi i .
The hard work of a font designer is to make monospaced fonts look aesthetically pleasing despite the fact that some letters are naturally wider than other, but that is not allowed to show.

Compare

• Avoid patching proportional sources #912

Edit: Change PR link to bullet point

[Finii]

Here for example with Caskaydia Cove, comparison before and after PR:

The right outline window is before the PR, the left/bottom one is after the PR. Note that now the clouds fit into a '2 cell wide' space. This is even more dramatic with ProggyClean:

Branch icon does not leak anymore into neigboring cell. The cloud with F00B was almost 3000 wide, while the cell is only 896 wide. Now it fits into 2 cells.

[Finii]

Oh my ....
these double development waste of time ...

now:
eb396e6 font-patcher: Remove overlap from line-ish glyphs

last year (hidden in other PR):
2d53894 font-patcher: Remove overlap for line style powerline glyphs

[Finii]

This closes PR #967 now.

Here some Issues that PR mentions:

• [Bug] Curved separators do not extend to full height of character box #661
• Symbols Only font with broken Powerline glyphs? #984