Add multi-channel SDF font generation and rendering support.

[bruvzg]

Support for M(T)SDF font rendering. Font texture is generated using https://github.com/Chlumsky/msdfgen.

Zoom video

Screen.Recording.2021-03-02.at.10.34.08.mov

Logos on background are rendered from 128x128 custom MSDF image.

• Added caching for multiple font variations.
• Added font importing, with character range pre-rendering option, font are imported to .fontdata (and set of textures) which store all its properties (see file description below). It's still possible to load any of .ttf / .otf / .font / .fnt files directly.
• Adds option to convert dynamic fonts to bitmap (removes source font file from import, which will disable advanced shaping and OpenType features for the font, and limits it to pre-rendered glyphs only).
• Adds option to override global oversampling value (set to 0 to restore default behavior).
• Adds M(T)SDF glyph and outline font support using combined multi-channel and true signed distance textures:
  

Import options:

antialiased - ignored for MSDF and bitmap fonts, if selected uses 8-bit greyscale rendering, otherwise uses 1-bit rendering.
force_autohinter - if selected prefers FreeType autohinter over font built-in hinter.
msdf - enables MSDF generation for dynamic fonts (or indicates that source is MSDF for bitmap fonts). MSDF generation is slow, but allows to use same texture for all fonts sizes, with almost no extra rendering cost.
msdf_px_range - MSDF pixel range (range around the shape between the minimum and maximum representable distance).
oversampling - font oversampling factor, ignored for MSDF and bitmap, if set to 0 global value is used.
hintinhg - ignored for bitmap fonts.
base_variation - default font size and variation, comma separated list of "size", "outline_size" or OpenType variation tags.
convert_to_bitmap - ignored for bitmap fonts, removes dynamic font source file form import (disables OpenType features, and limits font to pre-rendered glyphs only).
extra_spacing_* - extra spacing for glyphs and spaces.
preload\ranges - list of character ranges to preload in 'A'-'Z', U+0000-U+0000 or 0x0000-0x0000 format, or glyph ranges in G+0000-G+0000 format.
support_overrides - list of overrides of language and scripts, can be used to increase or decrease font fallback usage priority for specific language or script.

Supported import variants:

TrueType / OpenType source -> pre-renders glyphs and outlines for each size, original font file is preserved in cache and used for OpenType features, shaping and rendering missing glyphs.
TrueType / OpenType source + msdf selected -> Same as above but pre-renders glyphs as MSDF for one size only. Outlines are generated from the same MSDF. Generating MSDF require outline and will not work with bitmap fonts (e.g. emoji fonts).
TrueType / OpenType source + convert_to_bitmap selected -> Same as above, removes original font file from cache (font have no OpenType features and limited to pre-rendered glyphs only).
AngelCode bitmap font -> font is copied to cache as is (msdf flag should be set manually, if source image is MSDF or M(T)SDF).

".fontdata" file format:

==============================================================================================================================================
type                         name                         description / values
==============================================================================================================================================
U32                          magic                        ASCII "GDFT"
U8                           version_major                1
U8                           version_minor                0
U32                          font_type                    ASCII "BMP0" - bitmap font, "DYN0" - dynamic font
U32                          font_data_size               size of dynamic font data in memory (always 0 for bitmap fonts)
if font_data_size > 0:
    U32                      font_comp_size               size of dynamic font data in file
    U8[font_comp_size]       font_data                    ZSTD compressed dynamic font data (source ttf/otf file)
U8                           hinting                      0 - none, 1 - light, 2 - full (ignored for bitmap fonts)
U8                           msdf                         0 - non MSDF, 1 - MSDF
U8                           autohinter                   0 - use font built-in hinting, 1 - use auto hinter (ignored for bitmap fonts)
U8                           antialiased                  0 - 1-bit rendering, 1 - 8-bit rendering (ignored for MSDF and bitmap fonts)
U8                           spacing_glyph                extra glyph spacing in pixels
U8                           spacing_space                extra space spacing in pixels
F32                          oversampling                 oversampling factor (if <= 0 - global value is used, ignored for MSDF and bitmap fonts)
U32                          lang_support_overrides       number of language support overrides
"lang_support_overrides" records:
    U32                      lang_size                    length of language code string
    U8[lang_size]            lang_code                    language ISO code as UTF-8 string
    U8                       lang_enable                  0 - language not supported, 1 - language supported
U32                          script_support_overrides     number of script support overrides
"script_support_overrides" records:
    U32                      script_size
    U8[script_size]          script_code                  script ISO code as UTF-8 string
    U8                       script_enable                0 - script not supported, 1 - script supported
F32                          rect_margin                  extra spacing between glyphs in texture (ignored for bitmap fonts)
F32                          px_range                     MSDF pixel range (ignored for non MSDF fonts)
U32                          base_variation_tags          number of variation tags in base variation
"base_variation_tags" records:
    U32                      variation_tag                OpenType variation tag
    F64                      variation_value              value for variation tag
U16                          base_size                    base font size
U32                          cached_variations            number of cached font variants (always 1 for bitmap fonts)
"cached_variations" records:
    U32                      variation_tags               number of variation tags in this variation
    "variation_tags" records:
        U32                  variation_tag                OpenType variation tag
        F64                  variation_value              value for variation tag
    U32                      cached_sizes                 number of cached font sizes for this variation (always 1 for bitmap and MSDF fonts)
    "cached_sizes" records:
        U32                  size                         font size (always equal to base_size for bitmap fonts)
        F32                  ascent                       font ascent
        F32                  descent                      font descent
        F32                  height                       font height
        F32                  underline_position           font underline position
        F32                  underline_thickness          font underline thickness
        U32                  textures                     number of textures
        "textures" records:
            U32              width                        texture width
            U32              height                       texture height
            U32              offsets                      number of texture offsets
            U16[offsets]     offset_value                 texture offset values
            U32              name_size
            U8[name_size]    texture_name                 name of texture file
        U32                  glyphs                       number of cached glyphs
        "glyph" records:
            U32              index                        glyph index (for dynamic fonts) or UTF-32 code (for bitmap fonts)
            U8               found                        0 - glyph is not in the font (rest of the glyph data ignored) , 1 - glyph is in the font
            U16              texture_idx                  index of texture in the "textures" array
            F32              rect_pos_x
            F32              rect_pos_y                   glyph offset from the baseline
            F32              rect_size_x
            F32              rect_size_x                  glyph size
            F32              uv_pos_x
            F32              uv_pos_y
            F32              uv_size_x
            F32              uv_size_x                    glyph position and size in the texture
            F32              advance_x                    glyph advance for horizontal layout
            F32              advance_y                    glyph advance for vertical layout
        U32                  kerning_size                 number of kerning pairs (always 0 for dynamic fonts)
        "kerning_size" records:
            U32              glyph_index_a                first character UTF-32 code
            U32              glyph_index_b                second character UTF-32 code
            U32              value                        kerning offset value
==============================================================================================================================================

Bugsquad edit: This closes godotengine/godot-proposals#2021 and #28045 (by implementing per-font oversampling override).

[Calinou]

This looks great!

I noticed the MSDF becomes quite "thin" at lower zoom values. Is it possible to force using a minimal amount of sharpness/width for the MSDF font at the cost of worse antialiasing? This would help keep the font more readable at lower font sizes.

[bruvzg]

I noticed the MSDF becomes quite "thin" at lower zoom values. Is it possible to force using a minimal amount of sharpness/width for the MSDF font at the cost of worse antialiasing?

Probably, I have not tested any SDF generation parameters yet, and used the default shader with AA as is.

[fire]

I wonder how this approach compares? https://github.com/hypernewbie/VEFontCache

[jordi-star]

Is this for the master branch or 3.2?

[Calinou]

Is this for the master branch or 3.2?

This is for the master branch only, not 3.2.x. It can't be backported due to the reliance on backwards-incompatible TextServer changes.

[bruvzg]

Update:

Contour orientation detection and alignment issues are fixed, self intersection elimination is still desired, but rare and usually cause only small artifacts, probably can be omitted for now. Seems to work fine with all fonts I have tested.

Screenshots - click to expand

Stuff still missing (help needed):

• Outline drawing shader. Edit: Added from same texture, but blurred. (FT Stroker code is also added but commented for now)
• Some AA probably should be added to the shader. Edit: Changed format to 4-channel MTSDF, and fixed mismatching pixel ranges, now it's much better at small sizes.
  Current shader is doing absolute minimum:
  https://github.com/godotengine/godot/blob/490e3108c9618636d84c08a56b155ad6841c78ec/servers/rendering/renderer_rd/shaders/canvas.glsl#L457-L479

Other stuff:

• Import progress dialog is not displayed, am I missing something, or it's editor bug? Edit: It's working in single window mode.
• I'm not sure what's the best way to store cached font textures (right now they are store as pngs alongside fontdata).
• Also, who's responsible for deleting old files? I'm adding them to r_gen_files in importer, but files aren't deleted on reimport.

[bruvzg]

Also, exposed M(T)SDF drawing function to the canvas item for custom textures (draw_msdf_texture_rect_region).

Image taken from https://github.com/V-Sekai/godot-msdf-project/ (output.png).

[bruvzg]

msdfgen seems to have issues with sharp internal angles, usually resulting artifacts are small and barely noticeable. Unfortunately outlines generated by FreeType glyph stroker have tons of internal angles (in addition of common self-intersections) and the resulting distance field is total mess.

Distance filed for outline:

Small artifacts on the normal glyph (zoomed in to 3500%):

On the other hand, simply applying radial blur to the "true distance" component of M(T)SDF seems to have almost the same effect. Probably it's more than enough for all real use cases of the outline.

MSDF font on the top buttons, dynamic font on bottom buttons, scaled to: 25%, 50%, 75%, 100%, 200% and 500%.

[bruvzg]

I have changed outline rendering from "blur" to using max value in range, now MSDF font outlines look the same as dynamic font outlines at all sizes (right now MSDF font outline texture size is limited to 3 x glyph size, which should be more than enough for any real use case).

Also, added API for individual font oversampling override (which was used to disable oversampling for MSDF fonts).

Everything seems to be working as expected. I'll do a bit more testing and performance profiling, and it should be ready to go.

[reduz]

So far this looks good to me for the most part. I really dislike a custom binary format, though. Can we use regular resource serializing here?

[fire]

There may be some bug fixes for corner bugs. See https://github.com/Chlumsky/msdfgen/releases/tag/v1.9.1

[bruvzg]

Superseded by #51908