Noto complex variable fonts

[JelleBosmaMT]

I have been asked to share the findings of my investigation of making variable fonts for Noto complex scripts here on GitHub. It seems best to summarise this as one issue. This can always be split up into separate issues for the various projects that are part of fontmake, if the need arrises.

First I have to say, that if I generate a variable font from a Glyphs file with fontmake, I do not have the Glyphs.app hassle of having to add all sorts of custom parameters and to change values that I have to change back if I want to generate stand alone fonts. I like that.

GPOS:

There do not seem to be GPOS problems when we have external OTL sources in Monotype's FontDame format. (But I was surprised to see the decompiled binary class-kerning lookups using a different numbering as the sources, which makes it quite a puzzle to check the values.)

Noto Serif Devanagari and Noto Serif Kannada use embedded feature code with a dist feature with context rules. For the instances the dist feature is modified by a "replacement feature" custom parameter. These are lost when generating a variable font. The values for the regular are used for all instances. A possible solution might be to use instance UFO's rather than master UFO's to generate variable fonts (I will get back about that). Another option is to try Glyphs 3 ability to add number values to masters and refer to them in GPOS code.

For all complex scripts with Glyphs files with embedded feature code, the font generating tools are responsible for assembling kerning and attachment features. We discovered an anchor issue with Noto Serif Devanagari this week. From when we last supplied source files, I remember there were less noticeable issues with anchors too. It may make sense to get something in place to make certain it is working as should be. (It may also be a good idea to have the designer test their fonts before shipping.)

GSUB:

The Noto's Sans Myanmar, Sans Devanagari, Sans Gujarati and Serif Devanagari have more than one GSUB when we generate stand alone fonts. A am not familiar with the details of Myanmar, but the others have an i-vowel, with different lengths depending on the following base glyphs or half-vorm + base glyph with optional kerning between them. It is quite a puzzle to get a rule set right for 1 weight. Given that relative proportions of glyphs change depending on the weight of stems, we use different rules for weights to approximate the optimal i-vowel length.
For variable fonts we need to create a combined GSUB with the variants of the context lookups and a Feature Variation Table to select the rule needed depending on the axis location.

In principle it is possible to work out differences between source GSUBs for different positions on the axis and generate the variable GSUB on the fly. I haven't tried to do that myself.
But I did update my own old tool to support compiling and decompiling of feature variations. For the Myanmar where there are two GSUBs that differ by two context rules only, I created a combined GSUB in Monotype FontDame format with feature variations table and added that to a variable font generated earlier (without GSUB) with fontmake. That works for me, if I were called upon to make variable fonts today. If a hobby programmer like me can make a tool to do this, I guess support for feature variations can be added to mtiLib too and we supply variable GSUB sources.

The Noto Serif Devanagari has its GSUB feature variations stored as replacement features in instance custom parameters, which are ignored by fontmake and Glyphs 2. I do not think Glyphs 3 does any better, because currently it cannot generate font files for Serif Devanagari at all.
In theory Glyphs 3 has support for feature variations, so that may be a path to explore. I do not see an "official" support for feature variations in feature syntax specification of Adobe and current do not know how to make feature variations other than with my own tool or TTX.

document/UI sources:

In the scripts that have a single Glyphs file for document and UI versions there needs to be a solution for sub-setting, re-encoding, separate vertical metrics and the UI y-shift. I now get a document variable, which should be functional, but has all the UI specific glyphs that it does not need. The UI variable is the document variable with the OTL tables of the UI font and that does not work at all.

A simple solution is of course to have two Glyphs files. Not nice, if there is ever a need to update or extend the design.

When I look at fontmake, I see it creates a set of master UFOs and a design space file, that are used to generate font files from. For stand alone OTFs or TTFs, it first creates instance UFOs, for a variable the master UFOs are used directly. I think this works in most cases, but not always. In Glyphs the masters are containers for glyph data, and the font files are represented by instance definitions, that grab what they need from the masters. In our document/UI source files every master has two instances at the same axis location. If for every master you generate the matching instance UFO and use that as master to generate font files, I suspect that for stand alone font files the results would be the same as they are now, and we will get a UI and document variable font that are the same where they should be the same and different where they should be different.

The missing master issue:

In the old days of Ikarus, Fontographer and FontLab we could interpolate between 2 font files only. That meant that for a design space with masters in the corners, it did not matter what intermediate masters there were, as you would go trough steps to create intermediate masters anyway.

In Glyphs app all the masters are in one place, and the app tries to figure out interpolation values for any arbitrary combination of masters and instances, which may or may not have solutions which are ambiguous. But if we have masters in the corners of the design space and all the intermediate masters are on the edges of the design space, every instance on these edges is either a master or an unambiguous interpolation between two masters. And I think that settles what is in between the edges too.

For sources with weight and width axis that have 7 masters, the fontmake variable ExtraCondensed Bold is very different from the Glyphs interpolated. And the fontmake stand alone font is different too. I did not expect in 2015 that this could happen. It seems the master UFOs are treated as a variable font masters, rather than interpolation masters. But if you want a variable to match a set of masters designed for interpolation, we will have include the implicit masters that we get for free when interpolating.
I do not want to get into how to do that for any case, but in our simple case my thoughts are:

• We have a design space weight/width with masters in the corner.
• All masters are in corners or on the edge of the design space.
• For all masters there is an instance.
• All instances on the edges are paired on the opposite side of the design space.
  Then
• Generate instance UFO's for the masters
• Interpolate the instances that are paired with a master-instance between the nearest instance UFO's on its side of the design space. (Treat the edges as four 1 axis families.)
• Use the resulting instance UFOs as master for the complete set of individual font files or variable fonts.

That is what I would try if I would have to do everything with fontmake without touching the Glyphs file. Of course one could open Glyphs.app to create masters from instances, generate fonts, then remove these masters again. But having a tool that can deal with a wide range of possible sources is preferable.

GASP:

There is a very minimal GASP table. I could not help noticing when I tried out my experimental Sans Myanmar with feature variations in Windows that the little control panel item that allows you to try out variable fonts listed all 36 instances as "Regular". And I blame the GASP for that.

Note from two weeks later:
I was expecting that the instance UFOs produced by fontmake making fonts, would have had applied the instance properties of the Glyphs file instance definitions. In the actual instance UFO's that are currently generated that does not seem the case, and these properties are presumably applied when generating the font files. So when I mention to use instance UFO's to generate variable fonts, I should add that these should be instance UFO's as I expected them to be, not as they are now.

[punchcutter]

Quick minor notes:

Regarding AFDKO support for feature variations adobe-type-tools/afdko#153 has been open for a while, but we definitely want to get this figured out and implemented.

If Windows shows all instances as "Regular" then it's probably the STAT table. Windows uses the STAT to construct all instance names even though the fvar already has the names for those instances.

[simoncozens]

Hi Jelle. Apologies for chiming in out of the blue, but I think you might find this interesting...

I've been working on a layout editor similar to Microsoft VOLT, but one which deals properly with variable fonts. To do that, you either have to make the designer specify their layout rules for every master in the font (boring and repetitive) or you have to have the ability to create rules which can vary. And this is where I've hit some of the same problems that you have mentioned above: AFDKO not providing syntax for variable layout rules, fontmake having issues splitting and merging layout tables, and so on.

The dist issue is a particularly relevant one, as this is exactly the sort of thing a layout editor needs to deal with - manual positioning rules which vary between masters, in the context of a font with a single feature file (or equivalent).

What I've ended up doing it looking at the problem the other way around. Instead of using fontmake to generate per-master TTFs and then trying to merge them into a single variable font, why not just build the variable font straight from the Glyphs source? Take the deltas from the point positions and put them into the gvar table, take the deltas of the metrics and put them into the HVAR table, and so on.

The nice part about this is that building the GPOS/GSUB tables works the same way. When you have a mark attachment feature, for example, you just compute the deltas of the anchor positions and store those directly into an ItemVariationData table attached to the Anchor table.

And - if you have a syntax for layout rules which allows the numbers to vary - you can do the same thing with manual rules such as your dist feature as well. So I can create a contextual kern with a valuerecord which varies across the design space, and it gets compiled as normal but with an ItemVariationData record attached to it. As Zachary mentions, we will need to agree and implement syntax for this in AFDKO, but meanwhile I'm using my own layout language which already has this stuff built in.

I have some tooling which does the "variable-first" build process, but it's still in the development stage and not ready for prime-time yet. But I think when it is, going variable-first would solve a lot of the issues you mention.

Oh, and, it's faster. In fact, for fonts with a large number of masters, it's much faster - which should make sense, because you're only building one font, instead of building one font for each master, merging them into a variable font, and throwing the rest away.

[JelleBosmaMT]

I have been looking at Glyphs 3 to see if its support for feature variations would help us.

To make the feature variation table (needed for the GSUB of Noto Serif Devanagari) it offers a condition keyword, that is pretty much a high level equivalent of how it actually works in a binary table. In my experimental feature there is a lookup, followed by the replacements:

The part in between the condition is not much different from how I would decompile a feature variation table in FontDame format. (To investigate if the axis values are normalised taking the avar table values into account.)

The feature variation table replaces one set of lookups for a feature by an other set, for a segment of the design space. Of course GPOS values for the lookup formats used for kerning, cursive connection and mark attachment are valid for coordinates in design space: one value for the default with a reference to the item variation store delta's extracted from the other masters. And that should apply to values that end up in single positioning lookups too, whether or not they are used by a context rule. Up to recently we did not have a means to assign single positioning values to masters.

In Glyphs 3 we can add number values to masters and refer to them in feature code. I tried that, and it works for interpolating stand-alone fonts. But currently Glyphs 3 does not add the delta's to the item variation store when generating a variable font. It should work, but not yet.
Not sure if number values can work with the special layers for alternate point structures. But our Noto's do not have them, and even if they did, having proper support for feature variation tables means that we do not need the special layer solution.

@simoncozens @punchcutter
I tried to understand the discussion at adobe-type-tools/afdko#153. I do not know where that is going. Hopefully there will be one solution for the feature variation table, which does not have to be more advanced than what Glyphs offers. And an other solution to store GPOS values as a list of values, within a fixed lookup. Having a syntax that mixes the two, does not seem desirable.

[simoncozens]

Hopefully there will be one solution for the feature variation table, which does not have to be more advanced than what Glyphs offers. And an other solution to store GPOS values as a list of values, within a fixed lookup.

Exactly, yes.

Having a syntax that mixes the two, does not seem desirable.

No, nobody is suggesting that - just that the two syntaxes should feel similar, in terms of how they represent locations/masters etc.

[JelleBosmaMT]

In the mean time I discovered that fontmake likes to change spacing marks into non-spacing, based on the name. Which I have to fix by changing the official uniXXXX name into something that is not recognised.

[anthrotype]

glyphsLib assigns GDEF Mark class by looking up the glyph name in GlyphData.xml (when catetory is "Mark" and subCategory is "Nonspacing" or "Spacing combining"), but you can override that on selected glyphs via the Glyph property panel (CMD+ALT+I)

[JelleBosmaMT]

@anthrotype I forgot about that. Thanks, it saved a lot of trouble!