No GDEF ligature class is generated

[benkiel]

Working through some things in Recursive turned up the following (which may be a ufo2ft issue, it's a bit unclear where one is kicking in over the other).

Background:
I am building Recursive with a gdef table in feature code to include the ligature carets.

I've run into an issue with Recursive where if the base and mark glyph classes for gdef are defined in feature code, the variable font will not compile with a VarLibMergeError: ((2, [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]), 'int', '.LookupCount', 'Feature', '.Feature', 'FeatureRecord', '[0]', 'list', '.FeatureRecord', 'FeatureList', '.FeatureList', 'GPOS', '.table', 'table_G_P_O_S_') error. My best guess on this is that the addition of those two glyph classes mean that an extra lookup is added to the Recursive Sans fonts (because they have a lot of kerning) and not to the Mono fonts (as they have one kern pair).

I tried just specifying the ligature class, hoping that fontmake would fill in the base and mark groups, but it doesn't do this (for the very sane reason that it sees classes and doesn't try to change them).

The next logical thing would be to just let fontmake build the ligature classes. (for the record, fontmake deals with the written ligature caret code just fine (i.e., it shows up in the binary))

Issue

But, it appears that fontmake will not write gdef ligature classes. Recursive has ligature substitution rules in the feature code, but those glyphs are not added to a ligature class if I let fontmake/ufo2ft generate the GDEF. Also, no ligature carets are being written into the gdef table even if they are present in the font (as anchors starting with caret).

My expectation is that at the least, fontmake would fill in the ligature gdef glyph class with glyphs used in ligature substitution rules.

[anthrotype]

the ufo2ft MarkFeatureWriter checks for the presence of GDEF GlyphClassDef statements in the UFO features.fea and uses them to sort glyphs into base, mark and ligature buckets, excluding glyphs that aren't listed in there from its generated mark/mkmk feature definitions.
When an explicit GDEF is not present in features.fea, the MarkFeatureWriter treats glyphs with _-prefixed attaching anchors as mark glyphs, etc.

feaLib will use the explicit GDEF definition in features.fea to build the GDEF table. If it is not present in the features.fea, it will assign glyphs to the GDEF classes based on whether they are used in markToBase, markToMark and markToLiga lookups (search "inferGlyphClasses" in https://github.com/fonttools/fonttools/blob/main/Lib/fontTools/otlLib/builder.py).

Ideally one defines the GDEF classes in the features.fea and the rest just works.

Now the problem you're seeing with varLib failing to merge the masters' GPOS tables might be that for some masters you get both kern and mark lookups, for others only one of the two. VarLib currently requires that when merging GPOS tables, the masters all have the same number/order of lookups.

can you try installing ufo2ft from master branch and run python -m ufo2ft.featureWriters command with your various master UFOs and inspect the generated features.fea?

[anthrotype]

Recursive has ligature substitution rules in the feature code, but those glyphs are not added to a ligature class if I let fontmake/ufo2ft generate the GDEF.

it seems that feaLib/otlLib don't check ligature substitutions when they automatically infer glyph classes from lookups (i.e. in absence of explicit GDEF in features.fea), only the markToLigature positioning lookups are considered.

I believe (but not 100% sure) that the GDEF ligatures class is only relevant when one makes use of lookupflag ignoreLigatures.

[benkiel]

@anthrotype Yes, I understand that the ufo2ft MarkFeatureWriter looks for a GDEF, uses those glyphs if defined, but builds GDEF glyph classes if not defined. I was surprised that it doesn't build out a full list of the ligature glyphs in a font however; from what you wrote it's implied that the ligature class is filled with base glyphs from markToLiga, makeOTF fills that class with resulting glyphs from type 4 gsub lookups (at least from the liga and ccmp features, 100% if it grabs all type 4 lookups or narrows the scope to certain features)

(beat me to it while writing)

[anthrotype]

no ligature carets are being written into the gdef table even if they are present in the font (as anchors starting with caret)

I don't think "caret" anchors are supported currently in fontmake for designspace+ufo input; I think glyphsLib handles them by writing out LigatureCaret statements in its auto-generated GDEF. And the plan was to have a GDEFWriter in ufo2ft at some point to handle these /cc @moyogo @madig

[benkiel]

Correct on caret anchors, I just write out the GDEF for those per master

[anthrotype]

not that it's not that ufo2ft directly "builds GDEF glyph classes if not defined"; rather, feaLib builds GDEF table if one is not explicitly defined. And ufo2ft's heuristics on what is a mark, base or ligature in absence of explicit GDEF have an indirect influence on the auto-generated GDEF from feaLib. Does it make sense?

[anthrotype]

makeOTF fills that class with resulting glyphs from type 2 gsub lookups

might be worth filing an issue in fonttools about that. And check what the FEA spec says about it

[benkiel]

(the out of scope issue — I know this is likely not fontmake's problem!)

Now the problem you're seeing with varLib failing to merge the masters' GPOS tables might be that for some masters you get both kern and mark lookups, for others only one of the two. VarLib currently requires that when merging GPOS tables, the masters all have the same number/order of lookups.

Possible, may look into it. The weird issue I had was that the font would not compile to a variable font only when I specified the GDEF base and mark classes in the GDEF table. The feature code I'm using is from the markFeatureWriter, if I let fontmake fill in the GDEF classes, all is well. If I give it GDEF classes (all the same for each master, afaik) it fails to build.

Not a blocking issue, I just only write out the ligature caret GDEF for the variable sources, not the glyph classes.

[benkiel]

I don't think "caret" anchors are supported currently in fontmake for designspace+ufo input; I think glyphsLib handles them by writing out LigatureCaret statements in its auto-generated GDEF. And the plan was to have a GDEFWriter in ufo2ft at some point to handle these /cc @moyogo @madig

Dumb code to do that here: https://github.com/arrowtype/recursive/blob/948937aa33b1b04713627475e4ad75cb3e5cadf1/mastering/utils.py#L156

[benkiel]

might be worth filing an issue in fonttools about that. And check what the FEA spec says about it

Will do. Will close this also, as it seems not a fontmake issue.

FEA says:

If any mark class has been defined, or if any of the lookup flags for skipping glyphs of a certain class have been seen, the implementation will check if the GDEF keywords for defining the GlyphClassDef has been seen. If not, the implementation will fill them from the substitution and positioning rules, and will create a GDEF table even if there is no GDEF definition in the feature file. The LIGATURE and COMPONENT classes will be filled from the glyphs used in any GSUB Lookup Type 4, Ligature Substitution. The MARKS class will be filled from all the glyphs in any of the mark classes used in positioning rules.

https://adobe-type-tools.github.io/afdko/OpenTypeFeatureFileSpecification.html#9b-gdef-table

(so, I miswrote type 2 lookup above)

[benkiel]

I believe (but not 100% sure) that the GDEF ligatures class is only relevant when one makes use of lookupflag ignoreLigatures.

Good to know, thank you. I'm also unsure of what it's used for!

[madig]

Someone needs to write googlefonts/ufo2ft#456 for GDEF to be inferred from sources. :O