u+0268 in NotoSansDisplay Italic missing from soft-dotted GSUB rules

[jungshik]

Moved from googlei18n/noto-alpha#198 (reported by @roozbehp )

The 'ccmp' feature in the Noto LGC fonts that replace i, j, and U+0268 (ɨ, i with stroke) are incomplete. They need to include all soft-dotted character supported in the font.

The list of soft-dotted characters is provided at http://www.unicode.org/Public/UNIDATA/PropList.txt. These seem to be missing from the Noto LGC fonts:

012F ; Soft_Dotted # L& LATIN SMALL LETTER I WITH OGONEK
0249 ; Soft_Dotted # L& LATIN SMALL LETTER J WITH STROKE
029D ; Soft_Dotted # L& LATIN SMALL LETTER J WITH CROSSED-TAIL
02B2 ; Soft_Dotted # Lm MODIFIER LETTER SMALL J
03F3 ; Soft_Dotted # L& GREEK LETTER YOT
0456 ; Soft_Dotted # L& CYRILLIC SMALL LETTER BYELORUSSIAN-UKRAINIAN I
0458 ; Soft_Dotted # L& CYRILLIC SMALL LETTER JE
1D62 ; Soft_Dotted # Lm LATIN SUBSCRIPT SMALL LETTER I
1D96 ; Soft_Dotted # L& LATIN SMALL LETTER I WITH RETROFLEX HOOK
1DA4 ; Soft_Dotted # Lm MODIFIER LETTER SMALL I WITH STROKE
1DA8 ; Soft_Dotted # Lm MODIFIER LETTER SMALL J WITH CROSSED-TAIL
1E2D ; Soft_Dotted # L& LATIN SMALL LETTER I WITH TILDE BELOW
1ECB ; Soft_Dotted # L& LATIN SMALL LETTER I WITH DOT BELOW

For example, the sequence U+012F,U+0301 (small I with ogonek, combining acute) should remove the dot of the I, but it doesn't. Attaching a screenshot for Noto Sans and Serif.

Noto Sans :

Noto Serif:

[jungshik]

This is still broken in Noto Sans (p3) and Noto Serif.

In Noto Sans (p3), U+0268 (the 5th character from the left in the screenshot) regressed in that it's not treated as soft-dotted any more. Other characters listed above do get treated as soft-dotted, but the position of U+0301 is off.

[jungshik]

The following text was used to generate the images below:

í j́ į́ ɉ́ ɨ́ ʝ́ ʲ́ ϳ́ і́ ј́ ᵢ́ ᶖ́ ᶤ́ ᶨ́ ḭ́ ị́

Noto Sans (ttf - phase 2)

Noto Sans (otf built from glyphs source - p3)

Noto serif (the latest ttf)

[TrueTyper]

Hello Jungshik. The image you show of the phase 3 delivery of the Noto Sans does not match what I see in the Glyphs source we delivered, nor the sample binaries we exported and tested.

As you can see, everything is working with the exception of u+0268, which I have since fixed in source. I'm wondering if you used the custom GlyphData.xml file we provided with the delivery when you generated the binary? That is necessary to get the proper anchor point usage in the mark attachment features if you are generating the binary from Glyphs. I don't know how your own software stack will work, or what assumptions are made when interpreting anchor point data without the use of that GlyphData.xml.

Here is a screenshot of the latest source of the Sans.

We will replicate this in the future deliveries of the Serif Glyphs files as well.

[jungshik]

Thanks, @TrueTyper, for the update. I built phase 3 OT using Glyphs (instead of our pipeline which was not ready when I did it in September).

I've just talked to @behdad and it's very likely that I did not use GlyphsData.xml when building the font. Let me try it again. Thanks.

[jungshik]

I've just tested one instance (NotoSansUI-Regular.{otf, ttf}) created by our pipeline and this issue is fixed.

I haven't rebuilt fonts with Glyphs app, yet, but I don't expect any difference.

Thanks, @TrueTyper.

Noto to self and Googlers: we have to set up a test procedure/automation for all the instances.

[jungshik]

http://www.unicode.org/Public/UNIDATA/PropList.txt has more (I'm excluding math alphabet from the list):

2071          ; Soft_Dotted # Lm       SUPERSCRIPT LATIN SMALL LETTER I
2148..2149    ; Soft_Dotted # L&   [2] DOUBLE-STRUCK ITALIC SMALL I..DOUBLE-STRUCK ITALIC SMALL J
2C7C          ; Soft_Dotted # Lm       LATIN SUBSCRIPT SMALL LETTER J

[marekjez86]

Tested with the most recent delivery of
NotoSans
NotoSans Italic
NotoSansDisplay
NotoSansDisplay Italic
NotoSerif
NotoSerif Italic
NotoSerifDisplay
NotoSerifDisplay Italic

All fixed but u+0268 in NotoSansDisplay Italic (see the PDF files below):
en-418-NotoSans-Black.pdf
en-418-NotoSans-Italic.pdf
en-418-NotoSansDisplay-Black.pdf
en-418-NotoSansDisplay-Italic.pdf
en-418-NotoSerif-Black.pdf
en-418-NotoSerif-Italic.pdf
en-418-NotoSerifDisplay-Black.pdf
en-418-NotoSerifDisplay-Italic.pdf

@SteveMatteson : could you check on your side if this was fixed also in NotoSansDisplay Italic? If yes, then we might have a problem in our build or I just grabbed the wrong font to test with.

[marekjez86]

all fixed for the following test but u+0268 in NotoSansDisplay Italic (see png)

<html lang="en">
	<p style="font-weight:700">í j́ į́ ɉ́ ɨ́ ʝ́ ʲ́ ϳ́ і́ ј́ ᵢ́ ᶖ́ ᶤ́ ᶨ́ ḭ́ ị́</p>
</html>

en-418-NotoSans-Black.pdf
en-418-NotoSans-Italic.pdf
en-418-NotoSans-Regular.pdf
en-418-NotoSansDisplay-Black.pdf
en-418-NotoSansDisplay-Italic.pdf
en-418-NotoSansDisplay-Regular.pdf
en-418-NotoSansMono-Black.pdf
en-418-NotoSansMono-Regular.pdf
en-418-NotoSerif-Black.pdf
en-418-NotoSerif-Italic.pdf
en-418-NotoSerif-Regular.pdf
en-418-NotoSerifDisplay-Black.pdf
en-418-NotoSerifDisplay-Italic.pdf
en-418-NotoSerifDisplay-Regular.pdf
en-418-Roboto-Black.pdf
en-418-Roboto-Italic.pdf
en-418-Roboto-Regular.pdf

[moyogo]

0268 is wrong all over the place in many different ways. Here’s testing ɨɨ́ɨ̣ (0268 0268 0301 0268 0323):

[marekjez86]

I'm unable to reproduce this with the most recent build of fonts. The sources haven't changed hence this must have been something in the pipeline.

<html lang="en">
	<p style="font-weight:700">í j́ į́ ɉ́ ɨ́ ʝ́ ʲ́ ϳ́ і́ ј́ ᵢ́ ᶖ́ ᶤ́ ᶨ́ ḭ́ ị́</p>
	<p style="font-weight:700">ɨɨ́ɨ̣</p>
</html>

en-418-NotoSerif-Italic.pdf
en-418-NotoSerif-Regular.pdf
en-418-NotoSerifDisplay-Black.pdf
en-418-NotoSerifDisplay-Italic.pdf
en-418-NotoSerifDisplay-Regular.pdf
en-418-Roboto-Black.pdf
en-418-Roboto-Italic.pdf
en-418-Roboto-Regular.pdf
en-418-NotoSans-Black.pdf
en-418-NotoSans-Italic.pdf
en-418-NotoSans-Regular.pdf
en-418-NotoSansDisplay-Black.pdf
en-418-NotoSansDisplay-Italic.pdf
en-418-NotoSansDisplay-Regular.pdf
en-418-NotoSansMono-Black.pdf
en-418-NotoSansMono-Regular.pdf
en-418-NotoSerif-Black.pdf

[roozbehp]

Marek, look at the second line in your first PDF. You clearly see the bug still exists.

[marekjez86]

Thank you @roozbehp . I looked at the wrong results.

[dscorbett]

This bug also applies to the soft-dotted characters in Noto Sans Math version 2.000.

[nizarsq]

@moyogo Any opinion on the current rendering?

[moyogo]

It all looks good to me except for one thing.

Noto Sans Mono has an issue with U+1ECB LATIN SMALL LETTER I WITH DOT BELOW, it is substituted with uni1ECBdotless when combined with a mark above but uni1ECBdotless looks like a turned i instead of a dotless i with dot below.
Compare how ị́ looks in the Noto Serif in the test string, where it is correct.

[nizarsq]

It all looks good to me except for one thing.

Noto Sans Mono has an issue with U+1ECB LATIN SMALL LETTER I WITH DOT BELOW, it is substituted with uni1ECBdotless when combined with a mark above but uni1ECBdotless looks like a turned i instead of a dotless i with dot below.
Compare how ị́ looks in the Noto Serif in the test string, where it is correct.

Thank you