Font spacing changes

[NathanWailes]

What did you do?

1. I'm using Pillow to create many individual images which I then stitch together to create a lyrics video for a song.
   • The app is online at https://www.rhymecraft.guru
2. One of the features of the videos created by my app is that, while showing an entire line of lyrics on-screen at once, the currently-pronounced syllable can be highlighted.
   • You can see an example here: https://www.youtube.com/watch?v=9u9ylseLAfU
3. The way I achieve this is by:
   1. first calculating where the X, Y coordinates of the line would start if the entire line of lyrics was drawn at once
   2. then drawing the text preceding the highlighted syllable and recording the width of that text,
   3. then drawing the highlighted syllable, using the original X, Y coordinates and the width of the preceding text to calculate where the highlighted syllable's X, Y coordinates should start,
   4. then proceeding in the same way to draw the text following the highlighted syllable.

What did you expect to happen?

I expected that the current syllable would change color without any other visible changes.

What actually happened?

The text spacing always doesn't seem to work totally right, so the lyrics sometimes get spaced out.

You can see it:

• caused by the word 'of' at 0:47: https://www.youtube.com/watch?v=9u9ylseLAfU&t=47s
• in the word 'paradise' at 1:03: https://www.youtube.com/watch?v=9u9ylseLAfU&t=1m3s

What are your OS, Python and Pillow versions?

• OS: Windows 10 (local machine), Ubuntu (Digital Ocean)
• Python: 3.7
• Pillow: 5.0.0

Example code:

from PIL import ImageDraw, Image, ImageFont

img = Image.new('RGBA', (200, 60), color=(0, 0, 0))
draw = ImageDraw.Draw(img)
font = ImageFont.truetype('arial.ttf', size=48)
first_font_color = (255, 0, 0)
second_font_color = (255, 255, 255)

draw.text((0, 0), 'paradise', first_font_color, font=font)

draw.text((0, 0), 'par', second_font_color, font=font)
par_width, par_height = draw.textsize('par', font=font)
draw.text((0 + par_width, 0), 'adise', second_font_color, font=font)

img.save('example.png')

There's first a red 'paradise' created, then a white 'paradise' created in the same spot. If you zoom in you'll notice that the red one shows up underneath the white one starting with the 'a' and then that continues for the 'dise':

[radarhere]

Could you put together a simple script demonstrating the problem?

[NathanWailes]

@radarhere I've updated the first post with a simple example.

[radarhere]

What do you think of working from the end instead, subtracting the width of 'adise' from the width of 'paradise'?

from PIL import ImageDraw, Image, ImageFont

img = Image.new('RGBA', (200, 60), color=(0, 0, 0))
draw = ImageDraw.Draw(img)
font = ImageFont.truetype('arial.ttf', size=48)
first_font_color = (255, 0, 0)
second_font_color = (255, 255, 255)

draw.text((0, 0), 'paradise', first_font_color, font=font)

draw.text((0, 0), 'par', second_font_color, font=font)
par_width = draw.textsize('paradise', font=font)[0] - draw.textsize('adise', font=font)[0]
draw.text((0 + par_width, 0), 'adise', second_font_color, font=font)

img.save('example.png')

As to why this works better, I think it's because not all gaps between letters are the same. If you look at the pixels, there is zero gap between 'r' and the following 'a', but there are 3 pixels between the second 'a' and the following 'd'. On an abstract level, I think this also better describes your goal - you're not writing 'adise' after 'par' - you're writing 'aside' at the end of 'paradise'.

[NathanWailes]

@radarhere Thanks for the suggestion, I'll give that a try. I didn't think to try that because I didn't know it would make a difference. I'll close the ticket for now and reopen it if it doesn't work for some reason (although I can see that for this example it does work).

[NathanWailes]

That proposed fix does not seem to work in every case. As an example, consider the string "Ayo", with the "A" and "yo" being drawn separately:

from PIL import ImageDraw, Image, ImageFont

img = Image.new('RGBA', (200, 60), color=(0, 0, 0))
draw = ImageDraw.Draw(img)
font = ImageFont.truetype('arial.ttf', size=48)
first_font_color = (255, 0, 0)
second_font_color = (255, 255, 255)

draw.text((0, 0), 'Ayo', first_font_color, font=font)

draw.text((0, 0), 'A', second_font_color, font=font)
par_width = draw.textsize('Ayo', font=font)[0] - draw.textsize('yo', font=font)[0]
draw.text((0 + par_width, 0), 'yo', second_font_color, font=font)

img.save('example.png')

Here's the result:

[radarhere]

I'm not able to replicate this on my macOS machine. Does this happen for you on Ubuntu?

While unlikely, I don't suppose that the problem is just that some of the pixels being drawn are translucent?

[NathanWailes]

@radarhere I just confirmed it's happening on my Digital Ocean (Ubuntu) machine.

The issue seems to be the kerning between the letters 'A' and 'y'. I tried a fix where I would manually subtract from the preceding_text_width if the preceding_text_width + following_text_width was larger than the full_text_width (as recommended here), but that also does not work consistently, as different letter pairs seem to have different amounts of kerning even after making that adjustment.

[radarhere]

You may or may not be interested in this - as a workaround for your situation, instead of drawing 'Ayo' in red and then 'A' and 'yo' in white and finding they don't quite match, you could instead draw 'A' and 'yo' in red and then 'A' and 'yo' in white, always breaking the writing up into the final segments.

[NathanWailes]

@radarhere That workaround had occurred to me, and I'll keep it in mind, but I'm going to see if I can make it work with the kerning. I looked around and I think I found a way to extract the kerning values from the font's TTF file using the 'fontforge' library, and I'm going to see if I can use that to reliably adjust the distance between the two characters.

[radarhere]

Despite not being able to replicate, how's this for another idea?

If the problem is that the 'y' in 'Ayo' is moved right by the presence of the 'A', then what if we always wrote 'Ayo', and cropped and pasted the result to get the effect?

from PIL import ImageDraw, Image, ImageFont

img = Image.new('RGBA', (200, 220), color=(0, 0, 0))

draw = ImageDraw.Draw(img)
font = ImageFont.truetype('Arial.ttf', size=48)
first_font_color = (255, 0, 0)
second_font_color = (255, 255, 255)

for i, parts in enumerate([
	['Ayo',''],
	['A','yo'],
	['Ay','o'],
	['','Ayo']
]):
	first, second = parts
	x = 0
	y = i*50
	if first and second:
		draw.text((x, y), 'Ayo', first_font_color, font=font)

		im2 = Image.new('RGBA', (200, 220))
		draw2 = ImageDraw.Draw(im2)
		draw2.text((0, 0), 'Ayo', second_font_color, font=font)
		w = draw2.textsize('Ayo', font=font)[0] - draw2.textsize(second, font=font)[0]
		cropped_im = im2.crop((w, 0, 220, 220))
		img.paste(cropped_im, (x+w, y), cropped_im)
	elif first:
		draw.text((x, y), 'Ayo', first_font_color, font=font)
	else:
		draw.text((x, y), 'Ayo', second_font_color, font=font)

img.save('example.png')

[NathanWailes]

Thank you for the suggestion! One issue with this method is that sometimes one of the letters at the border will have a mix of the two colors (see the example below, I suspect the issue will be more dramatic in, for example, script-style fonts, which I'd like to be able to support):

I've taken a short break from this problem but my current idea is still to try to use pangocairocffi to solve this, see the linked cairocffi thread to see the problem I'm currently dealing with with that library (not that it concerns you; just if you're interested).

[radarhere]

Ok. If you're looking into other libraries, can this issue be closed for Pillow, or are you still interested in seeing a solution here?

[NathanWailes]

@radarhere Yeah it's fine to close it, I'll reopen it if the other library doesn't work. But it would be nice if Pillow had this functionality.

[radarhere]

The problem, at least for me, is that I can't replicate this - for my machine, this can be done with Pillow.

If you've moved on from using Pillow for this, then maybe you aren't interested in hearing my hit-and-miss ideas on how to address this - I don't currently have another way to figure out if this is solved apart from talking here. If you haven't moved on, then you can re-open this issue.

[NathanWailes]

@radarhere I would prefer to use Pillow since it would involve changing less of my code, but after learning more about kerning it seems to me that this problem/use-case is one that would require new code/features added to Pillow to get it working.

When you say you can't replicate it, does that include the "VA" example I gave above?

[radarhere]

When I run my code with 'VA', there is a problem, but the pixels do not exactly match yours. However, that seems irrelevant, since that approach requires separation between the letters, and it now sounds like you want to use slanted text.

[NathanWailes]

Just to be clear: I don't want to solely use slanted text, but I do want to be able to support slanted text. I want to be able to support all of the Google free fonts.

[nulano]

I'm pretty sure the cause of this issue is that sometimes glyphs extend outside their advance width.

I'll use the terms from the following diagram to explain this:

(from https://www.freetype.org/freetype2/docs/tutorial/step2.html#section-1)

The origin refers to the position at which a glyph is supposed to be rendered. For the first glyph, this refers to the xy parameter of the draw.text call, for subsequent glyphs it is incremented by the advance value. Typically, getsize should return the distance from the first origin to the last origin (sum of glyph advance values). There are two exceptions:

• If the first glyph has a negative bearingX. In this case, the C code will add the amount by which the glyph extends in front of its origin to the width, but it will also return the amount by which it is adjusted, which is then subtracted in Python code. Therefore this case doesn't cause issues. Edit: In Character bounding boxes and negative offsetx #4789 I realized that this value is added twice in C code. To account for this, you have to add the (negative) X offset from getoffset to the returned width.
• If bearingX plus width of the last glyph sum to a value larger than its advance. In this case C code adds the extra margin to the returned width, but it is not subtracted in Python code. This offset makes it impossible to use Pillow to reliably measure text width. Specifically, the glyph for 'r' in Arial has (rounded) bearingX=3px, width=14px, advance=16px. This gives an error of (14+3)-16=1px. I'm not sure if there is a good way to handle this with the current API while preserving backwards compatibility without adding a new function. Fix return value of FreeTypeFont.textsize() does not include font offsets #784 (comment) looks like a good suggestion.

Kerning could also affect this spacing, but the current implementation looks buggy to me. Specifically, kerning is being scaled twice, once in the basic layout function and a second time in the getsize and render functions. The effect is that the delta is scaled to 1/64 of the intended offset. The PIXEL macro call on the following lines looks unnecessary and wrong (added in #2576 with #2284):

Pillow/src/_imagingft.c

Lines 568 to 569 in 394f7a0

	(*glyph_info)[i-1].x_advance += PIXEL(delta.x);
	(*glyph_info)[i-1].y_advance += PIXEL(delta.y);

Note that there are further complications if you use Raqm layout instead of basic layout, see #4483 (comment).

[radarhere]

With the code from the original post, on my machine at least, I get

If I replace

par_width, par_height = draw.textsize('par', font=font)

with

par_width = draw.textlength('par', font=font)

I get

Is that it?

[NathanWailes]

@radarhere Thank you for letting me know about textlength! I switched the code to use it, and while the wiggling is significantly less than before, it's still wiggling. You can see an example here: https://youtu.be/LcNC5JRNL3o

[radarhere]

I've figured out the last piece, creating PR #6722 to resolve this.

Using textlength from my previous comment with that PR, the red vertical line at the end of 'i' is gone.