T1 tests fail under Strawberry perl 5.36 and 5.38, gcc13.1

[shawnlaffan]

Cross-ref StrawberryPerl/Perl-Dist-Strawberry#108

Imager is failing the T1 tests under Strawberry perl 5.36 and 5.38 RC (both currently in dev release status).

Both are compiled using gcc-13.1.

Test results and gdb backtrace are below.

FWIW, several other issues have arisen with gcc13 that were related to memory alignments (e.g. StrawberryPerl/Perl-Dist-Strawberry#93 (comment)). Perl 5.38 includes related changes and we are patching these onto 5.36. Possibly relevant is that MCE required a different approach.

If you need a build of perl 5.38.0 RC2 to test with then it can be downloaded from https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5.38.0_RC2_20230626_gcc13

gmake[1]: Entering directory 'C:/spbuild/msys64/.cpanm/work/1687746800.2752/Imager-1.019/T1'
"C:\strawberry\perl\bin\perl.exe" -MExtUtils::Command::MM -e cp_nonempty -- T1.bs ..\blib\arch\auto\Imager\Font\T1\T1.bs
 644
"C:\strawberry\perl\bin\perl.exe" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test
_harness(0, '..\blib\lib', '..\blib\arch')" t/*.t
t/t10type1.t ... Dubious, test returned 5 (wstat 1280, 0x500)
All 20 subtests passed
t/t20oo.t ...... Dubious, test returned 5 (wstat 1280, 0x500)
Failed 13/14 subtests
t/t30thread.t .. Dubious, test returned 5 (wstat 1280, 0x500)
Failed 8/8 subtests
t/t90std.t ..... Dubious, test returned 5 (wstat 1280, 0x500)
Failed 21/21 subtests

Test Summary Report
-------------------
t/t10type1.t (Wstat: 1280 (exited 5) Tests: 20 Failed: 0)
  Non-zero exit status: 5
  Parse errors: No plan found in TAP output
t/t20oo.t    (Wstat: 1280 (exited 5) Tests: 1 Failed: 0)
  Non-zero exit status: 5
  Parse errors: Bad plan.  You planned 14 tests but ran 1.
t/t30thread.t (Wstat: 1280 (exited 5) Tests: 0 Failed: 0)
  Non-zero exit status: 5
  Parse errors: Bad plan.  You planned 8 tests but ran 0.
t/t90std.t   (Wstat: 1280 (exited 5) Tests: 0 Failed: 0)
  Non-zero exit status: 5
  Parse errors: Bad plan.  You planned 21 tests but ran 0.
Files=4, Tests=21,  1 wallclock secs ( 0.05 usr +  0.00 sys =  0.05 CPU)
Result: FAIL
Failed 4/4 test programs. 0/21 subtests failed.
gmake[1]: *** [makefile:837: test_dynamic] Error 5
gmake[1]: Leaving directory 'C:/spbuild/msys64/.cpanm/work/1687746800.2752/Imager-1.019/T1'
gmake: *** [makefile:1357: subdirs-test_dynamic] Error 2

gdb backtrace (SP build extracted to a different machine):

(gdb) set args -Mblib t\t20oo.t
(gdb) set disable-randomization off
(gdb) run
Starting program: D:\5.38.0RC2_20230626\perl\bin\perl.exe -Mblib t\t20oo.t
[New Thread 19928.0x3348]
[New Thread 19928.0x4e18]
[New Thread 19928.0x1974]
1..14
ok 1 - make output directory

Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ff9db08d763 in strncpy () from C:\WINDOWS\System32\msvcrt.dll
(gdb) bt
#0  0x00007ff9db08d763 in strncpy () from C:\WINDOWS\System32\msvcrt.dll
#1  0x00000000567d958f in ?? () from D:\5.38.0RC2_20230626\c\bin\libt1-5__.dll
#2  0x00000000567e9552 in ?? () from D:\5.38.0RC2_20230626\c\bin\libt1-5__.dll
#3  0x00000000567e01a4 in ?? () from D:\5.38.0RC2_20230626\c\bin\libt1-5__.dll
#4  0x00007ff9a945315d in boot_Imager__Font__T1 ()
   from D:\5.38.0RC2_20230626\data\.cpanm\work\1687769011.2088\Imager-1.019\blib\arch\auto\Imager\Font\T1\T1.xs.dll
#5  0x00007ff9a94529b0 in ?? ()
   from D:\5.38.0RC2_20230626\data\.cpanm\work\1687769011.2088\Imager-1.019\blib\arch\auto\Imager\Font\T1\T1.xs.dll
#6  0x00007ff958df2fec in perl538!Perl_clear_defarray () from D:\5.38.0RC2_20230626\perl\bin\perl538.dll
#7  0x00007ff958e47982 in perl538!Perl_runops_standard () from D:\5.38.0RC2_20230626\perl\bin\perl538.dll
#8  0x00007ff958e0368b in perl_run () from D:\5.38.0RC2_20230626\perl\bin\perl538.dll
#9  0x00007ff958e56e73 in perl538!RunPerl () from D:\5.38.0RC2_20230626\perl\bin\perl538.dll
#10 0x00007ff7f6821340 in ?? ()
#11 0x00007ff7f6821146 in ?? ()
#12 0x00007ff9db197614 in KERNEL32!BaseThreadInitThunk () from C:\WINDOWS\System32\kernel32.dll
#13 0x00007ff9dd0226f1 in ntdll!RtlUserThreadStart () from C:\WINDOWS\SYSTEM32\ntdll.dll
#14 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

[tonycoz]

Is this exactly the same libt1 that shipped with strawberry perl 5.32.1.1?

[shawnlaffan]

It's the same libt1 code but compiled using gcc-13.1.

Source is http://www.ibiblio.org/pub/linux/libs/graphics/t1lib-5.1.2.tar.gz

Last change to the Strawberry build was four years ago; https://github.com/StrawberryPerl/build-extlibs/blame/master/sources.list#L169

Edit: Build args if relevant:
https://github.com/StrawberryPerl/build-extlibs/blob/56e3a3b834962b7cdcb459247b30514d213207ee/build.sh#L1313-L1322

[tonycoz]

Would it be possible to produce a backtrace with symbols in libt1 and in Imager::Font::T1? (it looks like it only has the exports available as symbols for Imager::Font::T1)

The two issues you've referenced seem irrelevant:

• the vmem.h change is part of perl 5.38
• Imager doesn't use pack() or unpack() to create pointers, or to pass values to ioctl() or similar

[shawnlaffan]

Would it be possible to produce a backtrace with symbols in libt1 and in Imager::Font::T1? (it looks like it only has the exports available as symbols for Imager::Font::T1)

I just rebuilt libt1 with CFLAGS="-g" but get the same backtrace. Possibly I need to add extra args. Please let me know.

The two issues you've referenced seem irrelevant:

They were provided given the other gcc-13 issues we'd hit. Probably not surprising they seem not relevant.

[tonycoz]

I did some testing with 20230705 with the built t1lib from 20230626 and reproduced it locally.

This means I don't have debug info for that file either, and I don't have the tooling setup you're using to build t1lib, but I did build Imager with debug symbols.

The crash is happening when T1_LoadFont() is called and the name of the font metrics file has been set, and currently that's always set (and the tests also test setting the font metrics file name.)

From looking at t1lib.log it appears to happen somewhere after line 533 in t1load.c, since the last thing logged is:

(S) T1_LoadFont(): Alert: Error (-5) sloppy-processing afm-file for Font 1!

I couldn't find a suspect strncpy() for this case, though it's possible the lack of debug info for msvcrt means the identification as strncpy() is wrong.

You mentioned supplying CFLAGS=-g, you may also need to add -g to LDFLAGS to get symbols into the final t1lib.

[shawnlaffan]

Adding -g to LDFLAGS seems not to make a difference.

A build of t1lib using those flags is attached.

Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ffe6f51db93 in strncpy () from C:\Windows\System32\msvcrt.dll
(gdb) bt
#0  0x00007ffe6f51db93 in strncpy () from C
[64bit_t1lib-5.1.2-bin_20230707.zip](https://github.com/tonycoz/imager/files/11979869/64bit_t1lib-5.1.2-bin_20230707.zip)
:\Windows\System32\msvcrt.dll
warning: could not convert 'main' from the host encoding (CP65001) to UTF-32.
This normally should not happen, please file a bug report.
#1  0x00000000612f5d49 in ?? () from C:\strawberry\c\bin\libt1-5__.dll
#2  0x0000000061310947 in ?? () from C:\strawberry\c\bin\libt1-5__.dll
#3  0x00000000613012fc in ?? () from C:\strawberry\c\bin\libt1-5__.dll
#4  0x00007ffe63e54bdb in boot_Imager__Font__T1 ()
   from C:\spbuild\msys64\.cpanm\work\1688718128.2480\Imager-1.019\blib\arch\auto\Imager\F
ont\T1\T1.xs.dll
#5  0x00007ffe63e51b59 in ?? ()
   from C:\spbuild\msys64\.cpanm\work\1688718128.2480\Imager-1.019\blib\arch\auto\Imager\F
ont\T1\T1.xs.dll
#6  0x00007ffe64e02fec in perl538!Perl_clear_defarray ()
   from C:\strawberry\perl\bin\perl538.dll
#7  0x00007ffe64e57932 in perl538!Perl_runops_standard ()
   from C:\strawberry\perl\bin\perl538.dll
#8  0x00007ffe64e1363a in perl_run () from C:\strawberry\perl\bin\perl538.dll
#9  0x00007ffe64e66e23 in perl538!RunPerl () from C:\strawberry\perl\bin\perl538.dll
#10 0x00007ff798501340 in ?? ()
#11 0x00007ff798501146 in ?? ()
#12 0x00007ffe70287ac4 in KERNEL32!BaseThreadInitThunk ()
   from C:\Windows\System32\kernel32.dll
#13 0x00007ffe72aea351 in ntdll!RtlUserThreadStart () from C:\Windows\SYSTEM32\ntdll.dll
#14 0x0000000000000000 in ?? ()
[64bit_t1lib-5.1.2-bin_20230707.zip](https://github.com/tonycoz/imager/files/11979882/64bit_t1lib-5.1.2-bin_20230707.zip)
[64bit_t1lib-5.1.2-bin_20230707.zip](https://github.com/tonycoz/imager/files/11979892/64bit_t1lib-5.1.2-bin_20230707.zip)

Backtrace stopped: previous frame inner to this frame (corrupt stack?)

64bit_t1lib-5.1.2-bin_20230707.zip

[tonycoz]

Are you building 32-bits of strawberry perl too? If so do they exhibit the same problem?

I managed an adhoc build of the t1lib dll with debug info and got the following backtrace:

Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ffb6337d763 in strncpy () from C:\Windows\System32\msvcrt.dll
(gdb) bt
#0  0x00007ffb6337d763 in strncpy () from C:\Windows\System32\msvcrt.dll
#1  0x00007ffb1e3f4139 in T1_GetAllCharNames (FontID=1) at t1finfo.c:1086
#2  0x00007ffb1e40ed27 in T1_GenerateAFMFallbackInfo (FontID=1) at t1afmtool.c:160
#3  0x00007ffb1e3ff6dc in T1_LoadFont (FontID=1) at t1load.c:535
#4  0x00007ffb1fe04c2b in i_t1_new (pfb=0x1e3f12c1220 "fontfiles/ExistenceTest.pfb",
    afm=0x1e3f12c11b0 "fontfiles/ExistenceText.afm") at imt1.c:147
#5  0x00007ffb1fe01bae in XS_Imager__Font__T1xs_new (my_perl=0x1e3eef64fc0, cv=0x1e3f186c7d0) at T1.c:215
#6  0x00007ffb20072fec in perl538!Perl_clear_defarray ()
   from c:\strawberry-perl-5.38.0.1-64bit-portable-20230705\perl\bin\perl538.dll
#7  0x00007ffb200c7932 in perl538!Perl_runops_standard ()
   from c:\strawberry-perl-5.38.0.1-64bit-portable-20230705\perl\bin\perl538.dll
#8  0x00007ffb2008363a in perl_run () from c:\strawberry-perl-5.38.0.1-64bit-portable-20230705\perl\bin\perl538.dll
#9  0x00007ffb200d6e23 in perl538!RunPerl ()
   from c:\strawberry-perl-5.38.0.1-64bit-portable-20230705\perl\bin\perl538.dll
#10 0x00007ff6b91a1340 in ?? ()
#11 0x00007ff6b91a1146 in ?? ()
#12 0x00007ffb62977614 in KERNEL32!BaseThreadInitThunk () from C:\Windows\System32\kernel32.dll
#13 0x00007ffb643e26f1 in ntdll!RtlUserThreadStart () from C:\Windows\SYSTEM32\ntdll.dll
#14 0x0000000000000000 in ?? ()

note this matches the other backtraces we've done.

The code around the call to strncpy():

  namedest=(char *)((long)bufmem + nameoffset);
  j=0;
  for ( i=0; i<len; i++){
    bufmem[i]=&(namedest[j]);
    strncpy( &(namedest[j]), pCharStrings[i+1].key.data.nameP,
	     pCharStrings[i+1].key.len);
    j += pCharStrings[i+1].key.len;
    namedest[j++]='\0';
  }
  bufmem[i++]=NULL;

and the variables involved:

(gdb) p namedest
$1 = 0xffffffffeee29ee8 <error: Cannot access memory at address 0xffffffffeee29ee8>
(gdb) p bufmem
$2 = (char **) 0x1e3eee29ec0
(gdb) p nameoffset
$3 = 40
(gdb) p sizeof(long)
$4 = 4
(gdb) p sizeof(void *)
$5 = 8

The cast to (long) is breaking the pointer (from a 64-bit pointer to a 32-bit signed integer and then back), and t1lib contains many casts of pointers to long.

I expect the older t1lib was patched to fix this.

This isn't a problem on Linux since long is 64-bits there.

[shawnlaffan]

Are you building 32-bits of strawberry perl too? If so do they exhibit the same problem?

Only 64 bit at this stage.

I managed an adhoc build of the t1lib dll with debug info and got the following backtrace:

Thanks for getting that running.

The cast to (long) is breaking the pointer (from a 64-bit pointer to a 32-bit signed integer and then back), and t1lib contains many casts of pointers to long.

I expect the older t1lib was patched to fix this.

That might be the case, but if so the patches were not uploaded to the git repo.

Is this is a case of running the code through sed (or perl) to change the cast? Or am I being needlessly optimistic?

[tonycoz]

There might be non-pointer casts to long too, so a simple search and replace won't work.

The build had many warnings like:

t1finfo.c:1082:21: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
 1082 |   namedest=(char *)((long)bufmem + nameoffset);
      |                     ^
t1finfo.c:1082:12: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
 1082 |   namedest=(char *)((long)bufmem + nameoffset);
      |            ^

which I expect reveals most cases of these problems.

While fixing these are good, I know Debian (used too) and other Linux distributions (I think at least archlinux) also include security fixes for t1lib which fix various potential security issues, though I haven't looked at them in detail.

Debian dropped t1lib since there was no upstream maintainer,

You can build Imager without t1lib by supplying --disable=T1 to Imager's Makefile.PL, or by setting all of the submodules you want to keep in the IM_ENABLE environment variable.

[shawnlaffan]

From the link it looks like Debian dropped support about a decade ago. Adobe has also EOL'd support for T1 fonts as of January this year.

Strawberry Perl can follow suit, and this issue can be closed.

Thanks for digging into the code and identifying the root cause of the issue.

[tonycoz]

Note that you still have Type 1 font support through freetype2.

[tonycoz]

Installing Imager no longer installs Imager::Font::T1 by default.