Assertion fails in multi-char regex match

[p5pRT]

Migrated from rt.perl.org#90536 (status was 'resolved')

Searchable as RT90536$

[p5pRT]

From @khwilliamson

This is a bug report for perl from khw@​karl.(none),
generated with the help of perlbug 1.39 running under perl 5.14.0.

---

% perl -E '"s\N{U+DF}" =~ /\x{00DF}/i'
perl​: re_exec.c​:6740​: S_reginclass​: Assertion `*lenp != 0' failed.
Abort

To occur, it has to be /i matching, have a char that has a multi-char
fold on both sides, and to match partially through the fold of the lhs
char

---

---

Flags​:
  category=core
  severity=high

---

Site configuration information for perl 5.14.0​:

Configured by khw at Fri May 13 07​:08​:22 MDT 2011.

Summary of my perl5 (revision 5 version 14 subversion 0) configuration​:
  Commit id​: eb70bb4
  Platform​:
  osname=linux, osvers=2.6.35-28-generic-pae,
archname=i686-linux-thread-multi-64int-ld
  uname='linux karl 2.6.35-28-generic-pae #50-ubuntu smp fri mar 18
20​:43​:15 utc 2011 i686 gnulinux '
  config_args='-des -Dprefix=/home/khw/fastbleadperl -Dusedevel
-Dman1dir=none -Dman3dir=none -Dusemorebits -Dusethreads'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=undef, uselongdouble=define
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
  optimize='-O2',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
  ccversion='', gccversion='4.4.5', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
  ivtype='long long', ivsize=8, nvtype='long double', nvsize=12,
Off_t='off_t', lseeksize=8
  alignbytes=4, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
  libpth=/usr/local/lib /lib/../lib /usr/lib/../lib /lib /usr/lib
/usr/lib/i686-linux-gnu
  libs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
  perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
  libc=/lib/libc-2.12.1.so, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version='2.12.1'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib
-fstack-protector'

Locally applied patches​:
  RC3

---

@​INC for perl 5.14.0​:
  /home/khw/perl/fastblead/lib

/home/khw/fastbleadperl/lib/site_perl/5.14.0/i686-linux-thread-multi-64int-ld
  /home/khw/fastbleadperl/lib/site_perl/5.14.0
  /home/khw/fastbleadperl/lib/5.14.0/i686-linux-thread-multi-64int-ld
  /home/khw/fastbleadperl/lib/5.14.0
  /home/khw/fastbleadperl/lib/site_perl
  .

---

Environment for perl 5.14.0​:
  HOME=/home/khw
  LANG=en_US.UTF-8
  LANGUAGE=en_US​:en
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)

PATH=/home/khw/bin​:/home/khw/print/bin​:/bin​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/bin​:/sbin​:/usr/games​:/home/khw/cxoffice/bin
  PERL5OPT=-w
  PERL_BADLANG (unset)
  SHELL=/bin/ksh

[p5pRT]

From @khwilliamson

See attached patch

[p5pRT]

From @khwilliamson

0001-Assertion-fails-in-multi-char-regex-match.patch

From 9d6ceae4ca52cbe80bba8e1819ccb48779d5797b Mon Sep 17 00:00:00 2001
From: Karl Williamson <public@khwilliamson.com>
Date: Fri, 13 May 2011 08:35:23 -0600
Subject: [PATCH] Assertion fails in multi-char regex match

In '"s\N{U+DF}" =~ /\x{00DF}/i, the LHS folds to 'sss', the RHS to 'ss'.
The bug occurs when the RHS tries to match the first two es's, but that
splits the LHS \xDF character, which Perl doesn't know how to handle,
and the assertion got triggered.  (This is similar to [perl #72998].)

The solution adopted here is to disallow a partial character match,
as #72998 did as well.
---
 regexec.c           |   10 ++++++----
 t/re/pat_advanced.t |    4 ++++
 2 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/regexec.c b/regexec.c
index 391fc16..fd90ad7 100644
--- a/regexec.c
+++ b/regexec.c
@@ -6726,9 +6726,12 @@ S_reginclass(pTHX_ const regexp * const prog, register const regnode * const n,
 			    STRLEN len;
 			    const char * const s = SvPV_const(sv, len);
 
-			    if (len <= total_foldlen && memEQ(s,
-							       (char*)folded,
-							       len))
+			    if (len <= total_foldlen
+				&& memEQ(s, (char*)folded, len)
+
+				   /* If 0, means matched a partial char. See
+				    * [perl #90536] */
+				&& map_fold_len_back[len])
 			    {
 
 				/* Advance the target string ptr to account for
@@ -6737,7 +6740,6 @@ S_reginclass(pTHX_ const regexp * const prog, register const regnode * const n,
 				 * length. */
 				if (lenp) {
 				    *lenp = map_fold_len_back[len];
-				    assert(*lenp != 0);	/* Otherwise will loop */
 				}
 				match = TRUE;
 				break;
diff --git a/t/re/pat_advanced.t b/t/re/pat_advanced.t
index 6d7624d..2a510d1 100644
--- a/t/re/pat_advanced.t
+++ b/t/re/pat_advanced.t
@@ -2108,6 +2108,10 @@ EOP
         like("\x{00DF}", qr/[\x{1E9E}_]*/i, "\"\\x{00DF}\" =~ /[\\x{1E9E}_]*/i was looping");
     }
 
+    {   # Bug #90536, caused failed assertion
+        unlike("s\N{U+DF}", qr/^\x{00DF}/i, "\"s\\N{U+DF}\", qr/^\\x{00DF}/i");
+    }
+
     # !!! NOTE that tests that aren't at all likely to crash perl should go
     # a ways above, above these last ones.
 
-- 
1.7.1

[p5pRT]

From @cpansprout

I see this has been applied as f912626.

[p5pRT]

@cpansprout - Status changed from 'new' to 'resolved'