Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

panic: sv_len_utf8 cache with -d and -DDEBUGGING #13604

Closed
p5pRT opened this issue Feb 15, 2014 · 16 comments
Closed

panic: sv_len_utf8 cache with -d and -DDEBUGGING #13604

p5pRT opened this issue Feb 15, 2014 · 16 comments

Comments

@p5pRT
Copy link

p5pRT commented Feb 15, 2014

Migrated from rt.perl.org#121255 (status was 'resolved')

Searchable as RT121255$

@p5pRT
Copy link
Author

p5pRT commented Feb 15, 2014

From @kentfredric

This is a bug report for perl from kentnl@​cpan.org,
generated with the help of perlbug 1.40 running under perl 5.19.8.


This problem is hard for me to replicate as a reduced test case, because it seems linked to the stack depth of the problem.

I have 2 scripts that trigger it, but they both require CPAN modules installed, and both require running under -d and -DDEBUGGING to trigger the failure.

The crux of it is this snippet in perl5debug.pl causes the error for some reason, as it for some reason does not like calling length($sub) or substr( $sub, )

  # If the last ten characters are '​::AUTOLOAD', note we've traced
  # into AUTOLOAD for $sub.
  if ( length($sub) > 10 and substr( $sub, -10, 10 ) eq '​::AUTOLOAD' ) {
  no strict 'refs';
  $al = " for $$sub" if defined $$sub;
  }

Simply wrapping both these calls as :

length("$sub") and substr( "$sub", )

Suppresses the problem entirely.

Augmenting debug.pl like such prior to the length call is interesting​:

  if ("$sub" =~ /^Git​::Wrapper​::Plus​::Branches​::/) {
  print Devel​::Peek​::Dump($DB​::sub);
  }

Because oddly, it yields

SV = PV(0x2325470) at 0x1c7a170
  REFCNT = 1
  FLAGS = (POK,pPOK,UTF8)
  PV = 0x21d8570 "Git​::Wrapper​::Plus​::Branches​::current_branch"\0 [UTF8 "Git​::Wrapper​::Plus​::Branches​::current_branch"]
  CUR = 44
  LEN = 64
SV = PVMG(0x2b2ad80) at 0x1c7a170
  REFCNT = 1
  FLAGS = (SMG,POK,pPOK,UTF8)
  IV = 0
  NV = 0
  PV = 0x239a610 "Git​::Wrapper​::Plus​::Branches​::_current_branch_name"\0 [UTF8 "Git​::Wrapper​::Plus​::Branches​::_current_branch_name"]
  CUR = 50
  LEN = 72
  MAGIC = 0x23a5dd0
  MG_VIRTUAL = &PL_vtbl_utf8
  MG_TYPE = PERL_MAGIC_utf8(w)
  MG_LEN = 44
panic​: sv_len_utf8 cache 44 real 50 for Git​::Wrapper​::Plus​::Branches​::_current_branch_name at /home/kent/perl5/perlbrew/perls/perl-5.19.7/lib/5.19.7/perl5db.pl line 4113.
at /home/kent/perl5/perlbrew/perls/perl-5.19.7/lib/5.19.7/perl5db.pl line 4113.
  Git​::Wrapper​::Plus​::Branches​::current_branch(Git​::Wrapper​::Plus​::Branches=HASH(0x2398fc8)) called at /tmp/fail.pl line 7

Take note​: current_branch has no magic, _current_branch_name has magic.

And when invoking _current_branch_name directly, it does this​:

SV = PV(0x170a3c0) at 0x105f170
  REFCNT = 1
  FLAGS = (POK,pPOK,UTF8)
  PV = 0x1600320 "Git​::Wrapper​::Plus​::Branches​::_current_branch_name"\0 [UTF8 "Git​::Wrapper​::Plus​::Branches​::_current_branch_name"]
  CUR = 50
  LEN = 72
SV = PVMG(0x1f0fab0) at 0x105f170
  REFCNT = 1
  FLAGS = (SMG,POK,pPOK,UTF8)
  IV = 0
  NV = 0
  PV = 0x1600320 "Git​::Wrapper​::Plus​::Branches​::git"\0 [UTF8 "Git​::Wrapper​::Plus​::Branches​::git"]
  CUR = 33
  LEN = 48
  MAGIC = 0x1f502f0
  MG_VIRTUAL = &PL_vtbl_utf8
  MG_TYPE = PERL_MAGIC_utf8(w)
  MG_LEN = 50
panic​: sv_len_utf8 cache 50 real 33 for Git​::Wrapper​::Plus​::Branches​::git at /home/kent/perl5/perlbrew/perls/perl-5.19.7/lib/5.19.7/perl5db.pl line 4113.
at /home/kent/perl5/perlbrew/perls/perl-5.19.7/lib/site_perl/5.19.7/Git/Wrapper/Plus/Util.pm line 64.
  Git​::Wrapper​::Plus​::Util​::catch {...} ("panic​: sv_len_utf8 cache 50 real 33 for Git​::Wrapper​::Plus​::B"...) called at /home/kent/perl5/perlbrew/perls/perl-5.19.7/lib/site_perl/5.19.7/Try/Tiny.pm line 104
  Try​::Tiny​::try(CODE(0x177df38), Try​::Tiny​::Catch=REF(0x1f2ca78)) called at /home/kent/perl5/perlbrew/perls/perl-5.19.7/lib/site_perl/5.19.7/Git/Wrapper/Plus/Util.pm line 79
  Git​::Wrapper​::Plus​::Util​::exit_status_handler(CODE(0x15b4758), HASH(0x170d240)) called at /home/kent/perl5/perlbrew/perls/perl-5.19.7/lib/site_perl/5.19.7/Git/Wrapper/Plus/Branches.pm line 137
  Git​::Wrapper​::Plus​::Branches​::_current_branch_name(Git​::Wrapper​::Plus​::Branches=HASH(0x177ded8)) called at /tmp/fail.pl line 7

Noting that the previous dump showed _current_branch_name with magic, and this dump not, and instead, the magic is moved to ->git

The other way to trigger it is​:

perl -d -E' use sanity;' , which causes the failure somewhere in Import​::Into , but that loads way too many XS Magic things for me to try digging there.

( I ruled out it being a common failure point with autoviv and friends by slicing Moo and Moo​::_Utils to not import strictures and checked the strictures modules were not being loaded with -d​:TraceUse )



Flags​:
  category=core
  severity=medium


Site configuration information for perl 5.19.8​:

Configured by kent at Wed Jan 29 21​:15​:13 NZDT 2014.

Summary of my perl5 (revision 5 version 19 subversion 8) configuration​:
 
  Platform​:
  osname=linux, osvers=3.13.0-gentoo-r1, archname=x86_64-linux
  uname='linux katipo2 3.13.0-gentoo-r1 #58 smp tue jan 28 20​:17​:45 nzdt 2014 x86_64 intel(r) core(tm) i5-2410m cpu @​ 2.30ghz genuineintel gnulinux '
  config_args='-de -Dprefix=/home/kent/perl5/perlbrew/perls/perl-5.19.8 -DDEBUGGING -Dusedevel -Aeval​:scriptdir=/home/kent/perl5/perlbrew/perls/perl-5.19.8/bin'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O2 -g',
  cppflags='-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector'
  ccversion='', gccversion='4.7.3', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector'
  libpth=/usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/include-fixed /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64
  libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
  perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
  libc=/lib/libc-2.17.so, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version='2.17'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -g -fstack-protector'


@​INC for perl 5.19.8​:
  /home/kent/perl5/perlbrew/perls/perl-5.19.8/lib/site_perl/5.19.8/x86_64-linux
  /home/kent/perl5/perlbrew/perls/perl-5.19.8/lib/site_perl/5.19.8
  /home/kent/perl5/perlbrew/perls/perl-5.19.8/lib/5.19.8/x86_64-linux
  /home/kent/perl5/perlbrew/perls/perl-5.19.8/lib/5.19.8
  .


Environment for perl 5.19.8​:
  HOME=/home/kent
  LANG (unset)
  LANGUAGE (unset)
  LC_CTYPE=en_NZ.UTF8
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/home/kent/perl5/perlbrew/bin​:/home/kent/perl5/perlbrew/perls/perl-5.19.8/bin​:/home/kent/.perl6/2013.04/bin​:/home/kent/.gem/ruby/1.8/bin/​:/usr/local/bin​:/usr/bin​:/bin​:/opt/bin​:/usr/x86_64-pc-linux-gnu/gcc-bin/4.7.3​:/usr/games/bin
  PERLBREW_BASHRC_VERSION=0.67
  PERLBREW_HOME=/home/kent/.perlbrew
  PERLBREW_MANPATH=/home/kent/perl5/perlbrew/perls/perl-5.19.8/man
  PERLBREW_PATH=/home/kent/perl5/perlbrew/bin​:/home/kent/perl5/perlbrew/perls/perl-5.19.8/bin
  PERLBREW_PERL=perl-5.19.8
  PERLBREW_ROOT=/home/kent/perl5/perlbrew
  PERLBREW_VERSION=0.67
  PERL_BADLANG (unset)
  SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Feb 15, 2014

From @kentfredric

fail.pl

@p5pRT
Copy link
Author

p5pRT commented Feb 15, 2014

From perl5-porters@perl.org

Kent Fredric wrote​:

This problem is hard for me to replicate as a reduced test case, because it see
ms linked to the stack depth of the problem.

I have 2 scripts that trigger it, but they both require CPAN modules installed,
and both require running under -d and -DDEBUGGING to trigger the failure.

Here is a smaller case​:

$ perl5.19.9 -Mblib -de 'use utf8; require Git​::Wrapper​::Plus​::Branches;Git​::Wrapper​::Plus​::Branches​::current_branch()'

And blib/lib/Git/Wrapper/Plus/Branches.pm has been gutted and reduced to​:

package Git​::Wrapper​::Plus​::Branches;
sub _current_branch_name {
}
sub current_branch {
  _current_branch_name;
}
1;

The 'use utf8' and 'require' are necessary, so it seems to have some-
thing to do with the UTF8 flag on the stash (guessing).

@p5pRT
Copy link
Author

p5pRT commented Feb 15, 2014

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Feb 15, 2014

From @kentfredric

My other institutions were that it might be related to being called with a utf8 name but not being defined with a utf8 name, or vice versa, maybe causing problem.

Or perhaps the problem flares up due to sharing a compile context somehow.

@p5pRT
Copy link
Author

p5pRT commented Feb 15, 2014

From @kentfredric

If you rig @​INC right, you can get this happening in a single file.

Though its strange, seems entirely dependent on the `use utf8` value of the top context, and the value of utf8 in the compiled context is ignored. :/

@p5pRT
Copy link
Author

p5pRT commented Feb 15, 2014

From @kentfredric

fail.pl

@p5pRT
Copy link
Author

p5pRT commented Feb 15, 2014

From perl5-porters@perl.org

Kent Fredric wrote​:

If you rig @​INC right,

%INC is enough, as I have found out.

you can get this happening in a single file.

Though its strange, seems entirely dependent on the `use utf8` value of the top
context, and the value of utf8 in the compiled context is ignored. :/

require Bareword causes the Bareword package to be vivified. But for
some reason 'require' propagates the utf8ness of the source, whereas
'package' does not​:

$ ./perl -Ilib -e 'use utf8; ++$INC{"Branches.pm"};require Branches; use Devel​::Peek; Dump ref bless [], Branches'
SV = PV(0x7fa82a806070) at 0x7fa82a82af90
  REFCNT = 1
  FLAGS = (PADTMP,POK,pPOK,UTF8)
  PV = 0x7fa82a409cf0 "Branches"\0 [UTF8 "Branches"]
  CUR = 8
  LEN = 32
$ ./perl -Ilib -e 'use utf8; {package Branches} use Devel​::Peek; Dump ref bless [], Branches'
SV = PV(0x7fbaf9806070) at 0x7fbaf982af00
  REFCNT = 1
  FLAGS = (PADTMP,POK,IsCOW,pPOK)
  PV = 0x7fbaf940c820 "Branches"
  CUR = 8
  LEN = 0

Look, a 'one'-liner!

$ PERL5DB='${^UTF8CACHE}=-1; sub DB​::DB{} sub DB​::sub{length($DB​::sub); goto &$DB​::sub}' ./miniperl -Ilib -de 'use utf8; ++$INC{"Branches.pm"};require Branches;Branches​::cb();package Branches;sub _cbn {}sub cb{ _cbn();}'
panic​: sv_len_utf8 cache 12 real 14 for Branches​::_cbn.

(BTW, ${^UTF8CACHE} is -1 by default under -DDEBUGGING, but you can
set it yourself, to trigger the same bug on non-debugging builds.)

Bisect points to me​:

commit a7999c0
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Fri Nov 18 09​:08​:32 2011 -0800

  Make sure $DB​::sub is callable

I suspect I just uncovered an existing utf8 cache bug, but I am
still looking.

@p5pRT
Copy link
Author

p5pRT commented Feb 15, 2014

From perl5-porters@perl.org

I wrote​:

I suspect I just uncovered an existing utf8 cache bug, but I am
still looking.

I think adding SvSETMAGIC to util.c​:Perl_get_db_sub will fix it.
Though I wonder why it did not occur before a7999c0. Maybe my
bisect was faulty.

@p5pRT
Copy link
Author

p5pRT commented Feb 15, 2014

From perl5-porters@perl.org

I wrote​:

require Bareword causes the Bareword package to be vivified. But for
some reason 'require' propagates the utf8ness of the source, whereas
'package' does not​:

Hence, the workaround (untested) is to do this at the top of your mod-
ule before the package declaration​:

BEGIN { delete $Git​::Wrapper​::Plus​::{"Branches​::"} }

@p5pRT
Copy link
Author

p5pRT commented Feb 16, 2014

From @ikegami

On Sat, Feb 15, 2014 at 1​:29 PM, Father Chrysostomos <sprout@​cpan.org>wrote​:

Kent Fredric wrote​:

This problem is hard for me to replicate as a reduced test case, because
it see
ms linked to the stack depth of the problem.

I have 2 scripts that trigger it, but they both require CPAN modules
installed,
and both require running under -d and -DDEBUGGING to trigger the
failure.

Here is a smaller case​:

$ perl5.19.9 -Mblib -de 'use utf8; require
Git​::Wrapper​::Plus​::Branches;Git​::Wrapper​::Plus​::Branches​::current_branch()'

And blib/lib/Git/Wrapper/Plus/Branches.pm has been gutted and reduced to​:

package Git​::Wrapper​::Plus​::Branches;
sub _current_branch_name {
}
sub current_branch {
_current_branch_name;
}
1;

The 'use utf8' and 'require' are necessary, so it seems to have some-
thing to do with the UTF8 flag on the stash (guessing).

Well, the length cache is only used for UTF8-1 strings, so that's not
surprising.

@p5pRT
Copy link
Author

p5pRT commented Feb 16, 2014

From @cpansprout

Fixed in 795eb8c.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Feb 16, 2014

@cpansprout - Status changed from 'open' to 'resolved'

@p5pRT p5pRT closed this as completed Feb 16, 2014
@p5pRT
Copy link
Author

p5pRT commented Feb 17, 2014

From @demerphq

On 16 February 2014 06​:00, Eric Brine <ikegami@​adaelis.com> wrote​:

On Sat, Feb 15, 2014 at 1​:29 PM, Father Chrysostomos <sprout@​cpan.org>
wrote​:

Kent Fredric wrote​:

This problem is hard for me to replicate as a reduced test case, because
it see
ms linked to the stack depth of the problem.

I have 2 scripts that trigger it, but they both require CPAN modules
installed,
and both require running under -d and -DDEBUGGING to trigger the
failure.

Here is a smaller case​:

$ perl5.19.9 -Mblib -de 'use utf8; require
Git​::Wrapper​::Plus​::Branches;Git​::Wrapper​::Plus​::Branches​::current_branch()'

And blib/lib/Git/Wrapper/Plus/Branches.pm has been gutted and reduced to​:

package Git​::Wrapper​::Plus​::Branches;
sub _current_branch_name {
}
sub current_branch {
_current_branch_name;
}
1;

The 'use utf8' and 'require' are necessary, so it seems to have some-
thing to do with the UTF8 flag on the stash (guessing).

Well, the length cache is only used for UTF8-1 strings, so that's not
surprising.

UTF8-1?

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Feb 17, 2014

From perl5-porters@perl.org

Yves Orton asked​:

UTF8-1?

Typo. UTF-8.

@p5pRT
Copy link
Author

p5pRT commented Feb 17, 2014

From @ikegami

On Mon, Feb 17, 2014 at 7​:00 AM, demerphq <demerphq@​gmail.com> wrote​:

On 16 February 2014 06​:00, Eric Brine <ikegami@​adaelis.com> wrote​:

Well, the length cache is only used for UTF8-1 strings, so that's not
surprising.

UTF8-1?

UTF8=1, as in the UTF8 flag set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant