Data::Dumper aborts with out-of-mem on cyclic threads::shared data

[p5pRT]

Migrated from rt.perl.org#120416 (status was 'open')

Searchable as RT120416$

[p5pRT]

From @jimav

This is a bug report for perl from james_avera@​yahoo.com,
generated with the help of perlbug 1.39 running under perl 5.14.2.

---

If a hash containing a direct or indirect self-reference is cloned with
threads​::shared​::shared_clone(), then if Data​::Dumper is called on the
resulting shared copy it allocates infinite memory and dies with sig 9.

Just guessing here, but maybe the "unreliability" of refaddr() in shared
data structures is the problem. See threads​::shared docs.

#!/usr/bin/perl
use strict; use warnings;
use threads; use threads​::shared;
use Data​::Dumper;

if (`bash -c "ulimit -v"` =~ /unlimited/) {
  # limit VM size to prevent locking up the system
  exec "bash", "-c", "set -x; ulimit -v 100000; exec $^X $0 @​ARGV"; die;
}

my $data = { foo => "bar" };
$data->{selfref} = \$data;

my $shared = shared_clone($data);
warn "shared->selfref = ", $shared->{selfref}, "\n";
warn "Calling Dumper on original \$data"; Dumper($data); warn "OK\n";
warn "Calling Dumper on dequeued \$shared"; Dumper($shared); warn "OK\n";
# this aborts with "out of memory"

---

---

Flags​:
  category=library
  severity=low
  module=Data​::Dumper

---

Site configuration information for perl 5.14.2​:

Configured by Debian Project at Thu Jul 18 22​:04​:35 UTC 2013.

Summary of my perl5 (revision 5 version 14 subversion 2) configuration​:
 
  Platform​:
  osname=linux, osvers=3.2.0-37-generic, archname=x86_64-linux-gnu-thread-multi
  uname='linux roseapple 3.2.0-37-generic #58-ubuntu smp thu jan 24 15​:28​:10 utc 2013 x86_64 x86_64 x86_64 gnulinux '
  config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Dldflags= -Wl,-Bsymbolic-functions -Wl,-z,relro -Dlddlflags=-shared -Wl,-Bsymbolic-functions -Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.14 -Darchlib=/usr/lib/perl/5.14 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.14.2 -Dsitearch=/usr/local/lib/perl/5.14.2 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.14.2 -des'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fstack-protector -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O2 -g',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fstack-protector -fno-strict-aliasing -pipe -I/usr/local/include'
  ccversion='', gccversion='4.8.1', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
  libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
  libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
  perllibs=-ldl -lm -lpthread -lc -lcrypt
  libc=, so=so, useshrplib=true, libperl=libperl.so.5.14.2
  gnulibc_version='2.17'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib -fstack-protector'

Locally applied patches​:
 

---

@​INC for perl 5.14.2​:
  /home/jima/lib/perl
  /home/jima/perl5/lib/perl5/x86_64-linux-gnu-thread-multi
  /home/jima/perl5/lib/perl5/x86_64-linux-gnu-thread-multi
  /home/jima/perl5/lib/perl5
  /etc/perl
  /usr/local/lib/perl/5.14.2
  /usr/local/share/perl/5.14.2
  /usr/lib/perl5
  /usr/share/perl5
  /usr/lib/perl/5.14
  /usr/share/perl/5.14
  /usr/local/lib/site_perl
  .

---

Environment for perl 5.14.2​:
  HOME=/home/jima
  LANG=en_US.UTF-8
  LANGUAGE=en_US
  LD_LIBRARY_PATH=/home/jima/local/lib
  LOGDIR (unset)
  PATH=/home/jima/perl5/bin​:/home/jima/bin​:/home/jima/local/bin​:/home/jima/jima_tools/x86_64/bin​:/home/jima/jima_tools/bin​:/usr/bin​:/bin​:/usr/sbin​:/sbin​:/usr/bin/X11​:/usr/local/bin​:/usr/lib/lightdm/lightdm​:/usr/local/sbin​:/usr/games​:/usr/local/games​:/usr/lib/jvm/java-7-oracle/bin​:/usr/lib/jvm/java-7-oracle/db/bin​:/usr/lib/jvm/java-7-oracle/jre/bin​:.
  PERL5LIB=/home/jima/lib/perl​:/home/jima/perl5/lib/perl5/x86_64-linux-gnu-thread-multi​:/home/jima/perl5/lib/perl5
  PERL_BADLANG (unset)
  PERL_LOCAL_LIB_ROOT=/home/jima/perl5
  PERL_MB_OPT=--install_base /home/jima/perl5
  PERL_MM_OPT=INSTALL_BASE=/home/jima/perl5
  SHELL=/bin/bash

[p5pRT]

From @jkeenan

On Wed, 30 Oct 2013 20​:21​:34 GMT, jimav wrote​:

This is a bug report for perl from james_avera@​yahoo.com,
generated with the help of perlbug 1.39 running under perl 5.14.2.

-----------------------------------------------------------------
If a hash containing a direct or indirect self-reference is cloned
with
threads​::shared​::shared_clone(), then if Data​::Dumper is called on the
resulting shared copy it allocates infinite memory and dies with sig
9.

Just guessing here, but maybe the "unreliability" of refaddr() in
shared
data structures is the problem. See threads​::shared docs.

#!/usr/bin/perl
use strict; use warnings;
use threads; use threads​::shared;
use Data​::Dumper;

if (`bash -c "ulimit -v"` =~ /unlimited/) {
# limit VM size to prevent locking up the system
exec "bash", "-c", "set -x; ulimit -v 100000; exec $^X $0 @​ARGV";
die;
}

my $data = { foo => "bar" };
$data->{selfref} = \$data;

my $shared = shared_clone($data);
warn "shared->selfref = ", $shared->{selfref}, "\n";
warn "Calling Dumper on original \$data"; Dumper($data); warn "OK\n";
warn "Calling Dumper on dequeued \$shared"; Dumper($shared); warn
"OK\n";
# this aborts with "out of memory"

Building a threaded perl on a very recent commit to blead (see attachment), I got slightly different output​:

#####
$ ./bin/perl -Ilib ~/learn/perl/p5p/120416-threads-shared.pl
+ ulimit -v 100000
+ exec /home/jkeenan/testing/blead/bin/perl /home/jkeenan/learn/perl/p5p/120416-threads-shared.pl
shared->selfref = REF(0x1a56110)
Calling Dumper on original $data at /home/jkeenan/learn/perl/p5p/120416-threads-shared.pl line 15.
OK
Calling Dumper on dequeued $shared at /home/jkeenan/learn/perl/p5p/120416-threads-shared.pl line 16.
Recursion limit of 1000 exceeded at /home/jkeenan/testing/blead/lib/perl5/5.25.9/x86_64-linux-thread-multi/Data/Dumper.pm line 231.
#####
--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

From @jkeenan

Summary of my perl5 (revision 5 version 25 subversion 9) configuration​:
  Commit id​: 8df0224
  Platform​:
  osname=linux
  osvers=4.4.0-57-generic
  archname=x86_64-linux-thread-multi
  uname='linux zareason 4.4.0-57-generic #78-ubuntu smp fri dec 9 23​:50​:32 utc 2016 x86_64 x86_64 x86_64 gnulinux '
  config_args='-des -Dusedevel -Dusethreads -Uversiononly -Dprefix=/home/jkeenan/testing/blead -Dman1dir=none -Dman3dir=none'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
  optimize='-O2'
  cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='5.4.0 20160609'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /lib64 /usr/lib64
  libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.23.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.23'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  HAS_TIMES
  MULTIPLICITY
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_IMPLICIT_CONTEXT
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  PERL_USE_DEVEL
  USE_64_BIT_ALL
  USE_64_BIT_INT
  USE_ITHREADS
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  USE_REENTRANT_API
  Built under linux
  Compiled at Dec 30 2016 21​:38​:50
  %ENV​:
  PERLBREW_BASHRC_VERSION="0.67"
  PERLBREW_HOME="/home/jkeenan/.perlbrew"
  PERLBREW_MANPATH="/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/man"
  PERLBREW_PATH="/home/jkeenan/perl5/perlbrew/bin​:/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/bin"
  PERLBREW_PERL="perl-5.24.0"
  PERLBREW_ROOT="/home/jkeenan/perl5/perlbrew"
  PERLBREW_VERSION="0.67"
  PERL_WORKDIR="/home/jkeenan/gitwork/perl"
  @​INC​:
  lib
  /home/jkeenan/testing/blead/lib/perl5/site_perl/5.25.9/x86_64-linux-thread-multi
  /home/jkeenan/testing/blead/lib/perl5/site_perl/5.25.9
  /home/jkeenan/testing/blead/lib/perl5/5.25.9/x86_64-linux-thread-multi
  /home/jkeenan/testing/blead/lib/perl5/5.25.9
  .

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'