threads async segfaults (5.8.1)

[p5pRT]

Migrated from rt.perl.org#24165 (status was 'resolved')

Searchable as RT24165$

[p5pRT]

From mjp-perl@pilcrow.madison.wi.us

Created by perlbuild@box.securepipe.com

With async-spawned threads, combinations of $thr->tid() and $thr->join()
can cause segfaults​:

  # tid, <delay>, join
  #
  $ perl -Mthreads -le '$t = async {1;}; $t->tid; sleep 2; $_->join for threads->list;'
  Segmentation fault

  # tid, join, tid
  #
  $ perl -Mthreads -le '$t = async {1;}; $t->tid; $t->join; $t->tid;'
  Segmentation fault

  # tid, join, tid, join
  #
  $ perl -Mthreads -le '$t = async {1;}; $t->join; $t->tid; $t->join; $t->tid'
  Thread already joined at -e line 1.
  Segmentation fault

This does _not_ happen if the threads are _not_ created via async{}. That is,
"threads->create(sub {1;}) ..." in the above examples does not segfault.
FWIW, (abbreviated) ltrace puts the SEGV in perl_destruct​:

  ...
  perl_run
  SIGRTMIN
  SIGRTMIN
  "Thread already joined at -e line 1."
  <... perl_run resumed>
  perl_destruct <--- async{} examples SEGV here, new/create continue
  perl_free
  pthread_mutex_destroy
  exit

(perl-5.8.0-55 from Redhat's Rawhide 3.2-5 does not segfault with the
above async{} examples.)

Perl Info

Flags:
    category=library
    severity=medium

Site configuration information for perl v5.8.1:

Configured by mjp at Fri Sep 26 09:06:03 CDT 2003.

Summary of my perl5 (revision 5.0 version 8 subversion 1) configuration:
  Platform:
    osname=linux, osvers=2.4.20-19.7, archname=i686-linux-thread-multi
    uname='linux box.securepipe.com 2.4.20-19.7 #1 tue jul 15 13:44:14 edt 2003 i686 unknown '
    config_args='-dse'
    hint=previous, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm -D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm'
    ccversion='', gccversion='2.96 20000731 (Red Hat Linux 7.3 2.96-113)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lndbm -lgdbm -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.2.5.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.2.5'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/home/mjp/opt/lib/perl5/5.8.1/i686-linux-thread-multi/CORE'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.1:
    /home/mjp/opt/lib/perl5/5.8.1/i686-linux-thread-multi
    /home/mjp/opt/lib/perl5/5.8.1
    /home/mjp/opt/lib/perl5/site_perl/5.8.1/i686-linux-thread-multi
    /home/mjp/opt/lib/perl5/site_perl/5.8.1
    /home/mjp/opt/lib/perl5/site_perl
    .


Environment for perl v5.8.1:
    HOME=/home/mjp
    LANG=en_US.iso885915
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/mjp/opt/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/mjp/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[p5pRT]

From @lizmat

At 05​:04 +0000 10/9/03, Michael J.Pomraning (via RT) wrote​:

-----------------------------------------------------------------
[Please enter your report here]

With async-spawned threads, combinations of $thr->tid() and $thr->join()
can cause segfaults​:

# tid, <delay>, join
#
$ perl -Mthreads -le '$t = async {1;}; $t->tid; sleep 2; $_->join
for threads->list;'
Segmentation fault

Bombs on Mac OS X as well. A stack trace​:

#0 0x080a59f0 in Perl_pad_swipe (my_perl=0x81acba0, po=13, refadjust=1 '\001')
  at pad.c​:1002
#1 0x0809209a in Perl_op_clear (my_perl=0x81acba0, o=0x81f5ab8) at op.c​:378
#2 0x08091f2d in Perl_op_free (my_perl=0x81acba0, o=0x81f5ab8) at op.c​:324
#3 0x08091ecc in Perl_op_free (my_perl=0x81acba0, o=0x81f5b50) at op.c​:312
#4 0x08091ecc in Perl_op_free (my_perl=0x81acba0, o=0x81adee0) at op.c​:312
#5 0x0806192c in perl_destruct (my_perl=0x81acba0) at perl.c​:470
#6 0x400196a7 in Perl_ithread_destruct (my_perl=0x8192878, thread=0x81aaa80,
  why=0x4001bef6 "no reference") at threads.xs​:146
#7 0x40019a01 in ithread_mg_free (my_perl=0x8192878, sv=0x819339c,
  mg=0x8232fb8) at threads.xs​:217
#8 0x080e878c in Perl_sv_unmagic (my_perl=0x8192878, sv=0x819339c, type=110)
  at sv.c​:4808
#9 0x4001aa76 in Perl_ithread_DESTROY (my_perl=0x8192878, sv=0x81f4354)
  at threads.xs​:621
#10 0x4001b713 in XS_threads_DESTROY (my_perl=0x8192878, cv=0x81ac8d4)
  at threads.c​:797
#11 0x080dde5e in Perl_pp_entersub (my_perl=0x8192878) at pp_hot.c​:2817
#12 0x08065c79 in S_call_body (my_perl=0x8192878, myop=0xbfffedd0, is_eval=0)
  at perl.c​:2190
#13 0x08065808 in Perl_call_sv (my_perl=0x8192878, sv=0x81ac8d4, flags=150)
  at perl.c​:2111
#14 0x080e9477 in Perl_sv_clear (my_perl=0x8192878, sv=0x819339c) at sv.c​:5095
#15 0x080e9e0c in Perl_sv_free (my_perl=0x8192878, sv=0x819339c) at sv.c​:5339
#16 0x080df5c7 in do_clean_objs (my_perl=0x8192878, sv=0x81a7350) at sv.c​:385
#17 0x080df46c in S_visit (my_perl=0x8192878, f=0x80df4e9 <do_clean_objs>)
  at sv.c​:331
#18 0x080df767 in Perl_sv_clean_objs (my_perl=0x8192878) at sv.c​:424
#19 0x080619ad in perl_destruct (my_perl=0x8192878) at perl.c​:492
#20 0x0805fbdc in main (argc=4, argv=0xbffff064, env=0xbffff078)
  at perlmain.c​:88

# tid, join, tid
#
$ perl -Mthreads -le '$t = async {1;}; $t->tid; $t->join; $t->tid;'
Segmentation fault

Can't get this to bomb on Mac OS X. Same backtrace as above.

# tid, join, tid, join
#
$ perl -Mthreads -le '$t = async {1;}; $t->join; $t->tid; $t->join; $t->tid'
Thread already joined at -e line 1.
Segmentation fault

Also bombs with Mac OS X. Same tracktrace.

Oddly enough these segfault also with Valgrind. the resulting core
dump is Perl's though (still looking at what is going on there, if I
know more I'll report)

Liz

[p5pRT]

From @lizmat

At 05​:04 +0000 10/9/03, Michael J.Pomraning (via RT) wrote​:

This does _not_ happen if the threads are _not_ created via async{}. That is,
"threads->create(sub {1;}) ..." in the above examples does not segfault.
FWIW, (abbreviated) ltrace puts the SEGV in perl_destruct​:

Based on this observation, I changed "async" in threads.pm from​:

  sub async (&;@​) {
  my $cref = shift;
  return threads->new($cref,@​_);
  }

to​:

  sub async (&;@​) {threads->new( @​_ )} # also segfaults

  sub async (&;@​) {new( 'threads',@​_ )} # also segfaults

  sub async (&;@​) {unshift @​_,'threads'; goto &new} # doesn't segfault!

The reason behind this last change would be to make the environment
of async to be exactly like that of new/create (by removing the extra
entry for "asunc" from the call stack altogether). With this last
change, I can't get it to segfault anymore. And all the 5.8.1
threads tests pass.

This means that it would in principle be possible to issue a new
version of threads.pm to CPAN to fix this problem.

Liz

[p5pRT]

From @iabyn

On Fri, Oct 10, 2003 at 10​:46​:17PM +0200, Elizabeth Mattijsen wrote​:

At 23​:35 +0300 10/10/03, Enache Adrian wrote​:

On Sat, Oct 11, 2003 a.d., Abhijit Menon-Sen wrote​:

At 2003-10-10 16​:37​:55 +0200, liz@​dijkmat.nl wrote​:

This patch for now fixes the problem of segfaulting when using async()
in 5.8.1.

Thanks, applied. (#21436)
However, please don't close the ticket. The bug is still there and
should be fixed :-)
$ perl -Mthreads -e 'sub a{threads->new(shift)} $t = a sub{};
$t->tid; $t->join; $t->tid'
Segmentation fault

Indeed, which was implied by my "for now". It's a workaround for the
async problem.

Looking at the stack trace (which I posted yesterday to p5p), I think
the problem is pad related. And since there have been pad related
fixes between 5.8.0 and 5.8.1, I think we need to look for the final
fix of the problem there. But that is defenitely out of my league.

The problem is that when threads call perl_destruct(), PL_comppad and
PL_curpad point to the pad of the CV from where they were created, rather
than to the pad of PL_main_cv. When the last thread exits, PL_main_root
is freed, and coredumps occur as op_free() tries to do things with the pad
slots pointed to by various ops.

In this particular case, when the main program exits it calls
perl_destruct(), which decrements the refcount of PL_main_root; it then
frees $t, which invokes perl_destruct() on the (previously joined) child
thread, which then causes PL_main_root to be freed while PL_comppad
points to the wrong place.

The following patch, applied as #21470, fixes the immediate problem (*).

I must confess that I was suprised to find that perl_destruct isn't
called on behalf of a thread immediately after it is joined, but rather
only after the thread ref is freed.

Dave.

(*) Whoo hoo, my first commit :-)

--
Please note that ash-trays are provided for the use of smokers,
whereas the floor is provided for the use of all patrons.
  - Bill Royston

Change 21470 by davem@​davem-percy on 2003/10/16 20​:03​:44

  Ensure PL_comppad/curpad point to PL_main_cv's padlist when
  PL_main_root is freed; this may not have been be the case if a
  thread other than the main one is the last to be destroyed

Affected files ...

... //depot/perl/ext/threads/t/thread.t#10 edit
... //depot/perl/pad.h#9 edit
... //depot/perl/perl.c#526 edit

Differences ...

==== //depot/perl/ext/threads/t/thread.t#10 (text) ====

@​@​ -12,7 +12,7 @​@​

use ExtUtils​::testlib;
use strict;
-BEGIN { $| = 1; print "1..25\n" };
+BEGIN { $| = 1; print "1..26\n" };
use threads;
use threads​::shared;

@​@​ -153,5 +153,10 @​@​
  ok((keys %rand == 25), "Check that rand works after a new thread");
}

+# bugid #24165
+
+run_perl(prog =>
+ 'use threads; sub a{threads->new(shift)} $t = a sub{}; $t->tid; $t->join; $t->tid');
+is($?, 0, 'coredump in global destruction');

==== //depot/perl/pad.h#9 (text) ====

@​@​ -105,6 +105,9 @​@​
Set the current pad to be pad C<n> in the padlist, saving
the previous current pad.

+=for apidoc m|void|PAD_SET_CUR_NOSAVE |PADLIST padlist|I32 n
+like PAD_SET_CUR, but without the save
+
=for apidoc m|void|PAD_SAVE_SETNULLPAD
Save the current pad then set it to null.

@​@​ -133,8 +136,7 @​@​
  ? AvARRAY((AV*)(AvARRAY(padlist)[1]))[po] : Nullsv;
 

-#define PAD_SET_CUR(padlist,n) \
- SAVECOMPPAD(); \
+#define PAD_SET_CUR_NOSAVE(padlist,n) \
  PL_comppad = (PAD*) (AvARRAY(padlist)[n]); \
  PL_curpad = AvARRAY(PL_comppad); \
  DEBUG_Xv(PerlIO_printf(Perl_debug_log, \
@​@​ -142,6 +144,11 @​@​
  PTR2UV(PL_comppad), PTR2UV(PL_curpad), (int)(n)));

+#define PAD_SET_CUR(padlist,n) \
+ SAVECOMPPAD(); \
+ PAD_SET_CUR_NOSAVE(padlist,n);
+
+
#define PAD_SAVE_SETNULLPAD() SAVECOMPPAD(); \
  PL_comppad = Null(PAD*); PL_curpad = Null(SV**); \
  DEBUG_Xv(PerlIO_printf(Perl_debug_log, "Pad set_null\n"));

==== //depot/perl/perl.c#526 (text) ====

@​@​ -354,6 +354,10 @​@​

  /* Destroy the main CV and syntax tree */
  if (PL_main_root) {
+ /* ensure comppad/curpad to refer to main's pad */
+ if (CvPADLIST(PL_main_cv)) {
+ PAD_SET_CUR_NOSAVE(CvPADLIST(PL_main_cv), 1);
+ }
  op_free(PL_main_root);
  PL_main_root = Nullop;
  }

[p5pRT]

@iabyn - Status changed from 'new' to 'resolved'