Odd problem with anonymous subroutines, lexical variable closures, and DESTROY (fwd)

[p5pRT]

Migrated from rt.perl.org#1028 (status was 'resolved')

Searchable as RT1028$

[p5pRT]

From kp11901@mail.cedarnet.org

Created by kp11901@cedarnet.org

OK, first things first (what I was trying to do). I have developed a perl
module that implements

try {
  # might die
} catch {
  #if it did
};

syntax for exception handling, using sub (&@​) try and sub (&) catch;

It seems to work nicely, except for one thing - if there is a try within a
try, variable scoping can become very confused. I tracked this down to a
problem in lexical closure (I think, anyhow), and have developed a test
case that exhibits the problem without needing any of the complex Tie​::DBI
routines (which is how I found the problem, the program was never
closing it's SQL queries and was overloading the database server).

The test case will loop creating new objects, and undef'ing them, every
time you hit return until you send EOF (with cntrl-d on most unices). Then
and only then will the objects get garbage collected (and their DESTROY
methods called). In a (possibly related) problem, running this test case
in perl -d causes it to always fail to garbage collect properly, even with
the commented-out line that purports to work around whatever is happening
uncommented (which normally fixes it). In perldebug, it also fails with
only one level of eval { sub { }->() };, whereas that normally works and
two fails. I suspect (though I have not verified) that perl's debug mode
has wrapped my whole function in a similar layer (so that it can trap
errors), thus exposing the problem one layer sooner than normal.

Running it with eval { eval { ... } } will work, it is only with the
subroutines that it fails - however, the subroutines are important to the
try { } catch { } syntax working properly and allowing me to centralize
how exceptions are passed up (and through RPC) as the statements to try
and catch with need to be passed in.

if any more detail would be helpful, just say so. I've tried not to leave
anything important out, but... I've only done perl for a few months, so my
understanding of the innards is limited.

Oh - this is a straight RedHat 6.0/i386 system, with their perl build
(5.00503, from their RPM). I haven't tried recompiling perl (on my poor
Pentium 75, that would take a while), but I can do some experimenting if
you need me to (ie you can't reproduce the problem with this testcase).

Here is the test case (about 70 lines)​:

$| = 1; # I want any order problems to be real!
while(<STDIN>) {
  $a = container->new();
  #undef $a->{b};
  # this would successfully clean up $a->{b}, but not $a still hangs
  # around, unreachable.
  undef $a;
  # the DESTROY method on $a is not getting called.
}
# you can accumulate these objects for as long as you want, leaking
# memory (just hold down return). When you finally exit (send EOF, cntrl-d
# on most unices) they will all be gotten at last during the special
# cleanup pass at the end. However, these structures are not
# self-referential in any way I can think of.
# kill(9,$$);
# this is to ensure that the subelements die when their object is undef'ed
# they do in fact go away (finally) during the exit() garbage collection.
print "exiting...\n";
exit;

package container;
sub new {
  my $self = bless {},shift;
# declaring $self with local also fixes the problem, but unsatisfactorily,
# since it really should be lexical.
  print "new $self\n";
  eval { sub {
  my $self = $self;
  # re-scoping the lexical here does not help
  eval { sub {
# my $self = shift;
# uncommenting the previous line resolves the problem, which leads me to
# believe it's a lexical binding (closure) problem. The correct value of
# $self does reach this inner subroutine, but somehow garbage collection
# is screwed up by the double-closure.
  print "new \$a for $self\n";
  $self->{b} = dummy->new();
  print "got \$b​: $self->{b}\n";
  }->($self) };
  }->() };
  print "have \$b in $self​: $self->{b}\n";
  return $self;
}
# this situation of eval{ sub {}->() } comes up in my implementation of a
# try { } catch { }; block using sub try (&@​) { ... } kinds of notation.
# this notation (see perlsub) gives me a reference to an anonymous
# subroutine formed of the code block following try. If the subrouting
# reference in the inner eval is named, it works normally - but that is to
# be expected, as it then recieves the $self reference via the object
# method call, not via it's enclosing lexical scope (which should be
# valid, shouldn't it?).

sub DESTROY {
  print "cleaning up " . shift() . "\n";
}

package dummy;
# this package just traces its lifespan
sub new {
  my $self = bless {},shift;
  print "new $self\n";
  return $self;
}
sub DESTROY {
  print "cleaning up " . shift() . "\n";
}

Perl Info

Site configuration information for perl 5.00503:

Configured by root at Tue Apr  6 23:33:05 EDT 1999.

Summary of my perl5 (5.0 patchlevel 5 subversion 3) configuration:
  Platform:
    osname=linux, osvers=2.2.1-ac1, archname=i386-linux
    uname='linux porky.devel.redhat.com 2.2.1-ac1 #1 smp mon feb 1 17:44:44 est 1999 i686 unknown '
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef useperlio=undef d_sfio=undef
  Compiler:
    cc='cc', optimize='-O2', gccversion=egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)
    cppflags='-Dbool=char -DHAS_BOOL -I/usr/local/include'
    ccflags ='-Dbool=char -DHAS_BOOL -I/usr/local/include'
    stdchar='char', d_stdstdio=undef, usevfork=false
    intsize=4, longsize=4, ptrsize=4, doublesize=8
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    alignbytes=4, usemymalloc=n, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lndbm -lgdbm -ldb -ldl -lm -lc -lposix -lcrypt
    libc=, so=so, useshrplib=false, libperl=libperl.a
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl 5.00503:
    /usr/lib/perl5/5.00503/i386-linux
    /usr/lib/perl5/5.00503
    /usr/lib/perl5/site_perl/5.005/i386-linux
    /usr/lib/perl5/site_perl/5.005
    .


Environment for perl 5.00503:
    HOME=/root
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/X11R6/bin:/root/bin:/usr/X11R6/bin:/root/bin:/usr/X11R6/bin:/root/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[p5pRT]

From @iabyn

(reviewing old bugs).

Problem appears to persist in 5.8.0

The following simplified code example exhibits the same problem.

I'm guessing that when the outer anon sub is cloned, it picks up
a reference
from the inner sub which is cloned at the same time, so that the outer
clone isn't immediately freed, which means its ref to $self persists.

  foreach (1..3) {
  my $a = Foo->new();
  undef $a;
  }
  print "exiting\n";
  exit;

  package Foo;
  sub new {
  my $self = bless {},shift;
  print "new $self\n";
  sub {
  $self;
  my $foo;
  # commenting out the following line causes the
  # destructor to be called each time round the loop
  sub { $foo; };
  };
  return $self;
  }

  sub DESTROY { print "destroy " . shift() . "\n"; }

[p5pRT]

From @iabyn

This bug has now been fixed in bleedperl by patch #18302,
so I'm closing the call.

Regards,

Dave M.

  Subject​: Proper fix for CvOUTSIDE weak refcounting
  From​: Dave Mitchell <davem@​fdgroup.com>
  Date​: Tue, 10 Dec 2002 01​:26​:44 +0000
  Message-ID​: <20021210012644.A7843@​fdgroup.com>

[p5pRT]

@iabyn - Status changed from 'open' to 'resolved'