Closures hang on to free variables of all their parents, causing subtle memory leaks

[scabug]

The following code will throw an OutOfMemoryException (with heap size below 1GB):

class Engine {
  
  var handlers: List[(Int => Unit)] = Nil
  
  def registerHandler(f: Int => Unit) = handlers ::= f
  
  def setup() {
    
    val buffer = new Array[Byte](1000000)
    
    for (s <- List("foo")) {
      buffer(0) = 1

      registerHandler { n =>
        callback(n)
      }
    }
  }
  
  def callback(n: Int) = println(n)
}
  
object Test extends Engine {  
  def main(args: Array[String]) {
    
    println("start")
    
    for (i <- 0 until 1000) {
      println("setup handler " + i)
      setup()
    }
    
  } 
}

The OOM occurs because the buffer objects cannot be garbage collected. This is unexpected because only closures passed to registerHandler are retained, which have no free variables except (implicitly) the enclosing Engine instance. Certainly, they don't refer to any buffer.

However, closures are implemented as anonymous inner classes, and inner classes refer to their enclosing classes by chasing $outer pointers. Thus, the closures passed to registerHandler do not refer directly to the Engine instance but indirectly through the statically enclosing closure (the one passed to List("foo").map, which does refer to the buffer).

This behavior is particularly troublesome when using continuations, as the nesting of closures is no longer apparent from the program text:

val buffer = new Array[Byte](1000000)
sleep(100) // cps version
buffer(0) = 1
registerHandler { n => ... } // closure will hang on to buffer

A possible fix could be to treat this-references as proper free variables. More generally, anonymous classes could use separate outer_k pointers for the k-th enclosing object (called a 'display' in the literature). Of course only the used ones should be kept, so #1419 may be related.

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-5367?orig=1
Reporter: @TiarkRompf
See #2617, #1419

[scabug]

@sbocq said:
The leak may affect for-comprehensions as well and cause serialization overhead. The following snippet is extracted from a program where this is causing an issue.

  case class Id[A](a:A) {
    def map[B](f:A => B):Id[B] = Id(f(a))
    def flatMap[B](f: A => Id[B]):Id[B] = f(a) 
  }
    
  def foo(x:Id[Int]):Id[Unit] = for {
    u <- Id(4)
    // v <- Id(5)
    w <- x.map(_ + u)
  } yield w

  foo(Id(1))

The closure passed to map captures only 'u' if "v <- Id(5)" is commented out.

-> f	      Test$$anonfun$foo$1$$anonfun$apply$1  (id=21)	
 -> u$1	      4	

If the line is uncommented, although it is completely unrelated, the closure captures x.

-> f          Test$$anonfun$foo$1$$anonfun$apply$3$$anonfun$apply$1  (id=22)	
 -> $outer    Test$$anonfun$foo$1$$anonfun$apply$3  (id=29)	
  -> $outer   Test$$anonfun$foo$1  (id=34)	
   -> x$2     Test$Id<A>  (id=21)	
   -> u$1     4	

[scabug]

Turing Eret (turingeret) said:
The behavior of unnecessarily hanging on to variables by closures is related to inner classes unnecessarily hanging on to variables, due to how closures are implemented as inner classes.

[scabug]

@adriaanm said:
Sorry, the continuations plugin has been unsupported for a while now. Please open an issue over at https://github.com/scala/scala-continuations (we're still looking for a maintainer for it, though).