Delay finalizers until after codegen is done

[Keno]

On the mailing lists @vtjnash noted that we have the potential for recursive codegen if the GC tries to run finalizers during codegen. @carnaval Can we delay running finalizers while in codegen?

[carnaval]

would probably be fine, put the run_finalizers() in gc.c under a global flag, then maybe do the call when you reenable them (and probably need to put it under a jl_in_gc = 1 to avoid running gc while doing the backlog)

[tkelman]

See a backtrace that might be relevant at #11937 (comment).

@vtjnash has a sneaky feeling that this may be behind at least part of #7942 / #11818

[ihnorton]

Semi-related: #5502

[tkelman]

Can someone propose a patch for this? I can easily check whether any change will make #7942 / #11818 any better.

[ihnorton]

I don't really understand the parenthetical of @carnaval's comment, but for the first part I think it could be as simple as a counter in to_function (there's actually a commented-out counter variable there already):

julia/src/codegen.cpp

Lines 730 to 793 in b824dc9

	// --- entry point ---
	//static int n_emit=0;
	static Function *emit_function(jl_lambda_info_t *lam);
	//static int n_compile=0;
	static Function *to_function(jl_lambda_info_t *li)
	{
	JL_SIGATOMIC_BEGIN();
	assert(!li->inInference);
	BasicBlock *old = nested_compile ? builder.GetInsertBlock() : NULL;
	DebugLoc olddl = builder.getCurrentDebugLocation();
	bool last_n_c = nested_compile;
	nested_compile = true;
	Function *f = NULL;
	JL_TRY {
	f = emit_function(li);
	//n_emit++;
	}
	JL_CATCH {
	li->functionObject = NULL;
	li->specFunctionObject = NULL;
	li->cFunctionList = NULL;
	nested_compile = last_n_c;
	if (old != NULL) {
	builder.SetInsertPoint(old);
	builder.SetCurrentDebugLocation(olddl);
	}
	JL_SIGATOMIC_END();
	jl_rethrow_with_add("error compiling %s", li->name->name);
	}
	assert(f != NULL);
	nested_compile = last_n_c;
	#ifdef JL_DEBUG_BUILD
	#ifdef LLVM35
	llvm::raw_fd_ostream out(1,false);
	#endif
	if (
	#ifdef LLVM35
	verifyFunction(*f,&out)
	#else
	verifyFunction(*f,PrintMessageAction)
	#endif
	) {
	f->dump();
	abort();
	}
	#endif
	FPM->run(*f);
	//n_compile++;
	// print out the function's LLVM code
	//jl_static_show(JL_STDERR, (jl_value_t*)li);
	//jl_printf(JL_STDERR, "%s:%d\n",
	// ((jl_sym_t*)li->file)->name, li->line);
	//f->dump();
	//if (verifyFunction(*f,PrintMessageAction)) {
	// f->dump();
	// abort();
	//}
	if (old != NULL) {
	builder.SetInsertPoint(old);
	builder.SetCurrentDebugLocation(olddl);
	}
	JL_SIGATOMIC_END();
	return f;
	}

Then pop the counter on the other side (be sure to reset it in the JL_TRY block too).

Then check if the counter is non-zero in run_finalizers:

julia/src/gc.c

Lines 149 to 159 in d2f065e

	static void run_finalizers(void)
	{
	void *o = NULL, *f = NULL;
	JL_GC_PUSH2(&o, &f);
	while (to_finalize.len > 0) {
	f = arraylist_pop(&to_finalize);
	o = arraylist_pop(&to_finalize);
	run_finalizer((jl_value_t*)o, (jl_value_t*)f);
	}
	JL_GC_POP();
	}

[tkelman]

Thanks, @ihnorton. What's more problematic here, emit or compile?