cmd/compile: incorrect package initialization order for spec example

[chadwhitacre]

From https://golang.org/ref/spec#Package_initialization:

Initialization proceeds by repeatedly initializing the next package-level variable that is earliest in declaration order and ready for initialization, until there are no variables ready for initialization.

[…]

For example, given the declarations

var (
  a = c + b
  b = f()
  c = f()
  d = 3
)

func f() int {
  d++
  return d
}

the initialization order is d, b, c, a.

If Go initializes b before c, then after initialization I'd expect the value of b to be 4 and c to be 5. However, this test outputs b as 5 and c as 4. Swapping the b and c declarations doesn't change the output, but swapping the order in the addition in the declaration of a does change the output. Does this mean that the initialization order in the example is actually d, c, b, a? And that both LHS and RHS are in scope in the phrase "earliest in declaration order"? Or (more likely) am I missing something about what it means to declare and initialize a variable?

P.S. Location in current master:

go/doc/go_spec.html

Line 6341 in a9afa4e

the initialization order is <code>d</code>, <code>b</code>, <code>c</code>, <code>a</code>.

[griesemer]

Thanks for the issue. The spec example is correct but cmd/compile is wrong; and I'm pretty sure we have an issue for it, but I can't find it at the moment. go/types also agrees with the spec, and there is a test case for this exact example here).

The spec says:

...by repeatedly initializing the next package-level variable that is earliest in declaration order and ready for initialization...

At first, a is dependent on c and b, so it must wait. Both b and c depend on d so they also must wait. Once d is initialized, both c and b have no (uninitialized) dependency and thus can be initialized. They must be initialized in declaration order; since b is declared before c, b must be initialized first. Hence you get the order d, b, c, a.

By introducing a little helper function, we can have the code print out the initialization order explicitly: https://play.golang.org/p/yiajBYfoSG . As you can see, the order is clearly wrong.

The reason it is wrong for cmd/compile is that the compiler doesn't sort the "ready to initialize" variables in declaration order for some reason.

[chadwhitacre]

for some reason

Hmm ... something related to initfix? 🤔

go/src/cmd/compile/internal/gc/sinit.go

Lines 243 to 254 in 2c1d2e0

	// initfix computes initialization order for a list l of top-level
	// declarations and outputs the corresponding list of statements
	// to include in the init() function body.
	func initfix(l []*Node) []*Node {
	var lout []*Node
	initplans = make(map[*Node]*InitPlan)
	lno := lineno
	initreorder(l, &lout)
	lineno = lno
	initplans = nil
	return lout
	}

[griesemer]

Yes, somewhere in that file. Haven't investigated.

[chadwhitacre]

@griesemer Cool, no worries. Verbose dive log on chadwhitacre#1, will report back here if I find anything. :)

[chadwhitacre]

@griesemer I've got the test running. It passes, which seems odd. My work plan:

• Modify TestInitOrderInfo so that it fails for the case in question (and perhaps others as well?).
• Fix it! :) Seems like initorder.go might be implicated.

[ianlancetaylor]

For the record, gccgo gets this right.

[griesemer]

@whit537 I'm not sure what you mean with the test passes. If you're referring to go/types's test then of course it passes. go/types does this correctly as I mentioned above. The bug is in the compiler (which doesn't use go/types).

[chadwhitacre]

@griesemer @ianlancetaylor I've been working on writing a test and thought I'd provide an update. I don't find any existing tests for package initialization ordering in the compiler. If I'm missing them by all means please point them out. I'm writing a new sinit_test.go file that so far looks like this (yes, it's broken):

// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

package gc

import (
	"cmd/compile/internal/syntax"
	"testing"
)

func TestInitOrder(t *testing.T) {
	tests := []struct {
		src string
	}{
		{`package foo; var (bar = 537)`},
	}
	for i, test := range tests {

		parsed, err := syntax.ParseBytes(nil, []byte(test.src), nil, nil, nil, 0)
		initfix(parsed)

		if test.src != `Greetings, program!` {
			t.Errorf("%d: y u no greet? :(", i)
		}
	}
}

I was hoping to use writing a test or three to comprehend the code so I can find the bug, and I'm trying to find a good place in the call chain at which to test even simple package initialization behavior. The call chain I see is: Main → fninit → initfix → initreorder. What I'm up against is that afaict Main does a lot of processing and mutating of xtop before handing it off to fninit and initfix. I'm looking for a way to short-circuit all of that in a test since initfix seems like the important function here. I have a hunch that noder.go might be the ticket.

That's my current status. I would be happy for feedback if you have it, otherwise I will keep plugging away as I can find the time. :)