BUG: JSON decodes array of structs wrong

[magiconair]

The JSON parser of the hcl package decodes two elements for the inner array instead of one:

{"x":[{"a":1, "b":2}]}

Here is the Go test to verify this:

// === RUN   TestArrayWithStruct
// === RUN   TestArrayWithStruct/encoding/json
// === RUN   TestArrayWithStruct/hashicorp/hcl
// --- FAIL: TestArrayWithStruct (0.00s)
//     --- PASS: TestArrayWithStruct/encoding/json (0.00s)
//     --- FAIL: TestArrayWithStruct/hashicorp/hcl (0.00s)
//         main_test.go:38: got main.Y{X:[]main.X{main.X{A:1, B:0}, main.X{A:0, B:2}}} want main.Y{X:[]main.X{main.X{A:1, B:2}}}
func TestArrayWithStruct(t *testing.T) {
    type X struct {
        A, B int
    }
    type Y struct {
        X []X
    }

    j := `{"x":[{"a":1,"b":2}]}`
    y := Y{X: []X{{A: 1, B: 2}}}
    t.Run("encoding/json", func(t *testing.T) {
        var v Y
        if err := json.Unmarshal([]byte(j), &v); err != nil {
            t.Fatalf("got error %v want nil", err)
        }
        if got, want := v, y; !reflect.DeepEqual(got, want) {
            t.Fatalf("got %#v want %#v", got, want)
        }
    })
    t.Run("hashicorp/hcl", func(t *testing.T) {
        var v Y
        if err := hcl.Decode(&v, j); err != nil {
            t.Fatalf("got error %v want nil", err)
        }
        if got, want := v, y; !reflect.DeepEqual(got, want) {
            t.Fatalf("got %#v want %#v", got, want)
        }
    })
}

[sstarcher]

This just bit us also when moving from direct hcl files to json files.

[thomasmitchell]

So, we're running into this now too. Patched in some print lines to verify a hunch that this was happening during the flattening phase.

INPUT:
{"x":[{"a":1, "b":2}, {"c": 3}]}
AFTER PARSE:
"x" = [{
  "a" = 1

  "b" = 2
}, {
  "c" = 3
}]
AFTER FLATTEN:
"x" = {
  "a" = 1

  "b" = 2
}

"x" = {
  "c" = 3
}

...and it sure seems like it. I'm not informed enough on the actual purpose of the flattening step to be able to fix it, though. The one case I can find where its doing something is this one, where it's creating undesirable behavior.

[bfrancojr]

I'm running into similar issue but looks like it is a hcl decoder problem.
Take a look at this.

[shakefu]

This is for sure a bug - I ran into it using json2hcl, and it deconstructs the list into a series of map assignments.

Unfortunately, this bug is also present in how Terraform parses JSON - Running terraform plan throws an error despite the JSON being correct in specifying a list of maps.

I was able to work around it in the module.container_load_balancer.health_check key - it seems interpolated maps don't suffer, so it's in parsing. However that doesn't help when you need calculated values.

Example JSON:

  // <SNIP>
  "module": {
    "container_load_balancer": {
      "health_check": [
        "${var.container_health_check}"
      ],
      "internal": false,
      "listener": [
        {
          "instance_port": 80,
          "instance_protocol": "HTTP",
          "lb_port": 443,
          "lb_protocol": "HTTPS",
          "ssl_certificate_id": "${aws_iam_server_certificate.domain.arn}"
        }
      ],
      "name": "${var.phase}-container-hosts",
      "security_groups": [
        "${module.egress_all_security_group.this_security_group_id}",
        "${module.http_security_group.this_security_group_id}"
      ],
      "source": "terraform-aws-modules/elb/aws",
      "subnets": "${module.vpc.public_subnets}",
      "tags": "${var.tags}"
    }
  },
  // <SNIP>

Error output:

Error: Error asking for user input: 1 error(s) occurred:

* module.container_load_balancer.var.listener: variable listener in module container_load_balancer should be type list, got map

... when

[nbering]

Will this work properly in HCL2? I wanted to try it out in Terraform 0.12 alpha, but would've had to recompile a provider for the new plugin protocol version.