Unable to Unmarshal into *toml.Tree

[A-UNDERSCORE-D]

Describe the bug
I want to unmarshal parts of a TOML file into an *toml.Tree for later inspection and unmarshaling (in this case the type is not known to the main unmarshal code in my config code)

To Reproduce
I tested this using a quick test file in my larger project while working out how to go about a few things. You can extract the toml data directly from here

package tomlconf

import (
	"testing"

	"github.com/pelletier/go-toml"
)

type outer struct {
	Thing string
	Other *toml.Tree `toml:"other"`
	Asd   struct {
		X string
	}
}
func TestTheThing(t *testing.T) {
	const test = `
	thing = "stuff"
	[other]
	stuff = "more stuff"
	otherStuff = 1337
	[asd]
	x = "v"
	`

	res, err := toml.Load(test)
	if err != nil {
		t.Error(err)
	}

	x := &outer{}

	if err := res.Unmarshal(x); err != nil {
		t.Error(err)
	}

	t.Logf("%#v", x)
	t.Logf("%#v", x.Other)
	t.Logf("%T: %[1]s", res.Get("other"))
}

Expected behavior
I expected x.Other to be filled with the same tree that res.Get returns, however what I got was an empty toml.Tree (it was not nil--but it was the zero value):

entire object:

&tomlconf.outer{Thing:"stuff", Other:(*toml.Tree)(0xc00010c5d0), Asd:struct { X string }{X:"v"}}

closer look at x.Other:

&toml.Tree{values:map[string]interface {}(nil), comment:"", commented:false, position:toml.Position{Line:0, Col:0}}

type and content (via stringer) of return from res.Get:

*toml.Tree: otherStuff = 1337
        stuff = "more stuff"

Versions

• go-toml: from go.mod: github.com/pelletier/go-toml v1.6.0
• go: 1.14
• operating system: Linux

Additional context
I'm not entirely familiar with how go-toml works as this is my first use of it, but as I understand your docs this should work?

[pelletier]

Hi! Sorry it took a bit to get back to you. Unfortunately, go-toml is not great at unmarshaling unknown objects (main issue to track this is #337).

If you are fine with parsing the file twice, you can unmarshal the outer struct, then replace its Other member with the tree you get from toml.Load (see code below). Not ideal but I think that's the best option at this moment. You can also only use toml.Load and manually fill the struct if parsing the file twice is not an option.

Hope that helps!

package main

import (
	"github.com/davecgh/go-spew/spew"
	"github.com/pelletier/go-toml"
)

type outer struct {
	Thing string
	Other *toml.Tree // will not be filled with unmarshal
	Asd   struct {
		X string
	}
}

func main() {
	const test = `
	thing = "stuff"
	[other]
	stuff = "more stuff"
	otherStuff = 1337
	[asd]
	x = "v" 
	`

	tree, err := toml.Load(test)
	if err != nil {
		panic(err)
	}

	x := outer{}
	err = toml.Unmarshal([]byte(test), &x)
	if err != nil {
		panic(err)
	}
	x.Other = tree.Get("other").(*toml.Tree)

	spew.Dump(x)

	/*
		(main.outer) {
		Thing: (string) (len=5) "stuff",
		Other: (*toml.Tree)(0xc42008a660)(otherStuff = 1337
		stuff = "more stuff"
		),
		Asd: (struct { X string }) {
		X: (string) (len=1) "v"
		}
		}
	*/
}

[A-UNDERSCORE-D]

Yeah thats about what I came to as a solution for a workaround, unfortunate that it doesnt work OOTB but its Im quite sure not a simple process to get that right. I may take a crack at it at some point if doing it the other way annoys me too much. Pretty much my only deviation from what you suggested is that I abuse a Tree from Load, rather than parsing twice:

func makeConfig(tree *toml.Tree) (*Config, error) {
	out := new(Config)
	if err := tree.Unmarshal(out); err != nil {
		return nil, err
	}

	if res, ok := tree.Get("connection.server").(*toml.Tree); ok {
		out.Connection.RealConf = res
	}

	for name, game := range out.Games {
		data := tree.GetPath([]string{"games", name, "transport"})
		if res, ok := data.(*toml.Tree); ok {
			game.Transport.RealConf = res
		}
	}

	return out, nil
}

[pelletier]

Ha right, good call on unmarshaling the tree after parsed. This is definitely and highly requested issue that I haven’t had time to fix.

[AllenX2018]

Does #341 satisfy this requirement?

[A-UNDERSCORE-D]

I may be able to use it to reduce some boilerplate code in my unmarshal, but it becomes more boilerplate elsewhere where I'd need to convert from a map[string]interface{} or similar to my concrete struct, whereas currently I can ask the *Tree to do that for me.

The reason I need/want to go about this like this is that my main config package has no idea what some parts of its config represent, as they're based on other parts. e.g. the transport config section can be completely different depending on what the field type says it is. And those are all defined elsewhere.