Consider Integration/Supersedence with Upcoming Go Structured Logging Capabilities

[bflad]

terraform-plugin-log version

v0.7.0

Description

Go Day 2022 included a talk on Go's proposed approach to structured logging. The golang.org/x/exp/slog package (currently experimental) has leveled logging, supports context, and is being designed for flexibility as a common interface for other packages. Its intended goals are: easy to use, fast, interoperates with existing packages, links to the log package. It uses an intermixed slice of string keys to values for structured logging fields, but does also support "attrs" for speed/correctness (e.g. slog.String("key", "value")).

This could potentially mean moving away or off github.com/hashicorp/go-hclog as a dependency.

References

• https://opensourcelive.withgoogle.com/events/go-day-2022/watch?talk=talk2
• https://pkg.go.dev/golang.org/x/exp/slog
• structured, leveled logging golang/go#54763
• https://go.dev/s/slog-design

[bflad]

Looks like the golang/go#56345 proposal has been accepted. 🎉

[ewbankkit]

Go 1.21 has been released.

[imjasonh]

Hacked this up briefly, in case it's a helpful start:

func init() {
	slog.SetDefault(slog.New(tfhandler{}))
}

type tfhandler struct{}

func (tfhandler) Handle(ctx context.Context, r slog.Record) error {
	addl := make(map[string]interface{})
	r.Attrs(func(s slog.Attr) bool {
		addl[s.Key] = s.Value.String()
		return true
	})
	switch r.Level {
	case slog.LevelDebug:
		tflog.Debug(ctx, r.Message, addl)
	case slog.LevelInfo:
		tflog.Info(ctx, r.Message, addl)
	case slog.LevelWarn:
		tflog.Warn(ctx, r.Message, addl)
	case slog.LevelError:
		tflog.Error(ctx, r.Message, addl)
	default:
		tflog.Info(ctx, r.Message, addl)
	}
	return nil
}

func (tfhandler) Enabled(context.Context, slog.Level) bool { return true }
func (tfhandler) WithAttrs(attrs []slog.Attr) slog.Handler { panic("unimplemented") }
func (tfhandler) WithGroup(name string) slog.Handler       { panic("unimplemented") }

Then if some code calls slog.InfoContext(ctx, "hello", "planet", "earth") it gets tflog.Info'ed with those additional fields.

There's more to add, and test, but it'd be great if just importing tflog had this side effect IMO.

[bflad]

This Go module has now been updated to Go 1.21 minimum, so usage of the Go standard library log/slog package in general is now okay. When I had originally written this issue, it was more along the angle of using slog to fully move off github.com/hashicorp/hc-log, but @imjasonh that is also an interesting and valid proposal. I think if we were to offer a slog to tflog default handler such as that, we would want to introduce it in a separate package (tflog/slog?) so developers can opt into the init() function behavior, rather than forcing them into the behavior. Maybe that is best captured in its own feature request since they can/should likely be implemented separately.

[maratori]

Here is my naive adapter

type slogHandler struct {
	attrs  []slog.Attr
	groups []string
}

var _ slog.Handler = (*slogHandler)(nil)

func (*slogHandler) Enabled(context.Context, slog.Level) bool {
	return true
}

func (h *slogHandler) Handle(ctx context.Context, record slog.Record) error {
	switch record.Level {
	case slog.LevelDebug:
		tflog.Debug(ctx, record.Message, h.fields(record))
	case slog.LevelInfo:
		tflog.Info(ctx, record.Message, h.fields(record))
	case slog.LevelWarn:
		tflog.Warn(ctx, record.Message, h.fields(record))
	case slog.LevelError:
		tflog.Error(ctx, record.Message, h.fields(record))
	default:
		tflog.Info(ctx, record.Message, h.fields(record))
	}
	return nil
}

func (h *slogHandler) WithAttrs(attrs []slog.Attr) slog.Handler {
	return &slogHandler{
		attrs:  append(h.attrs, attrs...),
		groups: h.groups,
	}
}

func (h *slogHandler) WithGroup(name string) slog.Handler {
	if name == "" {
		return h
	}
	return &slogHandler{
		attrs:  h.attrs,
		groups: append(h.groups, name),
	}
}

func (h *slogHandler) fields(record slog.Record) map[string]any {
	root := make(map[string]any, len(h.attrs)+record.NumAttrs())

	fields := root
	for _, name := range h.groups {
		nested := make(map[string]any, len(h.attrs)+record.NumAttrs())
		fields[name] = nested
		fields = nested
	}

	addAttrsToMap(h.attrs, fields)

	record.Attrs(func(attr slog.Attr) bool {
		addAttrToMap(attr, fields)
		return true
	})

	return root
}

func addAttrsToMap(attrs []slog.Attr, fields map[string]any) {
	for _, a := range attrs {
		addAttrToMap(a, fields)
	}
}

func addAttrToMap(attr slog.Attr, fields map[string]any) {
	if attr.Equal(slog.Attr{}) {
		return
	}

	val := attr.Value.Resolve()

	if val.Kind() == slog.KindGroup {
		attrs := val.Group()
		if len(attrs) == 0 {
			return
		}

		if attr.Key == "" {
			addAttrsToMap(attrs, fields)
			return
		}

		group := make(map[string]any, len(attrs))
		addAttrsToMap(attrs, group)
		fields[attr.Key] = group

		return
	}

	fields[attr.Key] = val.Any()
}