promtail: panic when fetching target Details

[tpaschalis]

Describe the bug
Calling the Details() method of a target that has a nil error results in a panic.

loki/clients/pkg/promtail/targets/docker/target.go

Lines 252 to 256 in 2911a74

	// Details returns target-specific details.
	func (t *Target) Details() interface{} {
	return map[string]string{
	"id": t.containerName,
	"error": t.err.Error(),

goroutine 2580 [running]:
net/http.(*conn).serve.func1()
	/usr/local/go/src/net/http/server.go:1825 +0xbf
panic({0x3f48720, 0x7666310})
	/usr/local/go/src/runtime/panic.go:844 +0x258
github.com/opentracing-contrib/go-stdlib/nethttp.MiddlewareFunc.func5.1()
	/go/pkg/mod/github.com/opentracing-contrib/go-stdlib@v1.0.0/nethttp/server.go:150 +0x139
panic({0x3f48720, 0x7666310})
	/usr/local/go/src/runtime/panic.go:838 +0x207
github.com/grafana/loki/clients/pkg/promtail/targets/docker.(*Target).Details(0xc000c91340)
	/go/pkg/mod/github.com/grafana/loki@v1.6.2-0.20220411060007-0315102c34cf/clients/pkg/promtail/targets/docker/target.go:238 +0xa5
github.com/grafana/agent/pkg/logs.listTargetsHandler.func1({0x52bdb88, 0xc000477c20}, 0xc000739c70?)
	/drone/src/pkg/logs/http.go:59 +0x2cf
net/http.HandlerFunc.ServeHTTP(0xc000477d10?, {0x52bdb88?, 0xc000477c20?}, 0xc?)
	/usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/grafana/agent/pkg/logs.(*Logs).ListTargetsHandler(0x3bd4540?, {0x52bdb88, 0xc000477c20}, 0xc000ea9360?)
	/drone/src/pkg/logs/http.go:43 +0x154

To Reproduce
We witnessed this behavior in the Agent code, when trying to list the status of Logs targets fetched from the Docker SD mechanism, but the same seems to be true for most implementations of the Details() method. As far as I can tell with a quick search, the method is only used in Loki for logging which might be why it hasn't been reported before.

Steps to reproduce the behavior:
Call the Details() method on a target that hasn't errored out.

Expected behavior
The Details() method should return an empty string or a nil value.

Environment:
Any.

Screenshots, Promtail config, or terminal output
For completeness' sake, here's the job definition used under Promtail's scrape_configs.

      - job_name: integrations/docker
        docker_sd_configs:
          - host: unix:///var/run/docker.sock
            refresh_interval: 5s
        relabel_configs:
          - action: replace
            replacement: integrations/docker
            source_labels:
              - __meta_docker_container_id
            target_label: job
          - source_labels:
              - __meta_docker_container_name
            regex: '/(.*)'
            target_label: container
          - source_labels:
            - __meta_docker_container_log_stream
            target_label: stream

[beningodfrey4]

Can confirm similar behavior while testing the grafana agent promtail targets endpoint: http://<ip>:12345/agent/api/v1/logs/targets.

Doesn't look like scraping is affected in any way, until the endpoint is probed that is.

[GeorgeTsilias]

I can try to tackle this if that's okay with you.

[GeorgeTsilias]

Sorry for being extra late on this, my bad.
I've opened a PR which hopefully addresses the issue above.
PR 7771