Promtail: Error on reload when using "metrics" stage

[darxriggs]

When using a metrics stage and trying to reload the Grafana Agent configuration, it logs the following error:

ts=2023-04-02T10:06:40.238329045Z caller=entrypoint.go:167 level=error msg="failed to update loki" err="failed to unregister all metrics from previous promtail. THIS IS A BUG"

How to reproduce
Scrape from files and configure a metrics stage (like the "counter" example in the documentation):

logs:
  positions_directory: /tmp/positions
  configs:
    - name: metrics example
      clients:
        - url: <FILL IN>
      scrape_configs:
        - job_name: log files
          static_configs:
            - labels:
                __path__: /var/log/*.log
          pipeline_stages:
            - metrics:
               log_lines_total:
                 type: Counter
                 description: "total number of log lines"
                 prefix: my_promtail_custom_
                 max_idle_duration: 24h
                 config:
                   match_all: true
                   action: inc

Send the process a SIGHUP signal.

Environment

• Grafana Agent v0.32.1

[darxriggs]

@tpaschalis A quick check shows that util.Unregisterer.Unregister() returns false for a metric.metricVec.

[darxriggs]

This seems to describe the same issue as in #2754.

[mattdurham]

Closing as duplicate of #2754