周期reload实例列表不生效

[BoringCat]

现象： Prometheus内搜不到特定label的指标

过程：

• 2021-07-20 启动exporter
• 2021-07-21 购买一个Redis集群
• 2021-08-12 发现没有新Redis集群的指标，统计指标数量发现少了一个实例
  查看启动日志发现部分namespace没有打印 reload ${namespace} instances every 300 minutes
  但重启后所有配置的namespace都有打印这一句

日志：

初始化日志

level=info ts=2021-07-20T08:57:08.074Z caller=qcloud_exporter.go:86 msg="Starting qcloud_exporter" version="(version=, branch=, revision=)"
level=info ts=2021-07-20T08:57:08.074Z caller=qcloud_exporter.go:87 msg="Build context" build_context="(go=go1.16.5, user=, date=)"
level=info ts=2021-07-20T08:57:08.080Z caller=qcloud_exporter.go:94 msg="Load config ok"
level=info ts=2021-07-20T08:57:08.634Z caller=cache.go:76 msg="Reload metric meta cache" namespace=QCE/CMONGO num=34
level=info ts=2021-07-20T08:57:09.521Z caller=cache.go:104 msg="Reload instance cache" num=4 changed=4
level=info ts=2021-07-20T08:57:09.522Z caller=product.go:227 msg="Init all query ok" Namespace=QCE/CMONGO numMetric=34 numSeries=160
level=info ts=2021-07-20T08:57:09.522Z caller=collector.go:117 msg="Create product collecter ok" Namespace=QCE/CMONGO
level=info ts=2021-07-20T08:57:09.522Z caller=collector.go:124 msg="reload QCE/CMONGO instances every 300 minutes"
level=info ts=2021-07-20T08:57:09.765Z caller=cache.go:76 msg="Reload metric meta cache" namespace=QCE/REDIS_MEM num=75
level=info ts=2021-07-20T08:57:10.187Z caller=cache.go:104 msg="Reload instance cache" num=16 changed=16
level=info ts=2021-07-20T08:57:11.838Z caller=product.go:227 msg="Init all query ok" Namespace=QCE/REDIS_MEM numMetric=26 numSeries=3896
level=info ts=2021-07-20T08:57:11.838Z caller=collector.go:117 msg="Create product collecter ok" Namespace=QCE/REDIS_MEM
level=info ts=2021-07-20T08:57:12.339Z caller=cache.go:76 msg="Reload metric meta cache" namespace=QCE/CDB num=302
level=info ts=2021-07-20T08:57:12.748Z caller=cache.go:104 msg="Reload instance cache" num=135 changed=135
level=info ts=2021-07-20T08:57:12.754Z caller=product.go:227 msg="Init all query ok" Namespace=QCE/CDB numMetric=11 numSeries=1485
level=info ts=2021-07-20T08:57:12.754Z caller=collector.go:117 msg="Create product collecter ok" Namespace=QCE/CDB
level=info ts=2021-07-20T08:57:12.988Z caller=cache.go:76 msg="Reload metric meta cache" namespace=QCE/NAT_GATEWAY num=6
level=info ts=2021-07-20T08:57:13.405Z caller=cache.go:104 msg="Reload instance cache" num=1 changed=1
level=info ts=2021-07-20T08:57:13.405Z caller=product.go:227 msg="Init all query ok" Namespace=QCE/NAT_GATEWAY numMetric=6 numSeries=6
level=info ts=2021-07-20T08:57:13.405Z caller=collector.go:117 msg="Create product collecter ok" Namespace=QCE/NAT_GATEWAY
level=info ts=2021-07-20T08:57:13.405Z caller=collector.go:124 msg="reload QCE/NAT_GATEWAY instances every 300 minutes"
level=info ts=2021-07-20T08:57:13.405Z caller=collector.go:131 msg="Create all product collecter ok" num=4
level=info ts=2021-07-20T08:57:13.405Z caller=qcloud_exporter.go:114 msg="Listening on" address=:9123

后面都是 Start collect ...... Collect done 没有有用的内容

重启后日志

level=info ts=2021-08-12T11:34:35.968Z caller=qcloud_exporter.go:86 msg="Starting qcloud_exporter" version="(version=, branch=, revision=)"
level=info ts=2021-08-12T11:34:36.044Z caller=qcloud_exporter.go:87 msg="Build context" build_context="(go=go1.16.5, user=, date=)"
level=info ts=2021-08-12T11:34:36.091Z caller=qcloud_exporter.go:94 msg="Load config ok"
level=info ts=2021-08-12T11:34:36.734Z caller=cache.go:76 msg="Reload metric meta cache" namespace=QCE/CMONGO num=34
level=info ts=2021-08-12T11:34:37.634Z caller=cache.go:104 msg="Reload instance cache" num=4 changed=4
level=info ts=2021-08-12T11:34:37.635Z caller=product.go:227 msg="Init all query ok" Namespace=QCE/CMONGO numMetric=34 numSeries=160
level=info ts=2021-08-12T11:34:37.635Z caller=collector.go:117 msg="Create product collecter ok" Namespace=QCE/CMONGO
level=info ts=2021-08-12T11:34:37.635Z caller=collector.go:124 msg="reload QCE/CMONGO instances every 300 minutes"
level=info ts=2021-08-12T11:34:37.882Z caller=cache.go:76 msg="Reload metric meta cache" namespace=QCE/REDIS_MEM num=77
level=info ts=2021-08-12T11:34:38.328Z caller=cache.go:104 msg="Reload instance cache" num=16 changed=16
level=info ts=2021-08-12T11:34:39.985Z caller=product.go:227 msg="Init all query ok" Namespace=QCE/REDIS_MEM numMetric=26 numSeries=2720
level=info ts=2021-08-12T11:34:39.985Z caller=collector.go:117 msg="Create product collecter ok" Namespace=QCE/REDIS_MEM
level=info ts=2021-08-12T11:34:40.379Z caller=cache.go:76 msg="Reload metric meta cache" namespace=QCE/CDB num=302
level=info ts=2021-08-12T11:34:40.736Z caller=cache.go:104 msg="Reload instance cache" num=134 changed=134
level=info ts=2021-08-12T11:34:40.744Z caller=product.go:227 msg="Init all query ok" Namespace=QCE/CDB numMetric=11 numSeries=1474
level=info ts=2021-08-12T11:34:40.744Z caller=collector.go:117 msg="Create product collecter ok" Namespace=QCE/CDB
level=info ts=2021-08-12T11:34:40.992Z caller=cache.go:76 msg="Reload metric meta cache" namespace=QCE/NAT_GATEWAY num=8
level=info ts=2021-08-12T11:34:41.195Z caller=cache.go:104 msg="Reload instance cache" num=1 changed=1
level=info ts=2021-08-12T11:34:41.195Z caller=product.go:227 msg="Init all query ok" Namespace=QCE/NAT_GATEWAY numMetric=8 numSeries=8
level=info ts=2021-08-12T11:34:41.195Z caller=collector.go:117 msg="Create product collecter ok" Namespace=QCE/NAT_GATEWAY
level=info ts=2021-08-12T11:34:41.195Z caller=collector.go:124 msg="reload QCE/NAT_GATEWAY instances every 300 minutes"
level=info ts=2021-08-12T11:34:41.195Z caller=collector.go:131 msg="Create all product collecter ok" num=4
level=info ts=2021-08-12T11:34:41.195Z caller=qcloud_exporter.go:114 msg="Listening on" address=:9123

使用的镜像： boringcat/qcloud-exporter:v2.3.0

Dockerfile

FROM golang:alpine as builder
ARG VERSION
RUN set -xe

;  [ -z "${VERSION}" ] && apk add --update curl jq

&& VERSION=curl -s https://api.github.com/repos/tencentyun/tencentcloud-exporter/releases/latest | jq -r .name

;  VERSION=${VERSION##*v}

&& wget https://github.com/tencentyun/tencentcloud-exporter/archive/refs/tags/v${VERSION}.tar.gz -O /tmp/v${VERSION}.tar.gz

&& tar xf /tmp/v${VERSION}.tar.gz tencentcloud-exporter-${VERSION}

&& cd tencentcloud-exporter-${VERSION}

&& go build -o /qcloud-exporter cmd/qcloud-exporter/qcloud_exporter.go
FROM alpine

COPY --from=builder /qcloud-exporter /usr/local/bin/qcloud-exporter

ENTRYPOINT [ "/usr/local/bin/qcloud-exporter" ]

EXPOSE 9123

部分配置如下：

products:
  - namespace: QCE/REDIS_MEM
    all_instances: true
    extra_labels: [InstanceName, WanIp]
    instance_filters:
        Status: 2
    relod_interval_minutes: 300