Prometheus service has no endpoint after deploying TidbMonitor

[DanielZhangQD]

Bug Report

What version of Kubernetes are you using?

1.12.8
What version of TiDB Operator are you using?

1.1.0-beta.2
What storage classes exist in the Kubernetes cluster and what are used for PD/TiKV pods?

local-storage
What's the status of the TiDB cluster pods?

Running
What did you do?

#1891 (comment)
What did you expect to see?
Prom service can be accessed via NodePort.
What did you see instead?
Prom service can not be accessed via NodePort.

[DanielZhangQD]

There is no endpoint for Prom service even if monitor Pod is running:

dan10-monitor-65455fcc64-9pdb5     3/3     Running   0          28m

# k describe svc dan10-monitor-reloader -n dan10
Name:                     dan10-monitor-reloader
Namespace:                dan10
Labels:                   app.kubernetes.io/component=monitor
                          app.kubernetes.io/instance=dan10
                          app.kubernetes.io/managed-by=tidb-operator
                          app.kubernetes.io/name=tidb-cluster
Annotations:              pingcap.com/last-applied-configuration:
                            {"ports":[{"name":"tcp-reloader","protocol":"TCP","port":9089,"targetPort":9089}],"selector":{"app.kubernetes.io/component":"monitor","app...
Selector:                 app.kubernetes.io/component=monitor,app.kubernetes.io/instance=dan10
Type:                     NodePort
IP:                       10.233.44.68
Port:                     tcp-reloader  9089/TCP
TargetPort:               9089/TCP
NodePort:                 tcp-reloader  30610/TCP
Endpoints:                10.233.102.211:9089
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

The selector of Prom service is as below:

  selector:
    app.kubernetes.io/component: monitor
    app.kubernetes.io/instance: dan10
    app.kubernetes.io/managed-by: tidb-operator
    app.kubernetes.io/name: monitor

But the labels of Pod are as below:

  labels:
    app.kubernetes.io/component: monitor
    app.kubernetes.io/instance: dan10
    app.kubernetes.io/name: tidb-cluster