You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
The current version of the gRPC client used has a behavior where it won't update the list of hosts if the initial list is empty. See jaegertracing/jaeger-operator#1175, which is likely caused by grpc/grpc-go#3353. Bumping the gRPC dependency to the latest (v1.31.1) makes it work.
To Reproduce
Steps to reproduce the behavior:
Using the Jaeger Operator, deploy the example agent-as-daemonset
Get the logs for the daemonset: $ kubectl logs daemonsets/agent-as-daemonset-agent-daemonset -f
See that the list of hosts isn't updated even after a couple of minutes:
$ kubectl logs daemonsets/agent-as-daemonset-agent-daemonset -f2020/09/01 15:13:00 maxprocs: Leaving GOMAXPROCS=6: CPU quota undefined{"level":"info","ts":1598973180.944437,"caller":"flags/service.go:116","msg":"Mounting metrics handler on admin server","route":"/metrics"}{"level":"info","ts":1598973180.9445577,"caller":"flags/admin.go:120","msg":"Mounting health check on admin server","route":"/"}{"level":"info","ts":1598973180.9446096,"caller":"flags/admin.go:126","msg":"Starting admin HTTP server","http-addr":":14271"}{"level":"info","ts":1598973180.944623,"caller":"flags/admin.go:112","msg":"Admin server started","http.host-port":"[::]:14271","health-status":"unavailable"}{"level":"info","ts":1598973180.9454503,"caller":"grpc/builder.go:66","msg":"Agent requested insecure grpc connection to collector(s)"}{"level":"info","ts":1598973180.9454787,"caller":"grpc@v1.28.0-pre/clientconn.go:106","msg":"parsed scheme: \"dns\"","system":"grpc","grpc_log":true}{"level":"info","ts":1598973180.9479086,"caller":"command-line-arguments/main.go:78","msg":"Starting agent"}{"level":"info","ts":1598973180.9479551,"caller":"healthcheck/handler.go:128","msg":"Health Check state change","status":"ready"}{"level":"info","ts":1598973180.947995,"caller":"app/agent.go:69","msg":"Starting jaeger-agent HTTP server","http-port":5778}{"level":"info","ts":1598973180.9573705,"caller":"dns/dns_resolver.go:212","msg":"ccResolverWrapper: sending update to cc: {[] <nil> <nil>}","system":"grpc","grpc_log":true}{"level":"info","ts":1598973180.9574049,"caller":"grpc@v1.28.0-pre/clientconn.go:948","msg":"ClientConn switching balancer to \"round_robin\"","system":"grpc","grpc_log":true}
Bumping gRPC to the latest, after 30s in this state, the list of servers is updated:
$ kubectl logs daemonsets/agent-as-daemonset-agent-daemonset -f2020/09/01 15:16:04 maxprocs: Leaving GOMAXPROCS=6: CPU quota undefined{"level":"info","ts":1598973364.7745779,"caller":"flags/service.go:116","msg":"Mounting metrics handler on admin server","route":"/metrics"}{"level":"info","ts":1598973364.7747014,"caller":"flags/admin.go:120","msg":"Mounting health check on admin server","route":"/"}{"level":"info","ts":1598973364.774737,"caller":"flags/admin.go:126","msg":"Starting admin HTTP server","http-addr":":14271"}{"level":"info","ts":1598973364.7747464,"caller":"flags/admin.go:112","msg":"Admin server started","http.host-port":"[::]:14271","health-status":"unavailable"}{"level":"info","ts":1598973364.7751777,"caller":"grpc/builder.go:66","msg":"Agent requested insecure grpc connection to collector(s)"}{"level":"info","ts":1598973364.7751987,"caller":"grpc@v1.27.1/clientconn.go:106","msg":"parsed scheme: \"dns\"","system":"grpc","grpc_log":true}{"level":"info","ts":1598973364.776601,"caller":"command-line-arguments/main.go:78","msg":"Starting agent"}{"level":"info","ts":1598973364.776632,"caller":"healthcheck/handler.go:128","msg":"Health Check state change","status":"ready"}{"level":"info","ts":1598973364.7766716,"caller":"app/agent.go:69","msg":"Starting jaeger-agent HTTP server","http-port":5778}{"level":"info","ts":1598973364.7881553,"caller":"dns/dns_resolver.go:212","msg":"ccResolverWrapper: sending update to cc: {[] <nil> <nil>}","system":"grpc","grpc_log":true}{"level":"info","ts":1598973364.7881958,"caller":"grpc@v1.27.1/clientconn.go:948","msg":"ClientConn switching balancer to \"round_robin\"","system":"grpc","grpc_log":true}{"level":"info","ts":1598973394.8044846,"caller":"dns/dns_resolver.go:212","msg":"ccResolverWrapper: sending update to cc: {[{10.244.0.14:14250 <nil> 0 <nil>}] <nil> <nil>}","system":"grpc","grpc_log":true}{"level":"info","ts":1598973394.8048198,"caller":"base/balancer.go:196","msg":"roundrobinPicker: newPicker called with info: {map[]}","system":"grpc","grpc_log":true}{"level":"info","ts":1598973394.8062818,"caller":"base/balancer.go:196","msg":"roundrobinPicker: newPicker called with info: {map[0xc0004121c0:{{10.244.0.14:14250 <nil> 0 <nil>}}]}","system":"grpc","grpc_log":true}
Expected behavior
The gRPC client should be able to get DNS updates, especially if the initial list is empty.
Version (please complete the following information):
Jaeger version: master as of now, v1.19.2
Deployment: Kubernetes
The text was updated successfully, but these errors were encountered:
Describe the bug
The current version of the gRPC client used has a behavior where it won't update the list of hosts if the initial list is empty. See jaegertracing/jaeger-operator#1175, which is likely caused by grpc/grpc-go#3353. Bumping the gRPC dependency to the latest (v1.31.1) makes it work.
To Reproduce
Steps to reproduce the behavior:
agent-as-daemonset
$ kubectl logs daemonsets/agent-as-daemonset-agent-daemonset -f
Bumping gRPC to the latest, after 30s in this state, the list of servers is updated:
Expected behavior
The gRPC client should be able to get DNS updates, especially if the initial list is empty.
Version (please complete the following information):
The text was updated successfully, but these errors were encountered: