Envoy not adding public listener or upstream listener when run with 1.10.1 Consul agent #10714
Labels
Comments
|
FYI, I got a notification that issue label workflow failed on this. |
dnephin
added
theme/connect
|
Hi @dschaaff, Thanks for reaching out! There are a few pieces of additional information that would help us dig into this further:
|
jkirschner-hashicorp
added
the
waiting-reply
envoy versionValues file for consul-helmfullnameOverride: consul
# Available parameters and their default values for the Consul chart.
global:
enabled: false
domain: consul
image: "xxxxx.dkr.ecr.us-west-2.amazonaws.com/consul:1.10.1"
imageK8S: "xxxx.dkr.ecr.us-west-2.amazonaws.com/consul-k8s:0.25.0"
imageEnvoy: "xxxxx.dkr.ecr.us-west-2.amazonaws.com/envoy:v1.16.4"
datacenter: xxxx
enablePodSecurityPolicies: false
gossipEncryption:
secretName: consul-secrets
secretKey: gossip-encryption-key
tls:
enabled: true
enableAutoEncrypt: true
serverAdditionalDNSSANs: []
serverAdditionalIPSANs: []
verify: true
httpsOnly: false
caCert:
secretName: consul-secrets
secretKey: ca.crt
caKey:
secretName: null
secretKey: null
server:
enabled: false
externalServers:
enabled: true
hosts: [xxxxxxxxxx]
httpsPort: 443
tlsServerName: null
useSystemRoots: true
client:
enabled: true
image: null
join:
- "provider=aws tag_key=consul-datacenter tag_value=xxxxxx"
grpc: true
exposeGossipPorts: false
resources:
requests:
memory: "400Mi"
cpu: "200m"
limits:
cpu: "500m"
memory: "400Mi"
extraConfig: |
{
"telemetry": {
"disable_hostname": true,
"prometheus_retention_time": "6h"
}
}
extraVolumes:
- type: secret
name: consul-secrets
load: false
- type: secret
name: consul-acl-config
load: true
tolerations: ""
nodeSelector: null
annotations: null
extraEnvironmentVars:
CONSUL_HTTP_TOKEN_FILE: /consul/userconfig/consul-secrets/consul.token
dns:
enabled: true
ui:
enabled: false
syncCatalog:
enabled: true
image: null
default: true # true will sync by default, otherwise requires annotation
toConsul: true
toK8S: false
k8sPrefix: null
consulPrefix: null
k8sTag: k8s-cluster-name
syncClusterIPServices: true
nodePortSyncType: ExternalFirst
aclSyncToken:
secretName: consul-secrets
secretKey: consul-k8s-sync.token
connectInject:
enabled: true
default: false
resources:
requests:
memory: "500Mi"
cpu: "100m"
limits:
memory: "750Mi"
healthChecks:
enabled: true
reconcilePeriod: "1m"
overrideAuthMethodName: kubernetes
aclInjectToken:
secretName: consul-secrets
secretKey: consul.token
centralConfig:
enabled: true
sidecarProxy:
resources:
requests:
memory: 150Mi
cpu: 100m
limits:
memory: 150Mi
On non k8s nodes I run envoy inside of a docker container managed by systemdSystemd unit fileEnvoy bootstrap configsoutput of bootstrap on consul 1.9.5{
"admin": {
"access_log_path": "/dev/null",
"address": {
"socket_address": {
"address": "127.0.0.1",
"port_value": 19000
}
}
},
"node": {
"cluster": "redacted",
"id": "redacted",
"metadata": {
"namespace": "default",
"envoy_version": "1.16.4"
}
},
"static_resources": {
"clusters": [
{
"name": "local_agent",
"connect_timeout": "1s",
"type": "STATIC",
"tls_context": {
"common_tls_context": {
"validation_context": {
"trusted_ca": {
"inline_string": "redacted"
}
}
}
},
"http2_protocol_options": {},
"hosts": [
{
"socket_address": {
"address": "127.0.0.1",
"port_value": 8502
}
}
]
},
{
"name": "self_admin",
"connect_timeout": "5s",
"type": "STATIC",
"http_protocol_options": {},
"hosts": [
{
"socket_address": {
"address": "127.0.0.1",
"port_value": 19000
}
}
]
}
],
"listeners": [
{
"name": "envoy_prometheus_metrics_listener",
"address": {
"socket_address": {
"address": "0.0.0.0",
"port_value": 9102
}
},
"filter_chains": [
{
"filters": [
{
"name": "envoy.http_connection_manager",
"config": {
"stat_prefix": "envoy_prometheus_metrics",
"codec_type": "HTTP1",
"route_config": {
"name": "self_admin_route",
"virtual_hosts": [
{
"name": "self_admin",
"domains": [
"*"
],
"routes": [
{
"match": {
"path": "/metrics"
},
"route": {
"cluster": "self_admin",
"prefix_rewrite": "/stats/prometheus"
}
},
{
"match": {
"prefix": "/"
},
"direct_response": {
"status": 404
}
}
]
}
]
},
"http_filters": [
{
"name": "envoy.router"
}
]
}
}
]
}
]
}
]
},
"stats_config": {
"stats_tags": [
{
"regex": "^cluster\\.((?:([^.]+)~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.custom_hash"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.service_subset"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.service"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.namespace"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.datacenter"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.routing_type"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.consul\\.)",
"tag_name": "consul.destination.trust_domain"
},
{
"regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.target"
},
{
"regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+)\\.consul\\.)",
"tag_name": "consul.destination.full_target"
},
{
"regex": "^(?:tcp|http)\\.upstream\\.(([^.]+)(?:\\.[^.]+)?\\.[^.]+\\.)",
"tag_name": "consul.upstream.service"
},
{
"regex": "^(?:tcp|http)\\.upstream\\.([^.]+(?:\\.[^.]+)?\\.([^.]+)\\.)",
"tag_name": "consul.upstream.datacenter"
},
{
"regex": "^(?:tcp|http)\\.upstream\\.([^.]+(?:\\.([^.]+))?\\.[^.]+\\.)",
"tag_name": "consul.upstream.namespace"
},
{
"regex": "^cluster\\.((?:([^.]+)~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.custom_hash"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.service_subset"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.service"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.namespace"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.datacenter"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.consul\\.)",
"tag_name": "consul.routing_type"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.consul\\.)",
"tag_name": "consul.trust_domain"
},
{
"regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.target"
},
{
"regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+)\\.consul\\.)",
"tag_name": "consul.full_target"
},
{
"tag_name": "local_cluster",
"fixed_value": "redacted"
},
{
"tag_name": "consul.source.service",
"fixed_value": "redacted"
},
{
"tag_name": "consul.source.namespace",
"fixed_value": "default"
},
{
"tag_name": "consul.source.datacenter",
"fixed_value": "redacted"
}
],
"use_all_default_tags": true
},
"dynamic_resources": {
"lds_config": {
"ads": {}
},
"cds_config": {
"ads": {}
},
"ads_config": {
"api_type": "GRPC",
"grpc_services": {
"initial_metadata": [
{
"key": "x-consul-token",
"value": "redacted"
}
],
"envoy_grpc": {
"cluster_name": "local_agent"
}
}
}
}
}output of bootstrap on consul 1.10.1{
"admin": {
"access_log_path": "/dev/null",
"address": {
"socket_address": {
"address": "127.0.0.1",
"port_value": 19000
}
}
},
"node": {
"cluster": "redacted",
"id": "redacted-sidecar-proxy",
"metadata": {
"namespace": "default",
"envoy_version": "1.16.4"
}
},
"static_resources": {
"clusters": [
{
"name": "local_agent",
"ignore_health_on_host_removal": false,
"connect_timeout": "1s",
"type": "STATIC",
"transport_socket": {
"name": "tls",
"typed_config": {
"@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext",
"common_tls_context": {
"validation_context": {
"trusted_ca": {
"inline_string": "redacted"
}
}
}
}
},
"http2_protocol_options": {},
"loadAssignment": {
"clusterName": "local_agent",
"endpoints": [
{
"lbEndpoints": [
{
"endpoint": {
"address": {
"socket_address": {
"address": "127.0.0.1",
"port_value": 8502
}
}
}
}
]
}
]
}
},
{
"name": "self_admin",
"ignore_health_on_host_removal": false,
"connect_timeout": "5s",
"type": "STATIC",
"http_protocol_options": {},
"loadAssignment": {
"clusterName": "self_admin",
"endpoints": [
{
"lbEndpoints": [
{
"endpoint": {
"address": {
"socket_address": {
"address": "127.0.0.1",
"port_value": 19000
}
}
}
}
]
}
]
}
}
],
"listeners": [
{
"name": "envoy_prometheus_metrics_listener",
"address": {
"socket_address": {
"address": "0.0.0.0",
"port_value": 9102
}
},
"filter_chains": [
{
"filters": [
{
"name": "envoy.filters.network.http_connection_manager",
"typedConfig": {
"@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager",
"stat_prefix": "envoy_prometheus_metrics",
"codec_type": "HTTP1",
"route_config": {
"name": "self_admin_route",
"virtual_hosts": [
{
"name": "self_admin",
"domains": [
"*"
],
"routes": [
{
"match": {
"path": "/metrics"
},
"route": {
"cluster": "self_admin",
"prefix_rewrite": "/stats/prometheus"
}
},
{
"match": {
"prefix": "/"
},
"direct_response": {
"status": 404
}
}
]
}
]
},
"http_filters": [
{
"name": "envoy.filters.http.router"
}
]
}
}
]
}
]
}
]
},
"stats_config": {
"stats_tags": [
{
"regex": "^cluster\\.(?:passthrough~)?((?:([^.]+)~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.custom_hash"
},
{
"regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.service_subset"
},
{
"regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.service"
},
{
"regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.namespace"
},
{
"regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.datacenter"
},
{
"regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.routing_type"
},
{
"regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.consul\\.)",
"tag_name": "consul.destination.trust_domain"
},
{
"regex": "^cluster\\.(?:passthrough~)?(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.destination.target"
},
{
"regex": "^cluster\\.(?:passthrough~)?(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+)\\.consul\\.)",
"tag_name": "consul.destination.full_target"
},
{
"regex": "^(?:tcp|http)\\.upstream\\.(([^.]+)(?:\\.[^.]+)?\\.[^.]+\\.)",
"tag_name": "consul.upstream.service"
},
{
"regex": "^(?:tcp|http)\\.upstream\\.([^.]+(?:\\.[^.]+)?\\.([^.]+)\\.)",
"tag_name": "consul.upstream.datacenter"
},
{
"regex": "^(?:tcp|http)\\.upstream\\.([^.]+(?:\\.([^.]+))?\\.[^.]+\\.)",
"tag_name": "consul.upstream.namespace"
},
{
"regex": "^cluster\\.((?:([^.]+)~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.custom_hash"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.service_subset"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.service"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.namespace"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.datacenter"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.consul\\.)",
"tag_name": "consul.routing_type"
},
{
"regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.consul\\.)",
"tag_name": "consul.trust_domain"
},
{
"regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
"tag_name": "consul.target"
},
{
"regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+)\\.consul\\.)",
"tag_name": "consul.full_target"
},
{
"tag_name": "local_cluster",
"fixed_value": "redacted"
},
{
"tag_name": "consul.source.service",
"fixed_value": "redacted"
},
{
"tag_name": "consul.source.namespace",
"fixed_value": "default"
},
{
"tag_name": "consul.source.datacenter",
"fixed_value": "redacted"
}
],
"use_all_default_tags": true
},
"dynamic_resources": {
"lds_config": {
"ads": {},
"resource_api_version": "V3"
},
"cds_config": {
"ads": {},
"resource_api_version": "V3"
},
"ads_config": {
"api_type": "DELTA_GRPC",
"transport_api_version": "V3",
"grpc_services": {
"initial_metadata": [
{
"key": "x-consul-token",
"value": "redacted"
}
],
"envoy_grpc": {
"cluster_name": "local_agent"
}
}
}
}
}envoy logsThe main thing I notice in the envoy logs is that it only sets up the prometheus listener and the local admin admin listener, though I could definitely be missing something. |
jkirschner-hashicorp
removed
the
waiting-reply
|
Hi @dschaaff : Thank you for providing that information. We don't see any obvious issues in what you're provided so far, so we'd need some additional information to explore further:
While we don't expect the following will change anything for you, we think it might be worth a try: you could disable the streaming backend on agents by setting |
|
I disabled streaming on both servers and agents and did not see any change in behavior. I'm attaching several files
Let me know if you need any information. Appreciate it! consul1.10_envoy_config_dump.txt |
|
I should have done this sooner, but I went ahead and tested the behavior with agent versions 1.9.6, 1.9.7, and 1.9.8. Each of those 3 versions presents the same behavior as described in the ticket. No envoy upstream listeners are configured. Once I downgrade to 1.9.5 everything starts working as expected. I've reviewed the changelogs and nothing jumps out at me as a major change. I'd appreciate any guidance on next steps to figure this out. |
|
Ok, I am making progress. Envoy listeners are configured correctly if my sidecar service definition includes an acl token. {
"service": {
"name": "ui-sidecar-proxy",
"id": "ui-sidecar-proxy",
"port": 20000,
"address": "10.20.203.196",
"kind": "connect-proxy",
"token": "redacted",
"proxy": {
"destination_service_id": "ui",
"destination_service_name": "ui",
"local_service_address": "127.0.0.1",
"local_service_port": 80,
"mode": "direct",
"upstreams": [
{
"destination_name": "upstream-1",
"destination_type": "service",
"local_bind_port": 2000
},
{
"destination_name": "upstream-2",
"destination_type": "service",
"local_bind_port": 3000
}
]
},
"meta": {
"availability_zone": "us-west-2a"
},
"checks": [
{
"deregister_critical_service_after": "10m",
"interval": "10s",
"name": "Proxy Public Listener",
"tcp": "10.20.203.196:20000"
},
{
"alias_service": "ui",
"name": "Destination Alias"
}
],
}
}
The clue was the RPC permission denied errors in the log. We previously did not include a token in the sidecar service definition. The service was registered using the consul agent's default token which had See the systemd unit file in my original comment for where that token is passed. This setup worked until agent version 1.9.6. Starting with that version I have to have the token in the sidecar service registration for envoy to be configured correctly. This is also the reason are k8s injected pods were not configuring envoy correctly. The consul-k8s service template does not include the token field https://github.com/hashicorp/consul-k8s/blob/v0.25.0/connect-inject/container_init.go#L320 Is this expected behavior or a bug? |
|
this behavior possibly changed due to #10188 |
|
Thanks for tracking that down! It does seem likely that #10188 is related based on what you found. To help debug the problem, can you tell me more about which of these tokens (https://www.consul.io/docs/security/acl/acl-system#configuring-acls) you have configured? You mentioned the That PR (and #9683 which proceeded it) should only have changed the behaviour for de-registering services, but it sounds like that is not the case. If you have the full error text of the RPC permission denied errors, that would also be helpful to debug. |
|
This is my acl config on the ec2 based nodes "acl": {
"default_policy": "deny",
"down_policy": "extend-cache",
"enable_token_persistence": true,
"enabled": true,
"token_ttl": "30s",
"tokens": {
"agent": "redacted",
"default": "redacted"
}
},
node_prefix "" {
policy = "write"
}
service_prefix "" {
policy = "read"
}
service_prefix "" {
policy = "write"
}
service "" {
policy = "write"
}I can successfully register the service and the proxy sidecar with the local agent without specifying a token in the sidecar service definition. The problem is when I bootstrap the envoy config and then start envoy. At that point, the logs are flooded with (see #10714 (comment) for full log file) Envoy is configured with it's own acl token with the policy. I have confirmed this token is correctly set in the envoy bootstrap config If the token that envoy is using is added to the sidecar service definition then the permission errors go away and envoy is configured as expected. |
This was referenced Aug 10, 2021
This was referenced Aug 20, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Overview of the Issue
I am in the process of testing the upgrade of our Consul Connect setup from 1.9.5 to 1.10.1. After upgrading the Consul agent to 1.10.1 none fo the expected listeners are added to envoy breaking service mesh communication.
Reproduction Steps
Steps to reproduce this issue, eg:
Register connect services. Here is my service config.
{ "service": { "name": "ui", "id": "ui", "port": 80, "address": "10.20.203.196", } }{ "service": { "name": "ui-sidecar-proxy", "id": "ui-sidecar-proxy", "port": 20000, "address": "10.20.203.196", "kind": "connect-proxy", "proxy": { "destination_service_id": "ui", "destination_service_name": "ui", "local_service_address": "127.0.0.1", "local_service_port": 80, "mode": "direct", "upstreams": [ { "destination_name": "upstream-1", "destination_type": "service", "local_bind_port": 2000 }, { "destination_name": "upstream-2", "destination_type": "service", "local_bind_port": 3000 } ] }, "meta": { "availability_zone": "us-west-2a" }, "checks": [ { "deregister_critical_service_after": "10m", "interval": "10s", "name": "Proxy Public Listener", "tcp": "10.20.203.196:20000" }, { "alias_service": "ui", "name": "Destination Alias" } ], } }In my case, the upstreams are registered on the nodes containing those services.
I then bootstrap the envoy config with the
consul connect envoy -envoy-version=1.16.4 -proxy-id=ui-sidecar-proxy -bootstrapon the node. Envoy is then started by a system service. With the 1.9 consul agent envoy is correctly configured with the expected listenersNext, upgrade the Consul agent to 1.10. After upgrading the agent run the
consul connect envoyboostrap command again and then restart envoy with the new bootstrap config. No envoy listeners are created apart from the local admin and prometheus endpointsAt this point no traffic can be sent over the service mesh.
Consul info for both Client and Server
Client info
Server info
Operating system and Environment details
This affects both our Amazon Linux 2 VMs as well as the sidecars injected by consul-k8s v0.25.0 in our Kubernetes clusters.
I'm happy to provide any additional info needed.
The text was updated successfully, but these errors were encountered: