Consul Connect is not cleared on reload of agent

[pierresouchay]

When filing a bug, please include the following headings if possible. Any example text in this template can be deleted.

Overview of the Issue

If you define a Consul Service in a configuration file /etc/consul/conf.d/my-service.json containing a connect proxy, the service my-service is created as well as my-service-proxy, but if you delete file /etc/consul/conf.d/my-service.json and reload the agent. The service my-service-proxy is not cleared. Restarting fully the agent works however

Reproduction Steps

Steps to reproduce this issue, eg:

1. Create a cluster with n client nodes n and n server nodes
2. Run curl ...
3. View error

Consul info for both Client and Server

Client info

consul info
agent:
	check_monitors = 0
	check_ttls = 0
	checks = 3
	services = 5
build:
	prerelease = criteo4
	revision =
	version = 1.3.1
consul:
	known_servers = 3
	server = false
runtime:
	arch = amd64
	cpu_count = 32
	goroutines = 64
	max_procs = 2
	os = linux
	version = go1.11.1
serf_lan:
	coordinate_resets = 0
	encrypted = true
	event_queue = 0
	event_time = 25111
	failed = 0
	health_score = 0
	intent_queue = 0
	left = 0
	member_time = 1507648
	members = 1065
	query_queue = 0
	query_time = 2056

Server info

agent:
	check_monitors = 0
	check_ttls = 0
	checks = 2
	services = 4
build:
	prerelease = criteo3
	revision =
	version = 1.4.2
consul:
	acl = enabled
	bootstrap = false
	known_datacenters = 3
	leader = false
	leader_addr = 10.236.122.27:8300
	server = true
raft:
	applied_index = 1030070460
	commit_index = 1030070460
	fsm_pending = 0
	last_contact = 87.092964ms
	last_log_index = 1030070460
	last_log_term = 29
	last_snapshot_index = 1030069518
	last_snapshot_term = 29
	latest_configuration = [{Suffrage:Voter ID:40c2e723-8ba6-8ad2-4b22-65cbcf9f65dd Address:10.236.123.11:8300} {Suffrage:Voter ID:a87df596-a0ed-dad6-6a9b-ed6e69884166 Address:10.236.122.27:8300} {Suffrage:Voter ID:512d1a79-fa38-7e74-db10-0c8ceeb263dd Address:10.236.100.11:8300}]
	latest_configuration_index = 1023317041
	num_peers = 2
	protocol_version = 3
	protocol_version_max = 3
	protocol_version_min = 0
	snapshot_version_max = 1
	snapshot_version_min = 0
	state = Follower
	term = 29
runtime:
	arch = amd64
	cpu_count = 16
	goroutines = 22120
	max_procs = 15
	os = linux
	version = go1.11.4
serf_lan:
	coordinate_resets = 0
	encrypted = true
	event_queue = 0
	event_time = 25111
	failed = 4
	health_score = 0
	intent_queue = 0
	left = 4
	member_time = 1507648
	members = 1073
	query_queue = 0
	query_time = 2056
serf_wan:
	coordinate_resets = 0
	encrypted = true
	event_queue = 0
	event_time = 1
	failed = 0
	health_score = 0
	intent_queue = 0
	left = 0
	member_time = 68985
	members = 11
	query_queue = 0
	query_time = 1

Operating system and Environment details

Note: Those are patched versions of Consul

[pierresouchay]

This bug is not clear to me, it seems that even a restart can trigger it sometimes.

I also wonder whether it could be solved by #5217
I'll try to investigate this further