Consul members not shown for a master token user

[VladV]

consul version for both Client and Server

Client & Server: Consul v0.8.3 Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

consul info for both Client and Server

Client & Server:

agent:
        check_monitors = 0
        check_ttls = 0
        checks = 0
        services = 1
build:
        prerelease =
        revision = ea2a82b
        version = 0.8.3
consul:
        bootstrap = true
        known_datacenters = 1
        leader = true
        leader_addr = 192.168.99.100:8300
        server = true
raft:
        applied_index = 524
        commit_index = 524
        fsm_pending = 0
        last_contact = 0
        last_log_index = 524
        last_log_term = 2
        last_snapshot_index = 0
        last_snapshot_term = 0
        latest_configuration = [{Suffrage:Voter ID:4e2e326f-3655-9ee1-cfaa-6e87b1c9a5c0 Address:192.168.99.100:8300}]
        latest_configuration_index = 1
        num_peers = 0
        protocol_version = 3
        protocol_version_max = 3
        protocol_version_min = 0
        snapshot_version_max = 1
        snapshot_version_min = 0
        state = Leader
        term = 2
runtime:
        arch = amd64
        cpu_count = 2
        goroutines = 76
        max_procs = 2
        os = linux
        version = go1.8.1
serf_lan:
        encrypted = true
        event_queue = 0
        event_time = 2
        failed = 0
        health_score = 0
        intent_queue = 0
        left = 0
        member_time = 5
        members = 2
        query_queue = 0
        query_time = 2
serf_wan:
        encrypted = true
        event_queue = 0
        event_time = 1
        failed = 0
        health_score = 0
        intent_queue = 0
        left = 0
        member_time = 1
        members = 1
        query_queue = 1
        query_time = 2

Operating system and Environment details

Linux.
A fresh install of 0.8.3 in Docker (FROM consul:0.8.3 plus some customized config).

Description of the Issue (and unexpected/desired result)

consul members output is empty if used with master token. Without any token or with another valid token it shows cluster members as expected.

API call /v1/agent/members behaves in the same way.

Reproduction steps

/ # consul members
Node                Address               Status  Type    Build  Protocol  DC
agent@docker.local  192.168.99.100:18301  alive   client  0.8.3  3         dc1
default             192.168.99.100:8301   alive   server  0.8.3  3         dc1
/ # consul members -token=$CONSUL_MASTER_TOKEN
/ # consul members -token=$CONSUL_CLIENT_TOKEN
Node                Address               Status  Type    Build  Protocol  DC
agent@docker.local  192.168.99.100:18301  alive   client  0.8.3  3         dc1
default             192.168.99.100:8301   alive   server  0.8.3  3         dc1

/ # curl http://[::1]:9500/v1/agent/members
[{"Name":"default","Addr":"192.168.99.100","Port":8301,"Tags":{"bootstrap":"1","build":"0.8.3:ea2a82b","dc":"dc1","id":"4e2e326f-3655-9ee1-cfaa-6e87b1c9a5c0","port":"8300","raft_vsn":"3","role":"consul","vsn":"3","vsn_max":"3","vsn_min":"2","wan_join_port":"8302"},"Status":1,"ProtocolMin":1,"ProtocolMax":5,"ProtocolCur":2,"DelegateMin":2,"DelegateMax":5,"DelegateCur":4},{"Name":"agent@docker.local","Addr":"192.168.99.100","Port":18301,"Tags":{"build":"0.8.3:ea2a82b","dc":"dc1","id":"1565436c-84e1-ae56-5654-617e6ebcaaf3","role":"node","vsn":"3","vsn_max":"3","vsn_min":"2"},"Status":1,"ProtocolMin":1,"ProtocolMax":5,"ProtocolCur":2,"DelegateMin":2,"DelegateMax":5,"DelegateCur":4}]/
/ # curl http://[::1]:9500/v1/agent/members?token=$CONSUL_MASTER_TOKEN
[]

ACL is configured as following (with CONSUL_MASTER_TOKEN/CONSUL_CLIENT_TOKEN replaced with actual values):

acl.json
{
  "acl_datacenter": "dc1",
  "acl_default_policy": "deny",
  "acl_ttl": "300s",
  "acl_master_token": "$CONSUL_MASTER_TOKEN",
  "acl_agent_master_token": "$CONSUL_MASTER_TOKEN",
  "acl_agent_token": "$CONSUL_CLIENT_TOKEN"
} 

/v1/acl/list
[{
		"ID" : "anonymous",
		"Name" : "",
		"Type" : "client",
		"Rules" : " key \"\" { policy = \"read\" } service \"\" { policy = \"read\" } agent \"\" { policy = \"read\" } node \"\" { policy = \"read\" } session \"\" { policy = \"read\" }
		 ",
		"CreateIndex" : 4,
		"ModifyIndex" : 8
	}, {
		"ID" : "$CONSUL_CLIENT_TOKEN",
		"Name" : "default client",
		"Type" : "client",
		"Rules" : " key \"\" { policy = \"write\" } service \"\" { policy = \"write\" } agent \"\" { policy = \"write\" } node \"\" { policy = \"write\" } session \"\" { policy = \"writ
		e\" } ",
		"CreateIndex" : 9,
		"ModifyIndex" : 9
	}, {
		"ID" : "$CONSUL_MASTER_TOKEN",
		"Name" : "Master Token",
		"Type" : "management",
		"Rules" : "",
		"CreateIndex" : 5,
		"ModifyIndex" : 5
	}
]

[VladV]

The problem seems to do with acl_agent_master_token setting - if I remove it, everything starts to work as expected.

Also if I configure different values for acl_master_token and acl_agent_master_token, the problem manifests for agent master token only.

[jhmartin]

Occurring in Consul 0.8.5 as well.
Debug log is:
2017/07/13 19:40:29 [DEBUG] agent: dropping node "[SNIP]" from result due to ACLs
Input is:

export CONSUL_HTTP_TOKEN=the-mgmt-token-also-agent-master-token
consul members
[ no output emitted ]