Allow service deregistration with node write permission

[ShimmerGlass]

Fixes: #5209

With ACLs enabled if an agent is wiped and restarted without a leave
it can no longer deregister the services it had previously registered
because it no longer has the tokens the services were registered with.
To remedy that we allow service deregistration from tokens with node
write permission.

[pierresouchay]

It make sense because as long as the user has the ACL to leave the cluster (which effectively remove all services of agent), it should have the rights to remove services the agent is "owning". Having the right on the agent being the correct right to leave, it sounds like a reasonable patch to me.

[ShimmerGlass]

We saw this problem again today : several service instances were registered in the catalog while the agent they were registered on before was trying to deregister them without success (since it had no token with permissions to do so) becode the node had been reinstalled.
The only solution we had was to manually deregister each instance via /catalog calls. These instances were green in Consul and revieving traffic.
One other argument toward this change is the fact that service checks can already be deregistered with only node permissions (see https://github.com/hashicorp/consul/blob/master/agent/local/state.go#L1233).

[pierresouchay]

Could someone have a look to this PR?

[ShimmerGlass]

Any update on this ?
This issue is still affecting us with ghost passing service instances.

[hanshasselberg]

Thanks for your work, I left some comments.

[hanshasselberg]

Why did this file change?

[ShimmerGlass]

With the previous behavior this call (write access on node but not on service) would fail. It now succeeds if the token has write access on the node, which is the case here, so I changed the test to target a node the token does not has write access to.

[hanshasselberg]

This looks good! Left a tiny comment, something we just noticed.

If you don't get to it, I will make the change so that we can include your PR in the release that we are hopefully cutting tomorrow.

[hanshasselberg]

Thank you!

[pierresouchay]

@i0rek 🥇 Thank you so much, this will really help on large infrastructures! (That's actually one of our last pain points)

[hanshasselberg]

We are closing the gap! 🙏

[ShimmerGlass]

@i0rek thx (:

[charnet1019]

how to call /catalog to delete instance?
Thanks.