Snapshots are not occurring in consul version 0.8.5 and permissions for snapshot directory is 0755

[hbmelachuru10]

Consul Server : v0.8.5
Consul Client: v0.8.5

Consul Server configuration:
{
"data_dir": "/data",
"client_addr": "%%CLIENT_ADDRESS%%",
"bind_addr": "%%BIND_ADDRESS%%",
"start_join": [
%%LIST_PODIPS%%
],
"bootstrap_expect": 3,
"datacenter" : "%%DATACENTER%%",
"advertise_addr_wan": %%LIST_INGRESS_IP%%,
"translate_wan_addrs": true,
"skip_leave_on_interrupt": false,
"acl_default_policy": "allow",
"acl_enforce_version_8": true,
"acl_master_token": "xxxxxxxxxxxxxxxxxxx",
"acl_datacenter": "%%DATACENTER%%",
"leave_on_terminate": true,
"performance": {
"raft_multiplier": 1
},
"disable_update_check": true,
"log_level": "debug",
"enable_syslog": false,
"telemetry": {
"dogstatsd_addr": "%%TELEMETRY%%",
"dogstatsd_tags": [
"%%DATACENTER%%:%%HOSTNAME%%"
]
}
}

sh-4.2$ cd /data/raft/
sh-4.2$ ls -la
total 64
drwxr-xr-x. .3 root 4096 Jun 29 16:09 .
drwxrwxrwx. 5 root 4096 Jun 29 16:15 ..
-rwxr-xr-x. 1 root 2318 Jun 29 16:09 peers.info
-rw-------. 1 root 65536 Jun 29 17:01 raft.db
drwxr-xr-x. 2 root 4096 Jun 29 16:09 snapshots
sh-4.2$ cd snapshots/
sh-4.2$ ls -la
total 8
drwxr-xr-x. 2 root 4096 Jun 29 16:09 .
drwxr-xr-x. 3 root 4096 Jun 29 16:09 ..

sh-4.2$ consul version
Consul v0.8.5
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

The defaults snapshots were occurring in the old version ( 0.7.4) but when i have upgraded the version the snapshots are not occurring and also the snapshot directory permission is 0755. But in consul changelog, the permission has changed to 0644 from version 0.8.0 ( agent: Serf snapshots no longer have the executable bit set on the file. [GH-2722] ). Am I missing something?

1. How is default snapshot happening in the consul?
2. How frequent the snapshots occurs, any time-frame?
3. If the executable files in the snapshot doesn't exist (as per the changelog), how can the consul server recovers when there is a failure?
4. New ACL enforce policies are little confusing, a better documentation would be recommended

Appreciate your help and time

[slackpad]

Hi @hbmelachuru10 that issue was related to Serf snapshots, which are totally different from the Raft ones. The Raft ones are dependent on the write load on the cluster, so if you have a quiet cluster it could take a while before one gets kicked out. The easiest way to check things is to manually take a snapshot via https://www.consul.io/docs/commands/snapshot.html (or the API). This uses the exact same mechanism under the hood as the regular Raft snapshots, so will vet if there are any issues with writing those to disk. I don't know of any issues like that going from 0.7.x to 0.8.x, so I'd be surprised if there was an issue there.

Totally agree on the ACL documentation. We are working on improving https://www.consul.io/docs/guides/acl.html by the next release of Consul with clearer explanations and more examples!

[hbmelachuru10]

Hi @slackpad . That you for clarifying the snapshot issue. I tried the consul snapshot API, it worked as expected. Appreciate your quick response.

[slackpad]

Cool - I'll close this out but let us know if you have any other issues.