Test new persistence mechanism and PITR

• Persistent data in the new mechanism is composed of one rdb file and multiple AOF binlogs. AOF binlog is rotated automatically and the size of a single AOF binlog is controlled by aof-max-size config. AOF rewrite is no longer needed. appendonly config need to be yes.

• The data directory looks like this:

-rw-r--r-- 1 tony tony  210 12月 21 22:50 aof-inc.index
-rw-r--r-- 1 tony tony    0 12月 20 20:00 appendonly-inc-1513771258.aof
-rw-r--r-- 1 tony tony  16M 12月 20 20:03 appendonly-inc-1513771327.aof
-rw-r--r-- 1 tony tony    0 12月 20 20:21 appendonly-inc-1513772518.aof
-rw-r--r-- 1 tony tony 101M 12月 20 20:25 appendonly-inc-1513772519.aof
-rw-r--r-- 1 tony tony 101M 12月 20 20:27 appendonly-inc-1513772749.aof
-rw-r--r-- 1 tony tony 101M 12月 21 22:50 appendonly-inc-1513772858.aof
-rw-r--r-- 1 tony tony  28M 12月 22 12:26 appendonly-inc-1513867824.aof
-rw-rw-r-- 1 tony tony 3.3M 12月 20 20:21 dump111.rdb
-rw-rw-r-- 1 tony tony   52 12月 20 20:21 rdb.index

We maintain two index files aof-inc.index and rdb.index to record some information.

aof-inc.index maintains the names of all the AOF binlog currently owned by the instance.

$ cat aof-inc.index 
appendonly-inc-1513771258.aof
appendonly-inc-1513771327.aof
appendonly-inc-1513772518.aof
appendonly-inc-1513772519.aof
appendonly-inc-1513772749.aof
appendonly-inc-1513772858.aof
appendonly-inc-1513867824.aof

rdb.index holds the mapping between current rdb and the file offset of some AOF binlog.

$  cat rdb.index 
dump111.rdb appendonly-inc-1513772519.aof 0 160509

• Test point in time recovery(PITR):
  • Start an ApsaraCache instance.

    config set appendonly yes

  • Execute bgsave command, then check the rdb.index file that the new RDB file name is dump.rdb and the corresponding AOF binlog is x1.aof.
  • Write some data, then record the current timestamp t0 and the current written AOF binlog xn.aof.
  • After we have made some changes to the data, we want to restore the data status to t0.
  • Copy dump.rdb, rdb.index, x1.aof to xn.aof to the data directory of another ApsaraCache instance Tmp_Ins. Create a new aof-inc.index file which contains the names of x1.aof to xn.aof in the data directory.
  • Create a restore.timestamp file which contains the target timestamp t0 in the data directory of Tmp_ins.
  • Start Tmp_Ins and it will automatically restore the data status to t0.
  • On success, we can see logs similar to the following,

7656] 22 Jun 15:04:16.582 * restored data to timestamp 1498115056 in aof appendonly-inc-1498115050.aof