Redis持久化——RDB、AOF

Redis持久化——RDB、AOF

什么是持久化

redis所有数据都保存在内存中，对数据的更新异步保存到磁盘上。
在Redis中持久化的方式有两种，一种是快照持久化，一种是AOF持久化，各有各的优缺点，在项目中我们得根据实际的情况来选择具体的持久化方式。

快照持久化（RDB）

也叫RDB持久化方式。就是通过拍摄快照的方式来实现持久化，将某个时间的内存数据存储在一个rdb文件中。在redis服务重新启动的时候会加载rdb文件中的数据。
通过RDB方式实现持久化，一旦Redis异常退出，就会丢失最后一次快照以后更改的所有数据。这就需要开发者根据具体的应用场合，通过组合设置自动快照条件的方式来将可能发生的数据损失控制在能够接受的范围。例如，使用Redis存储缓存数据时，丢失最近几秒的数据或者丢失最近更新的几十个键并不会有很大的影响。如果数据相对重要，希望将损失降到最小，则可以使用AOF方式进行持久化。

配置快照持久化

先来看一下redis.conf中关于持久化以及RDB的内容：

################################ SNAPSHOTTING  ################################
#
# Save the DB on disk:
#
#   save <seconds> <changes>
#
#   Will save the DB if both the given number of seconds and the given
#   number of write operations against the DB occurred.
#
#   In the example below the behaviour will be to save:
#   after 900 sec (15 min) if at least 1 key changed
#   after 300 sec (5 min) if at least 10 keys changed
#   after 60 sec if at least 10000 keys changed
#
#   Note: you can disable saving completely by commenting out all "save" lines.
#
#   It is also possible to remove all the previously configured save
#   points by adding a save directive with a single empty string argument
#   like in the following example:
#
#   save ""

save 900 1   #900秒内至少有1个key被更改就执行快照
save 300 10  #300内描述至少有10个key被更改就执行快照
save 60 10000  #60秒内至少有10000个key被更改就执行快照

# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
stop-writes-on-bgsave-error yes #拍摄快照失败是否继续执行写命令

# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes #是否对快照文件进行压缩

# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
#
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
rdbchecksum yes #是否进行数据校验

# The filename where to dump the DB
dbfilename dump.rdb #快照文件存储的名称
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir ./ #快照文件存储的位置

阅读一下注释，配置文件已经告诉我们如何禁用RDB

1. 注释掉所有的save配置
2. 追加save ""配置

save自动配置满足任意配置就执行。在生产环境下60 10000是很容易达成的，频繁地进行save会消耗cpu、内存，一般都会对自动保存进行修改。
所有的配置都是可以修改的，修改后以配置文件的方式启动服务即可！

RDB命令

1. save命令

在redis运行中，我们可以显示的发送一条save命令来拍摄快照。save命令是阻塞命令，也就是当服务器接收了一条save命令之后就会开始拍摄快照，在此期间不会再去处理其他的请求，其他请求会被挂起直到备份结束。如果存在老的RDB文件，会用新生成的RDB文件（临时文件）替换老的文件。

由于save命令是阻塞的，在生产环境，大数据量的情况下，是不适合使用的！

bgsave命令

bgsave命令也是立即拍摄快照，有别于save命令，bgsave并不是一条阻塞命令，而是fork一个子线程，然后这个子线程负责备份操作。而父进程继续处理客户端的请求，这样就不会造成阻塞了。但如果fork是存在阻塞的，只是fork通常的时间都非常短。如果系统设计有问题，也会造成长时间的阻塞。

127.0.0.1:6379> bgsave
Background saving started

1. Redis使用fork函数复制一份当前进程（父进程）的副本（子进程）；
2. 父进程继续接收并处理客户端发来的命令，而子进程开始将内存中的数据写入硬盘中的临时文件；
3. 当子进程写入完所有数据后会用该临时文件替换旧的 RDB 文件，至此一次快照操作完成。

1. 在执行 fork 的时候操作系统（类 Unix 操作系统）会使用写时复制（copy-on-write）策略，即fork函数发生的一刻父子进程共享同一内存数据，当父进程要更改其中某片数据时（如执行一个写命令），操作系统会将该片数据复制一份以保证子进程的数据不受影响，所以新的RDB文件存储的是执行fork一刻的内存数据。
2. 另外需要注意的是，当进行快照的过程中，如果写入操作较多，造成 fork 前后数据差异较大，是会使得内存使用量显著超过实际数据大小的，因为内存中不仅保存了当前的数据库数据，而且还保存着 fork 时刻的内存数据。进行内存用量估算时很容易忽略这一问题，造成内存用量超限。

save bgsave对比

不容忽视的触发机制

• 主从复制（全量复制）
• debug reload
• shutdown

都会主动触发RDB持久化。

AOF持久化

AOF( append only file )持久化以独立日志的方式记录每次写命令，并在 Redis 重启时在重新执行 AOF 文件中的命令以达到恢复数据的目的。AOF 的主要作用是解决数据持久化的实时性。

AOF的三种策略

redis执行aof操作不是直接写入磁盘的aof文件当中，而是先写入到磁盘的缓冲区，再通过fsync到磁盘中去。
这是操作系统层面，提供了缓冲区，为了提高数据存储的性能，如果大量数据直接落到磁盘，必定大大增加磁盘负担，影响IO性能。

• always
  
  每条命令都会执行一次fsync
• everysec
  
  每秒执行一次fsync
• no
  
  完全交给操作系统管理，无法预知什么时候执行fsync

三种策略的优缺点

AOF重写

因为 AOF 持久化是通过保存被执行的写命令来记录 Redis 状态的，所以随着 Redis 长时间运行，AOF 文件中的内容会越来越多，文件的体积也会越来越大，如果不加以控制的话，体积过大的 AOF 文件很可能对 Redis 甚至宿主计算机造成影响。

为了解决 AOF 文件体积膨胀的问题，Redis 提供了 AOF 文件重写( rewrite) 功能。通过该功能，Redis 可以创建一个新的 AOF 文件来替代现有的 AOF 文件。新旧两个 AOF 文件所保存的 Redis 状态相同，但是新的 AOF 文件不会包含任何浪费空间的命令，所以新 AOF 文件的体积通常比旧 AOF 文件的体积要小得很多。

AOF重写的作用

• 减少硬盘占用量
• 加速恢复速度

AOF重写流程

执行bgrewriteaof命令后，父进程会fork出一个子进程，子进程会通知父进程aof开始，子进程会回溯父进程的内存（共享内存），生成一个新的AOF文件。
这个时候主进程是正常运作的，如果执行了新的命令，主进程会将命令写入到aof_buf以及aof_rewrite_buf中，子进程将aof_rewrite_buf中的数据追加到新的AOF文件中。同时aof_buf也将写入到旧的AOF文件中。最后新的AOF文件替换旧的AOF文件，AOF重写成功。如果失败，旧的AOF文件也保持最新的状态，不会造成数据的丢失。

AOF重写实现的两种方式

1. bgrewriteaof
2. AOF重写配置

bgrewriteaof命令

AOF重写配置

# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.

appendonly no #开启aof

# The name of the append only file (default: "appendonly.aof")

appendfilename "appendonly.aof" #保存的aof文件名称

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".

# appendfsync always  #aof的三种文件策略
appendfsync everysec
# appendfsync no

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.

no-appendfsync-on-rewrite no #aof重写的时候不执行append操作

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

# aof自动化配置
auto-aof-rewrite-percentage 100 #增长率
auto-aof-rewrite-min-size 64mb  # 最小尺寸

# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes

# When rewriting the AOF file, Redis is able to use an RDB preamble in the
# AOF file for faster rewrites and recoveries. When this option is turned
# on the rewritten AOF file is composed of two different stanzas:
#
#   [RDB file][AOF tail]
#
# When loading Redis recognizes that the AOF file starts with the "REDIS"
# string and loads the prefixed RDB file, and continues loading the AOF
# tail.
aof-use-rdb-preamble yes #开启混合存储

• auto-aof-rewrite-percentage 100
• auto-aof-rewrite-min-size 64mb

只有当aof文件大小的增长率达到配置以及aof文件的大小大于配置的最小值时，才自动执行AOF重写。
no-appendfsync-on-rewrite但进行重写时是否允许对aof文件进行新命令的写入磁盘（已写入缓冲区），两个操作同时进行会消耗更多的资源，耗费性能。但如果不允许的话，如果redis突然挂掉，旧文件也没有得到更新，会使得数据丢失。

RDB与AOF区别