redis支持使用aof来进行持久化,防止数据丢失,aof的刷新策略通过参数appendfsync控制,有三个值:always、everysec、no,默认是everysec。
下面从源码的角度剖析一下aof的刷新策略。
每次redis进入event循环准备执行这个event时,会调用beforeSleep方法
void aeMain(aeEventLoop *eventLoop) {
eventLoop->stop = 0;
while (!eventLoop->stop) {
if (eventLoop->beforesleep != NULL)
eventLoop->beforesleep(eventLoop);
aeProcessEvents(eventLoop, AE_ALL_EVENTS);
}
}
/* This function gets called every time Redis is entering the
* main loop of the event driven library, that is, before to sleep
* for ready file descriptors. */
void beforeSleep(struct aeEventLoop *eventLoop) {
......
/* Write the AOF buffer on disk */
flushAppendOnlyFile(0);
......
}
上面的代码中的flushAppendOnlyFile(int force)进行实际的执行。
src/aof.c
void flushAppendOnlyFile(int force) {
......
/* Perform the fsync if needed. */
if (server.aof_fsync == AOF_FSYNC_ALWAYS) {
/* aof_fsync is defined as fdatasync() for Linux in order to avoid
* flushing metadata. */
latencyStartMonitor(latency);
aof_fsync(server.aof_fd); /* Let's try to get this data on the disk */
latencyEndMonitor(latency);
latencyAddSampleIfNeeded("aof-fsync-always",latency);
server.aof_last_fsync = server.unixtime;
} else if ((server.aof_fsync == AOF_FSYNC_EVERYSEC &&
server.unixtime > server.aof_last_fsync)) {
if (!sync_in_progress) aof_background_fsync(server.aof_fd);
server.aof_last_fsync = server.unixtime;
}
}
AOF_FSYNC_ALWAYS会调用aof_fsync进行同步写入,而aof_fsync在linux下就是fdatasync,
AOF_FSYNC_EVERYSEC会调用aof_background_fsync,而aof_background_fsync会创建一个任务交给后台的bio线程进行处理。
/* Define aof_fsync to fdatasync() in Linux and fsync() for all the rest */
#ifdef __linux__
#define aof_fsync fdatasync
#else
#define aof_fsync fsync
#endif
/* Starts a background task that performs fsync() against the specified
* file descriptor (the one of the AOF file) in another thread. */
void aof_background_fsync(int fd) {
bioCreateBackgroundJob(REDIS_BIO_AOF_FSYNC,(void*)(long)fd,NULL,NULL);
}
其中everysec是通过下面的逻辑来进行的,检测后台是否fsync任务在进行,如果有的话,判断上次的fsync距离现在的时间,如果大于2s,则阻塞,否则直接进行后台队列。
如果上一次的fsync执行了2s多,则会阻塞执行,直到写入成功,这个时候日志中会记录下面一条记录,并且增加info中对应的aof_delayed_fsync值 [5750] 12 Aug 09:56:17.057 * Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down Redis.详细逻辑如下:
/*
* When the fsync policy is set to 'everysec' we may delay the flush if there
* is still an fsync() going on in the background thread, since for instance
* on Linux write(2) will be blocked by the background fsync anyway.
* When this happens we remember that there is some aof buffer to be
* flushed ASAP, and will try to do that in the serverCron() function.
*
* However if force is set to 1 we'll write regardless of the background
* fsync.
*
* 但是如果上一次的fsync执行了2s多,则会阻塞执行,直到写入成功
*/
/* With this append fsync policy we do background fsyncing.
* If the fsync is still in progress we can try to delay
* the write for a couple of seconds. */
if (sync_in_progress) {
if (server.aof_flush_postponed_start == 0) {
/* No previous write postponinig, remember that we are
* postponing the flush and return. */
server.aof_flush_postponed_start = server.unixtime;
return;
} else if (server.unixtime - server.aof_flush_postponed_start < 2) {
/* We were already waiting for fsync to finish, but for less
* than two seconds this is still ok. Postpone again. */
return;
}
/* Otherwise fall trough, and go write since we can't wait
* over two seconds. */
aof_pending_bio_fsync
/* Return the number of pending jobs of the specified type. */
bioPendingJobsOfType(REDIS_BIO_AOF_FSYNC),
serverCron中检查server.aof_flush_postponed_start,如果有的话,就追加一次flush,但是只有在上面的情况下会导致阻塞,其他情况下都会很快返回;
/* AOF postponed flush: Try at every cron cycle if the slow fsync
* completed. */
if (server.aof_flush_postponed_start) flushAppendOnlyFile(0);
分享到:
相关推荐
Redis AOF持久化.flv
redis配置文件aof持久化方式,修改了redis密码为123456
redis-analyzer解析rdb, aof, 以及执行monitor, 来查找key和分析各种top-key(big key, hot-key, expiry-key, slowlog-key)安装go get github.com/lanfang/redis-analyzer如何使用执行redis-analyzer来查看帮助信息,...
RDB持久化是指在指定的时间间隔内将内存中的数据集快照写入磁盘。 也是默认的持久化方式,这种方式是就是将内存中数据以快照的方式写入到二进制文件中,默认的文件名为dump.rdb。
redis-check-aof.exe-
springboot整和jwt、shiro、redis实现token自动刷新
Redis内存过期策略,Redis的内存回收机制,相关配置项
Redis 有两种持久化方案,RDB (Redis DataBase)和 AOF (Append Only File)。如果你想快速了解和使用RDB和AOF,可以直接跳到文章底部看总结。本章节通过配置文件,触发快照的方式,恢复数据的操作,命令操作演示...
Redis持久化策略详解
redis-淘汰策略
redis源码阅读中文分析注释
基于springboot+echarts+redis开发的数据分析及可视化系统源码-毕业设计作品.zip基于springboot+echarts+redis开发的数据分析及可视化系统源码-毕业设计作品.zip基于springboot+echarts+redis开发的数据分析及可视化...
redis源码级别的分析。需要具备c语言基础的开发学习,十分详细,带你全程深入了解redis。
https://mp.csdn.net/console/uploadResources?spm=1011.2124.3001.4171
但是由于数据量的不断增大,单机的Redis物理内存远远无法满足大数据的需要,因此需要搭建分布式的Redis,可以动态扩展内存,弥补单机Redis物理内存不够的缺点。本次测试旨在对Redis各方面性能有深入的了解,为今后的...
redis缓存解决方案和基本命令,redisredis内存存储结构分析.Redis的数据全部放在内存带来了高速的性能,但是也带来一些不合理之处。比如一个中型网站有100万注册用户,如果这些资料要用Redis来存储,内存的容量必须...
redis_check_aof.c 用于更新日志检查的实现。 redis_check_dump.c 用于本地数据库检查的实现。 testhelp.c 一个C风格的小型测试框架。 struct:(结构体) adlist.c 用于对list的定义,它是个双向链表结构 dict.c ...
#资源达人分享计划#
Redis从入门到精通高清,迅雷播放器组件可顺利播放
面试官:Redis 过期删除策略和内存淘汰策略有什么区别?.doc