數據存儲我們在應用設計過程中非常重要的一部分,無論是關系型數據庫,還是Redis、MongoDB等非關系型數據庫,都有很多的高可用方案,還有一些針對不同業務設計的中間件,使其性能更有特色,更能保證數據存儲的穩定和安全。
目前主流的Redis的數據存儲架構有Redis單點,Redis主從,基于Sentinel的Redis主備、基于keepalive的redis主備,以及Redis集群Cluster,還有豌豆莢開源的Codis等是目前業內比較流行的解決方案,不同的存儲架構,是若干個技術工程師,根據自己公司的業務,踩過N次坑,總結出的結果,沒有最好的解決方案,只有最適合的解決方案。
在上家公司出,采用了redis的主從架構,版本是2.8,上線一年多,尚未出現過問題,但是潛在的隱患是大家都知道的,主從架構中,當主宕機或者意外情況時,主與從不能平滑切換,將會影響業務,所以這里就是一個×××,而Sentinel(哨兵)方案是主從架構的升級版,能夠讓主從之間自動切換。提高系統穩定性和高可用性。以下是最近自己測試哨兵架構的相關記錄。
Sentinel官方文檔:https://redis.io/topics/sentinel
操作系統:CentOS 6.7
redis版本:3.2.9
主機規劃:
序號 | Host | Redis Role | Redis Port | Sentinel Port |
1 | 10.1.4.82 | slave | 6379 | 26379 26380 |
2 | 10.1.4.99 | master | 6379 | 26379 26380 |
整體采用,主從結構,每臺主機再部署2個哨兵,整體是2個Redis 4個Sentinel集群的架構。
步驟:
1、配置Redis主從
由于配置redis主從比較簡單,所以在這里不會去詳細的介紹,想了解如何配置的朋友,請參考以下文章。
http://dreamlinux.blog.51cto.com/9079323/1832029
http://dreamlinux.blog.51cto.com/9079323/1832069
2、檢驗redis主從是否正常
登錄redis的Master主節點檢查redis主從
127.0.0.1:6379> info 127.0.0.1:6379> info # Server redis_version:3.2.9 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:66cf707c3c51d9f redis_mode:standalone os:Linux 2.6.32-696.6.3.el6.x86_64 x86_64 arch_bits:64 multiplexing_api:epoll gcc_version:4.4.7 process_id:26318 run_id:7954f18bcb4084964efb58460027399d583fba57 tcp_port:6379 uptime_in_seconds:14402 uptime_in_days:0 hz:10 lru_clock:9455690 executable:/usr/local/bin/redis-server config_file:/etc/redis/6379.conf # Clients connected_clients:7 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0 # Memory used_memory:2017176 used_memory_human:1.92M used_memory_rss:9969664 used_memory_rss_human:9.51M used_memory_peak:2098064 used_memory_peak_human:2.00M total_system_memory:2103533568 total_system_memory_human:1.96G used_memory_lua:37888 used_memory_lua_human:37.00K maxmemory:0 maxmemory_human:0B maxmemory_policy:noeviction mem_fragmentation_ratio:4.94 mem_allocator:jemalloc-4.0.3 # Persistence loading:0 rdb_changes_since_last_save:0 rdb_bgsave_in_progress:0 rdb_last_save_time:1502627793 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:0 rdb_current_bgsave_time_sec:-1 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok # Stats total_connections_received:20 total_commands_processed:89212 instantaneous_ops_per_sec:6 total_net_input_bytes:6172607 total_net_output_bytes:33781397 instantaneous_input_kbps:0.26 instantaneous_output_kbps:0.67 rejected_connections:0 sync_full:1 sync_partial_ok:0 sync_partial_err:0 expired_keys:0 evicted_keys:0 keyspace_hits:0 keyspace_misses:0 pubsub_channels:1 pubsub_patterns:0 latest_fork_usec:643 migrate_cached_sockets:0 # Replication role:master #當前host的角色 connected_slaves:1 slave0:ip=10.1.4.82,port=6379,state=online,offset=23160,lag=1 #看到從節點的一些信息 master_repl_offset:23160 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:2 repl_backlog_histlen:23159 # CPU used_cpu_sys:10.40 used_cpu_user:6.24 used_cpu_sys_children:0.00 used_cpu_user_children:0.00 # Cluster cluster_enabled:0 # Keyspace db0:keys=5,expires=0,avg_ttl=0
設置相關的key檢查
master
127.0.0.1:6379> dbsize (integer) 5 127.0.0.1:6379> keys * 1) "winnerlook" 2) "redis" 3) "bund" 4) "baidu" 5) "bund33" 127.0.0.1:6379>
slave
127.0.0.1:6379> dbsize (integer) 5 127.0.0.1:6379> keys * 1) "winnerlook" 2) "redis" 3) "bund" 4) "baidu" 5) "bund33" 127.0.0.1:6379>
由此可見配置的redis主從是可以正常工作的,接下來就是準備配置Sentinel,并啟動相關的服務。
10.1.4.99 sentinel 1配置文件
pdaemonize yes #后臺運行 protected-mode no #保護模式關閉 # Generated by CONFIG REWRITE dir "/etc/redis" port 26379 #哨兵端口 sentinel monitor master1 10.1.4.99 6379 2 # 監聽IP 端口 默認最少投票數量 sentinel down-after-milliseconds master1 5000 sentinel failover-timeout master1 5000 #選舉超時時間 logfile "/data/redis/sentinel/sentinel_26379.log" #日志文件 sentinel auth-pass master1 Redis2017 #redis密碼
10.1.4.99 sentinel 2配置文件
pdaemonize yes #后臺運行 protected-mode no #保護模式關閉 # Generated by CONFIG REWRITE dir "/etc/redis" port 26380 #哨兵端口 sentinel monitor master1 10.1.4.99 6379 2 # 監聽IP 端口 默認最少投票數量 sentinel down-after-milliseconds master1 5000 sentinel failover-timeout master1 5000 #選舉超時時間 logfile "/data/redis/sentinel/sentinel_26379.log" #日志文件 sentinel auth-pass master1 Redis2017 #redis密碼
10.1.4.82 sentinel 1配置
pdaemonize yes #后臺運行 protected-mode no #保護模式關閉 # Generated by CONFIG REWRITE dir "/etc/redis" port 26379 #哨兵端口 sentinel monitor master1 10.1.4.99 6379 2 # 監聽IP 端口 默認最少投票數量 sentinel down-after-milliseconds master1 5000 sentinel failover-timeout master1 5000 #選舉超時時間 logfile "/data/redis/sentinel/sentinel_26379.log" #日志文件 sentinel auth-pass master1 Redis2017 #redis密碼
10.1.4.82 sentinel 2配置
pdaemonize yes #后臺運行 protected-mode no #保護模式關閉 # Generated by CONFIG REWRITE dir "/etc/redis" port 26380 #哨兵端口 sentinel monitor master1 10.1.4.99 6379 2 # 監聽IP 端口 默認最少投票數量 sentinel down-after-milliseconds master1 5000 sentinel failover-timeout master1 5000 #選舉超時時間 logfile "/data/redis/sentinel/sentinel_26379.log" #日志文件 sentinel auth-pass master1 Redis2017 #redis密碼
啟動哨兵服務的兩種方式
1、redis-sentinel /path/to/sentinel.conf
2、redis-server /path/to/sentinel.conf --sentinel
啟動10.1.4.99的哨兵服務
redis-sentinel /etc/redis/sentinel_26380.conf redis-sentinel /etc/redis/sentinel_26379.conf
檢查26379端口相關日志
[root@bundcredit sentinel]# tail -f sentinel_26379.log `-._ `-._`-.__.-'_.-' _.-' `-._ `-.__.-' _.-' `-._ _.-' `-.__.-' 27098:X 13 Aug 21:51:25.242 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. 27098:X 13 Aug 21:51:25.242 # Sentinel ID is 361c7d3014a041a1a74163355882d88f5fab7581 27098:X 13 Aug 21:51:25.242 # +monitor master master1 10.1.4.99 6379 quorum 2 27098:X 13 Aug 21:51:30.271 # +sdown sentinel 76d5a26748410e72ca703e255f44a5ed2b3f83e4 10.1.4.99 26380 @ master1 10.1.4.99 6379 27098:X 13 Aug 21:51:30.410 # -sdown sentinel 76d5a26748410e72ca703e255f44a5ed2b3f83e4 10.1.4.99 26380 @ master1 10.1.4.99 6379 27098:X 13 Aug 21:52:30.617 # +sdown sentinel 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac 10.1.4.82 26379 @ master1 10.1.4.99 6379 27098:X 13 Aug 21:53:23.855 # -sdown sentinel 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac 10.1.4.82 26379 @ master1 10.1.4.99 6379 27098:X 13 Aug 21:53:34.463 # +sdown master master1 10.1.4.99 6379 27098:X 13 Aug 21:53:34.532 # +odown master master1 10.1.4.99 6379 #quorum 2/2 27098:X 13 Aug 21:53:34.532 # +new-epoch 1 27098:X 13 Aug 21:53:34.532 # +try-failover master master1 10.1.4.99 6379 27098:X 13 Aug 21:53:34.541 # +vote-for-leader 361c7d3014a041a1a74163355882d88f5fab7581 1 27098:X 13 Aug 21:53:34.542 # b73258f1019f65c5fee3b88b9c2eae7f85d985f3 voted for b73258f1019f65c5fee3b88b9c2eae7f85d985f3 1 27098:X 13 Aug 21:53:34.559 # 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac voted for b73258f1019f65c5fee3b88b9c2eae7f85d985f3 1 27098:X 13 Aug 21:53:34.560 # 76d5a26748410e72ca703e255f44a5ed2b3f83e4 voted for b73258f1019f65c5fee3b88b9c2eae7f85d985f3 1 27098:X 13 Aug 21:53:35.156 # +config-update-from sentinel b73258f1019f65c5fee3b88b9c2eae7f85d985f3 10.1.4.82 26380 @ master1 10.1.4.99 6379 27098:X 13 Aug 21:53:35.156 # +switch-master master1 10.1.4.99 6379 10.1.4.82 6379 27098:X 13 Aug 21:53:35.156 * +slave slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 27098:X 13 Aug 21:53:40.173 # +sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 27098:X 13 Aug 21:54:07.583 # -sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379
檢查26380端口相關日志
[root@bundcredit sentinel]# tail -f sentinel_26379.log `-._ `-._`-.__.-'_.-' _.-' `-._ `-.__.-' _.-' `-._ _.-' `-.__.-' 27098:X 13 Aug 21:51:25.242 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. 27098:X 13 Aug 21:51:25.242 # Sentinel ID is 361c7d3014a041a1a74163355882d88f5fab7581 27098:X 13 Aug 21:51:25.242 # +monitor master master1 10.1.4.99 6379 quorum 2 27098:X 13 Aug 21:51:30.271 # +sdown sentinel 76d5a26748410e72ca703e255f44a5ed2b3f83e4 10.1.4.99 26380 @ master1 10.1.4.99 6379 27098:X 13 Aug 21:51:30.410 # -sdown sentinel 76d5a26748410e72ca703e255f44a5ed2b3f83e4 10.1.4.99 26380 @ master1 10.1.4.99 6379 27098:X 13 Aug 21:52:30.617 # +sdown sentinel 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac 10.1.4.82 26379 @ master1 10.1.4.99 6379 27098:X 13 Aug 21:53:23.855 # -sdown sentinel 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac 10.1.4.82 26379 @ master1 10.1.4.99 6379 27098:X 13 Aug 21:53:34.463 # +sdown master master1 10.1.4.99 6379 27098:X 13 Aug 21:53:34.532 # +odown master master1 10.1.4.99 6379 #quorum 2/2 27098:X 13 Aug 21:53:34.532 # +new-epoch 1 27098:X 13 Aug 21:53:34.532 # +try-failover master master1 10.1.4.99 6379 27098:X 13 Aug 21:53:34.541 # +vote-for-leader 361c7d3014a041a1a74163355882d88f5fab7581 1 27098:X 13 Aug 21:53:34.542 # b73258f1019f65c5fee3b88b9c2eae7f85d985f3 voted for b73258f1019f65c5fee3b88b9c2eae7f85d985f3 1 27098:X 13 Aug 21:53:34.559 # 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac voted for b73258f1019f65c5fee3b88b9c2eae7f85d985f3 1 27098:X 13 Aug 21:53:34.560 # 76d5a26748410e72ca703e255f44a5ed2b3f83e4 voted for b73258f1019f65c5fee3b88b9c2eae7f85d985f3 1 27098:X 13 Aug 21:53:35.156 # +config-update-from sentinel b73258f1019f65c5fee3b88b9c2eae7f85d985f3 10.1.4.82 26380 @ master1 10.1.4.99 6379 27098:X 13 Aug 21:53:35.156 # +switch-master master1 10.1.4.99 6379 10.1.4.82 6379 27098:X 13 Aug 21:53:35.156 * +slave slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 27098:X 13 Aug 21:53:40.173 # +sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 27098:X 13 Aug 21:54:07.583 # -sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 ^C [root@bundcredit sentinel]# tail -f sentinel_26369.log tail: cannot open `sentinel_26369.log' for reading: No such file or directory tail: no files remaining [root@bundcredit sentinel]# tail -f sentinel_26380.log 27102:X 13 Aug 21:53:34.496 # +sdown master master1 10.1.4.99 6379 27102:X 13 Aug 21:53:34.558 # +new-epoch 1 27102:X 13 Aug 21:53:34.560 # +vote-for-leader b73258f1019f65c5fee3b88b9c2eae7f85d985f3 1 27102:X 13 Aug 21:53:34.580 # +odown master master1 10.1.4.99 6379 #quorum 4/2 27102:X 13 Aug 21:53:34.580 # Next failover delay: I will not start a failover before Sun Aug 13 21:53:45 2017 27102:X 13 Aug 21:53:35.156 # +config-update-from sentinel b73258f1019f65c5fee3b88b9c2eae7f85d985f3 10.1.4.82 26380 @ master1 10.1.4.99 6379 27102:X 13 Aug 21:53:35.156 # +switch-master master1 10.1.4.99 6379 10.1.4.82 6379 27102:X 13 Aug 21:53:35.156 * +slave slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 27102:X 13 Aug 21:53:40.201 # +sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 27102:X 13 Aug 21:54:07.583 # -sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379
啟動10.1.4.82服務器相關服務
redis-sentinel /etc/redis/sentinel_26380.conf redis-sentinel /etc/redis/sentinel_26379.conf
檢查26379端口相關日志
15947:X 13 Aug 21:45:33.882 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. 15947:X 13 Aug 21:45:33.893 # Sentinel ID is 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac 15947:X 13 Aug 21:45:33.893 # +monitor master master1 10.1.4.99 6379 quorum 2 15947:X 13 Aug 21:45:33.894 * +slave slave 10.1.4.82:6379 10.1.4.82 6379 @ master1 10.1.4.99 6379 15951:X 13 Aug 21:45:39.170 * Increased maximum number of open files to 10032 (it was originally set to 1024). _._ _.-``__ ''-._ _.-`` `. `_. ''-._ Redis 3.2.9 (00000000/0) 64 bit .-`` .-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in sentinel mode |`-._`-...-` __...-.``-._|'` _.-'| Port: 26380 | `-._ `._ / _.-' | PID: 15951 `-._ `-._ `-./ _.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | http://redis.io `-._ `-._`-.__.-'_.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | `-._ `-._`-.__.-'_.-' _.-' `-._ `-.__.-' _.-' `-._ _.-' `-.__.-' 15951:X 13 Aug 21:45:39.171 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. 15951:X 13 Aug 21:45:39.174 # Sentinel ID is b73258f1019f65c5fee3b88b9c2eae7f85d985f3 15951:X 13 Aug 21:45:39.174 # +monitor master master1 10.1.4.99 6379 quorum 2 15951:X 13 Aug 21:45:39.175 * +slave slave 10.1.4.82:6379 10.1.4.82 6379 @ master1 10.1.4.99 6379 15951:X 13 Aug 21:45:40.025 * +sentinel sentinel 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac 10.1.4.82 26379 @ master1 10.1.4.99 6379 15947:X 13 Aug 21:45:41.196 * +sentinel sentinel b73258f1019f65c5fee3b88b9c2eae7f85d985f3 10.1.4.82 26380 @ master1 10.1.4.99 6379 15951:X 13 Aug 21:45:57.729 * +sentinel sentinel 361c7d3014a041a1a74163355882d88f5fab7581 10.1.4.99 26379 @ master1 10.1.4.99 6379 15947:X 13 Aug 21:45:57.729 * +sentinel sentinel 361c7d3014a041a1a74163355882d88f5fab7581 10.1.4.99 26379 @ master1 10.1.4.99 6379 15947:X 13 Aug 21:46:05.114 * +sentinel sentinel 76d5a26748410e72ca703e255f44a5ed2b3f83e4 10.1.4.99 26380 @ master1 10.1.4.99 6379 15951:X 13 Aug 21:46:05.115 * +sentinel sentinel 76d5a26748410e72ca703e255f44a5ed2b3f83e4 10.1.4.99 26380 @ master1 10.1.4.99 6379 15951:X 13 Aug 21:47:16.648 # +sdown master master1 10.1.4.99 6379 15951:X 13 Aug 21:47:20.907 # +sdown sentinel 361c7d3014a041a1a74163355882d88f5fab7581 10.1.4.99 26379 @ master1 10.1.4.99 6379 15951:X 13 Aug 21:47:20.908 # +sdown sentinel 76d5a26748410e72ca703e255f44a5ed2b3f83e4 10.1.4.99 26380 @ master1 10.1.4.99 6379 15947:X 13 Aug 21:47:20.908 # +sdown sentinel 361c7d3014a041a1a74163355882d88f5fab7581 10.1.4.99 26379 @ master1 10.1.4.99 6379 15951:X 13 Aug 21:47:20.979 # +tilt #tilt mode entered 15947:X 13 Aug 21:47:20.979 # +tilt #tilt mode entered 15947:X 13 Aug 21:47:50.982 # -tilt #tilt mode exited 15947:X 13 Aug 21:47:50.983 # -sdown sentinel 361c7d3014a041a1a74163355882d88f5fab7581 10.1.4.99 26379 @ master1 10.1.4.99 6379 15951:X 13 Aug 21:47:51.030 # -tilt #tilt mode exited 15951:X 13 Aug 21:47:51.030 # -sdown master master1 10.1.4.99 6379 15951:X 13 Aug 21:47:51.030 # -sdown sentinel 361c7d3014a041a1a74163355882d88f5fab7581 10.1.4.99 26379 @ master1 10.1.4.99 6379 15951:X 13 Aug 21:47:51.030 # -sdown sentinel 76d5a26748410e72ca703e255f44a5ed2b3f83e4 10.1.4.99 26380 @ master1 10.1.4.99 6379 15947:X 13 Aug 21:48:42.378 # +sdown sentinel b73258f1019f65c5fee3b88b9c2eae7f85d985f3 10.1.4.82 26380 @ master1 10.1.4.99 6379 15947:X 13 Aug 21:49:06.458 # -sdown sentinel b73258f1019f65c5fee3b88b9c2eae7f85d985f3 10.1.4.82 26380 @ master1 10.1.4.99 6379 15947:X 13 Aug 21:49:42.140 # +sdown sentinel 361c7d3014a041a1a74163355882d88f5fab7581 10.1.4.99 26379 @ master1 10.1.4.99 6379 15947:X 13 Aug 21:50:59.960 # +sdown sentinel 76d5a26748410e72ca703e255f44a5ed2b3f83e4 10.1.4.99 26380 @ master1 10.1.4.99 6379 15947:X 13 Aug 21:51:20.923 # -sdown sentinel 361c7d3014a041a1a74163355882d88f5fab7581 10.1.4.99 26379 @ master1 10.1.4.99 6379 15947:X 13 Aug 21:51:26.062 # -sdown sentinel 76d5a26748410e72ca703e255f44a5ed2b3f83e4 10.1.4.99 26380 @ master1 10.1.4.99 6379 15947:X 13 Aug 21:52:20.592 # User requested shutdown... 15947:X 13 Aug 21:52:20.592 * Removing the pid file. 15947:X 13 Aug 21:52:20.592 # Sentinel is now ready to exit, bye bye... 17056:X 13 Aug 21:53:18.720 * Increased maximum number of open files to 10032 (it was originally set to 1024). _._ _.-``__ ''-._ _.-`` `. `_. ''-._ Redis 3.2.9 (00000000/0) 64 bit .-`` .-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in sentinel mode |`-._`-...-` __...-.``-._|'` _.-'| Port: 26379 | `-._ `._ / _.-' | PID: 17056 `-._ `-._ `-./ _.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | http://redis.io `-._ `-._`-.__.-'_.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | `-._ `-._`-.__.-'_.-' _.-' `-._ `-.__.-' _.-' `-._ _.-' `-.__.-' 17056:X 13 Aug 21:53:18.721 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. 17056:X 13 Aug 21:53:18.721 # Sentinel ID is 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac 17056:X 13 Aug 21:53:18.721 # +monitor master master1 10.1.4.99 6379 quorum 2 17056:X 13 Aug 21:53:29.549 # +sdown master master1 10.1.4.99 6379 17056:X 13 Aug 21:53:29.643 # +new-epoch 1 17056:X 13 Aug 21:53:29.645 # +vote-for-leader b73258f1019f65c5fee3b88b9c2eae7f85d985f3 1 17056:X 13 Aug 21:53:29.645 # +odown master master1 10.1.4.99 6379 #quorum 2/2 17056:X 13 Aug 21:53:29.645 # Next failover delay: I will not start a failover before Sun Aug 13 21:53:39 2017 17056:X 13 Aug 21:53:30.241 # +config-update-from sentinel b73258f1019f65c5fee3b88b9c2eae7f85d985f3 10.1.4.82 26380 @ master1 10.1.4.99 6379 17056:X 13 Aug 21:53:30.241 # +switch-master master1 10.1.4.99 6379 10.1.4.82 6379 17056:X 13 Aug 21:53:30.241 * +slave slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 17056:X 13 Aug 21:53:35.285 # +sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 17056:X 13 Aug 21:54:02.488 # -sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 17056:X 13 Aug 21:54:12.441 * +convert-to-slave slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379
檢查26380端口相關日志
[root@localhost redis]# tail -200f /data/redis/sentinel/sentinel_26380.log 16390:X 13 Aug 21:49:05.965 * Increased maximum number of open files to 10032 (it was originally set to 1024). _._ _.-``__ ''-._ _.-`` `. `_. ''-._ Redis 3.2.9 (00000000/0) 64 bit .-`` .-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in sentinel mode |`-._`-...-` __...-.``-._|'` _.-'| Port: 26380 | `-._ `._ / _.-' | PID: 16390 `-._ `-._ `-./ _.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | http://redis.io `-._ `-._`-.__.-'_.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | `-._ `-._`-.__.-'_.-' _.-' `-._ `-.__.-' _.-' `-._ _.-' `-.__.-' 16390:X 13 Aug 21:49:05.966 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. 16390:X 13 Aug 21:49:05.966 # Sentinel ID is b73258f1019f65c5fee3b88b9c2eae7f85d985f3 16390:X 13 Aug 21:49:05.966 # +monitor master master1 10.1.4.99 6379 quorum 2 16390:X 13 Aug 21:49:42.173 # +sdown sentinel 361c7d3014a041a1a74163355882d88f5fab7581 10.1.4.99 26379 @ master1 10.1.4.99 6379 16390:X 13 Aug 21:50:59.918 # +sdown sentinel 76d5a26748410e72ca703e255f44a5ed2b3f83e4 10.1.4.99 26380 @ master1 10.1.4.99 6379 16390:X 13 Aug 21:51:21.049 # -sdown sentinel 361c7d3014a041a1a74163355882d88f5fab7581 10.1.4.99 26379 @ master1 10.1.4.99 6379 16390:X 13 Aug 21:51:25.188 # -sdown sentinel 76d5a26748410e72ca703e255f44a5ed2b3f83e4 10.1.4.99 26380 @ master1 10.1.4.99 6379 16390:X 13 Aug 21:52:25.659 # +sdown sentinel 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac 10.1.4.82 26379 @ master1 10.1.4.99 6379 16390:X 13 Aug 21:53:19.651 # -sdown sentinel 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac 10.1.4.82 26379 @ master1 10.1.4.99 6379 16390:X 13 Aug 21:53:29.563 # +sdown master master1 10.1.4.99 6379 16390:X 13 Aug 21:53:29.615 # +odown master master1 10.1.4.99 6379 #quorum 3/2 16390:X 13 Aug 21:53:29.615 # +new-epoch 1 16390:X 13 Aug 21:53:29.615 # +try-failover master master1 10.1.4.99 6379 16390:X 13 Aug 21:53:29.626 # +vote-for-leader b73258f1019f65c5fee3b88b9c2eae7f85d985f3 1 16390:X 13 Aug 21:53:29.627 # 361c7d3014a041a1a74163355882d88f5fab7581 voted for 361c7d3014a041a1a74163355882d88f5fab7581 1 16390:X 13 Aug 21:53:29.645 # 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac voted for b73258f1019f65c5fee3b88b9c2eae7f85d985f3 1 16390:X 13 Aug 21:53:29.646 # 76d5a26748410e72ca703e255f44a5ed2b3f83e4 voted for b73258f1019f65c5fee3b88b9c2eae7f85d985f3 1 16390:X 13 Aug 21:53:29.698 # +elected-leader master master1 10.1.4.99 6379 16390:X 13 Aug 21:53:29.698 # +failover-state-select-slave master master1 10.1.4.99 6379 16390:X 13 Aug 21:53:29.764 # +selected-slave slave 10.1.4.82:6379 10.1.4.82 6379 @ master1 10.1.4.99 6379 16390:X 13 Aug 21:53:29.764 * +failover-state-send-slaveof-noone slave 10.1.4.82:6379 10.1.4.82 6379 @ master1 10.1.4.99 6379 16390:X 13 Aug 21:53:29.826 * +failover-state-wait-promotion slave 10.1.4.82:6379 10.1.4.82 6379 @ master1 10.1.4.99 6379 16390:X 13 Aug 21:53:30.179 # +promoted-slave slave 10.1.4.82:6379 10.1.4.82 6379 @ master1 10.1.4.99 6379 16390:X 13 Aug 21:53:30.179 # +failover-state-reconf-slaves master master1 10.1.4.99 6379 16390:X 13 Aug 21:53:30.237 # +failover-end master master1 10.1.4.99 6379 16390:X 13 Aug 21:53:30.237 # +switch-master master1 10.1.4.99 6379 10.1.4.82 6379 16390:X 13 Aug 21:53:30.238 * +slave slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 16390:X 13 Aug 21:53:35.295 # +sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 16390:X 13 Aug 21:54:02.671 # -sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379
檢查redis主從和哨兵的狀態
10.1.4.99 主從狀態
127.0.0.1:6379> info [section] # Replication role:master connected_slaves:1 slave0:ip=10.1.4.82,port=6379,state=online,offset=2678,lag=0 master_repl_offset:2678 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:2 repl_backlog_histlen:2677
哨兵狀態
[root@bundcredit redis]# redis-cli -p 26379 -a "Redis2017" 127.0.0.1:26379> info # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=master1,status=ok,address=10.1.4.99:6379,slaves=1,sentinels=4
10.1.4.82 主從狀態
127.0.0.1:6379> info [section] # Replication # Replication role:slave master_host:10.1.4.99 master_port:6379 master_link_status:up master_last_io_seconds_ago:1 master_sync_in_progress:0 slave_repl_offset:49526 slave_priority:100 slave_read_only:0 connected_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0
哨兵狀態
[root@bundcredit redis]# redis-cli -p 26379 -a "Redis2017" 127.0.0.1:26379> info # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=master1,status=ok,address=10.1.4.99:6379,slaves=1,sentinels=4 127.0.0.1:26379>
異常測試
1、停止10.1.4.99的redis服務
[root@bundcredit redis]# redis-cli -h 10.1.4.99 -p 6379 -a "Redis2017" 10.1.4.99:6379> SHUTDOWN not connected>
2、哨兵日志
27102:X 13 Aug 21:53:40.201 # +sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 27102:X 13 Aug 21:54:07.583 # -sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 27102:X 13 Aug 22:17:23.410 # +sdown master master1 10.1.4.82 6379 27102:X 13 Aug 22:17:23.445 # +new-epoch 2 27102:X 13 Aug 22:17:23.447 # +vote-for-leader 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac 2 27102:X 13 Aug 22:17:23.462 # +odown master master1 10.1.4.82 6379 #quorum 3/2 27102:X 13 Aug 22:17:23.462 # Next failover delay: I will not start a failover before Sun Aug 13 22:17:34 2017 27102:X 13 Aug 22:17:24.499 # -odown master master1 10.1.4.82 6379 27102:X 13 Aug 22:17:24.533 # +config-update-from sentinel 5c3033c0ea6d1e0d861e9d9e99e94edb49c5daac 10.1.4.82 26379 @ master1 10.1.4.82 6379 27102:X 13 Aug 22:17:24.533 # +switch-master master1 10.1.4.82 6379 10.1.4.99 6379 27102:X 13 Aug 22:17:24.534 * +slave slave 10.1.4.82:6379 10.1.4.82 6379 @ master1 10.1.4.99 6379 27102:X 13 Aug 22:17:29.580 # +sdown slave 10.1.4.82:6379 10.1.4.82 6379 @ master1 10.1.4.99 6379 27102:X 13 Aug 22:17:33.951 # -sdown slave 10.1.4.82:6379 10.1.4.82 6379 @ master1 10.1.4.99 6379 27102:X 13 Aug 22:17:43.855 * +convert-to-slave slave 10.1.4.82:6379 10.1.4.82 6379 @ master1 10.1.4.99 6379 27102:X 13 Aug 22:24:14.405 # +sdown master master1 10.1.4.99 6379 27102:X 13 Aug 22:24:14.445 # +new-epoch 3 27102:X 13 Aug 22:24:14.446 # +vote-for-leader b73258f1019f65c5fee3b88b9c2eae7f85d985f3 3 27102:X 13 Aug 22:24:14.477 # +odown master master1 10.1.4.99 6379 #quorum 4/2 27102:X 13 Aug 22:24:14.477 # Next failover delay: I will not start a failover before Sun Aug 13 22:24:24 2017 27102:X 13 Aug 22:24:15.600 # +config-update-from sentinel b73258f1019f65c5fee3b88b9c2eae7f85d985f3 10.1.4.82 26380 @ master1 10.1.4.99 6379 27102:X 13 Aug 22:24:15.601 # +switch-master master1 10.1.4.99 6379 10.1.4.82 6379 27102:X 13 Aug 22:24:15.601 * +slave slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 27102:X 13 Aug 22:24:20.627 # +sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 27102:X 13 Aug 22:25:49.007 # -sdown slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379 27102:X 13 Aug 22:25:58.984 * +convert-to-slave slave 10.1.4.99:6379 10.1.4.99 6379 @ master1 10.1.4.82 6379
3、檢查服務
10.1.4.82 # Replication role:master connected_slaves:1 slave0:ip=10.1.4.99,port=6379,state=online,offset=17942,lag=0 master_repl_offset:17942 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:2 repl_backlog_histlen:17941
啟動10.1.4.99
檢查服務
# Replication role:slave master_host:10.1.4.82 master_port:6379 master_link_status:up master_last_io_seconds_ago:0 master_sync_in_progress:0 slave_repl_offset:2546 slave_priority:100 slave_read_only:0 connected_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0
再次停止10.1.4.82 redis服務服務正常
注意事項:
在配置redis.conf時需要注意以下事項
1、protected-mode no #保護模式關閉 在3.2版本后添加的安全模式 最好關閉 2、slave-read-only no #默認的從庫不支持寫入,這里需要開啟,當經過哨兵寫入時才支持寫入數據
在配置sentinel.conf時需要注意
1、pdaemonize yes #后臺運行 2、protected-mode no #保護模式關閉 3、sentinel monitor master1 10.1.4.99 6379 2 # 監聽IP 端口 默認最少投票數量 4、sentinel down-after-milliseconds master1 5000 5、sentinel failover-timeout master1 5000 #選舉超時時間 6、logfile "/data/redis/sentinel/sentinel_26379.log" #日志文件 7、sentinel auth-pass master1 Redis2017 #redis密碼 如果主從設置密碼后記得要設置密碼,否則不能登錄
免責聲明:本站發布的內容(圖片、視頻和文字)以原創、轉載和分享為主,文章觀點不代表本網站立場,如果涉及侵權請聯系站長郵箱:is@yisu.com進行舉報,并提供相關證據,一經查實,將立刻刪除涉嫌侵權內容。