Redis
是一个 K-V
存储系统,是一个高性能的 K-V
数据库,是目前大规模使用的缓存中间件。单节点的Redis
已经就达到了很高的性能,为了提高可用性我们可以使用 Redis
集群。
Redis Website
集群简介 Redis
集群是一个可以在多个 Redis
节点之间进行数据共享的设施(installation
)。
Redis
集群不支持那些需要同时处理多个键的 Redis
命令, 因为执行这些命令需要在多个 Redis
节点之间移动数据, 并且在高负载的情况下, 这些命令将降低 Redis
集群的性能,并导致不可预测的行为。
Redis
集群通过分区(partition
)来提供一定程度的可用性(availability
):即使集群中有一部分节点失效或者无法进行通讯,集群也可以继续处理命令请求。
Redis
集群提供了以下两个好处:
将数据自动切分(split
)到多个节点的能力 当集群中的一部分节点失效或者无法进行通讯时,仍然可以继续处理命令请求的能力 集群说明 集群数据共享 Redis
集群使用数据分片(sharding
)而非一致性哈希(consistency hashing
)来实现:一个 Redis
集群包含 16384
个哈希槽(hash slot
),数据库中的每个键都属于这 16384
个哈希槽的其中一个,集群使用公式 CRC16(key) % 16384
来计算键 key
属于哪个槽,其中 CRC16(key)
语句用于计算键 key
的 CRC16
校验和。
集群中的每个节点负责处理一部分哈希槽 一个集群可以有三个哈希槽,其中:
节点 A
负责处理 0
号至 5500
号哈希槽 节点 B
负责处理 5501
号至 11000
号哈希槽 节点 C
负责处理 11001
号至 16384
号哈希槽 集群中的主从复制 为了使得集群在一部分节点下线或者无法与集群的大多数(majority
)节点进行通讯的情况下,仍然可以正常运作,Redis
集群对节点使用了主从复制功能:集群中的每个节点都有 1
个至 N
个复制品(replica
),其中一个复制品为主节点(master
),而其余的 N-1
个复制品为从节点(slave
)
集群 TCP
端口 每个 Redis Cluster
节点都需要 2
个 TCP
连接打开,正常的 Redis TCP
端口被服务于客户端,例如:6379
,加上通过增加 10000
到数据端口获取的端口,例如:16379
。
第二个高端口被用来 Redis Cluster Bus
,也就是使用二进制协议的一个节点到节点的通信信道。Redis Cluster Bus
被节点用于故障检测,配置更新,故障转移授权等等。客户端不应该尝试与 Redis Cluster Bus
端口进行通信,而是始终使用正常的 Redis
命令端口,然而要确保在防火墙中打开两个端口,否则 Redis Cluster
节点将无法通信。
命令端口和 Redis Cluster Bus
端口偏移是固定的,始终为 10000
。
注意,为了使 Redis Cluster
正常工作,你需要为每个节点:
用于客户端进行通信的普通客户端通信端口(通常使用 6379
)对所有需要到达集群的客户端,以及所有其它集群节点(使用客户端端口进行密钥迁移)都是开放的 Redis Cluster Bus
端口(客户端端口 +10000
)必须可从所有其它集群节点访问如果不打开两个 TCP
端口,集群将无法按预期工作
Redis Cluster Bus
使用不同的二进制协议进行节占到节点的数据交换,这更适合于使用少量的带宽和处理时间交换节点之间的信息
安装依赖 1 2 3 4 5 6 7 8 $ sudo gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 $ sudo curl -sSL https://get.rvm.io | bash -s stable $ sudo source ~/.rvm/scripts/rvm $ sudo echo "ruby_url=https://cache.ruby-china.org/pub/ruby" > ~/.rvm/user/db $ rvm -v $ sudo rvm install 2.4.1 $ sudo gem update $ sudo gem install redis
集群配置 安装 Redis 1 2 3 4 5 $ cd ~ $ wget http://download.redis.io/releases/redis-4.0.9.tar.gz $ tar xzf redis-4.0.9.tar.gz $ cd redis-4.0.9 $ sudo make
创建配置目录 1 $ sudo mkdir -pv /usr/local /redis-cluster/{bin,etc,log ,run,var}
创建配置文件 1 $ sudo cp ~/redis-4.0.9/redis.conf /usr/local /redis-cluster/etc/redis-7001.conf
复制命令文件 1 2 $ cd ~/redis-4.0.9/src/ $ sudo cp mkreleasehdr.sh redis-benchmark redis-check-aof redis-check-rdb redis-sentinel redis-cli redis-server /usr/local /redis-cluster/bin
修改配置文件 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 $ sudo vim /usr/local /redis-cluster/etc/redis-7001.conf bind 192.168.33.110port 7001 daemonize yes pidfile /user/local /redis-cluster/run/7001.pid logfile /usr/local /redis-cluster/log /7001.log dbfilename dump-7001.rdb dir /usr/local /redis-cluster/var appendonly yes appendfilename "appendonly-7001.aof" cluster-enabled yes cluster-config-file nodes-7001.conf cluster-node-timeout 15000
复制配置文件 将复制后的文件中的 7001
换成相应的名称
1 2 3 4 5 $ sudo cp /usr/local /redis-cluster/etc/redis-7001.conf /usr/local /redis-cluster/etc/redis-7002.conf $ sudo cp /usr/local /redis-cluster/etc/redis-7001.conf /usr/local /redis-cluster/etc/redis-7003.conf $ sudo cp /usr/local /redis-cluster/etc/redis-7001.conf /usr/local /redis-cluster/etc/redis-7004.conf $ sudo cp /usr/local /redis-cluster/etc/redis-7001.conf /usr/local /redis-cluster/etc/redis-7005.conf $ sudo cp /usr/local /redis-cluster/etc/redis-7001.conf /usr/local /redis-cluster/etc/redis-7006.conf
集群部署 启动节点 1 2 3 4 5 6 7 // 启动配置的所有节点 $ sudo /usr/local /redis-cluster/bin/redis-cli /usr/local /redis-cluster/etc/redis-7001.conf $ sudo /usr/local /redis-cluster/bin/redis-cli /usr/local /redis-cluster/etc/redis-7002.conf $ sudo /usr/local /redis-cluster/bin/redis-cli /usr/local /redis-cluster/etc/redis-7003.conf $ sudo /usr/local /redis-cluster/bin/redis-cli /usr/local /redis-cluster/etc/redis-7004.conf $ sudo /usr/local /redis-cluster/bin/redis-cli /usr/local /redis-cluster/etc/redis-7005.conf $ sudo /usr/local /redis-cluster/bin/redis-cli /usr/local /redis-cluster/etc/redis-7006.conf
检查服务 1 2 3 4 5 6 7 $ ps -ef | grep redis root 20077 1 0 May18 ? 00:01:04 /usr/local /bin/redis-server 192.168.33.110:7001 [cluster] root 20079 1 0 May18 ? 00:01:03 /usr/local /bin/redis-server 192.168.33.110:7002 [cluster] root 20081 1 0 May18 ? 00:01:03 /usr/local /bin/redis-server 192.168.33.110:7003 [cluster] root 20089 1 0 May18 ? 00:01:03 /usr/local /bin/redis-server 192.168.33.110:7004 [cluster] root 20091 1 0 May18 ? 00:01:03 /usr/local /bin/redis-server 192.168.33.110:7005 [cluster] root 20096 1 0 May18 ? 00:01:03 /usr/local /bin/redis-server 192.168.33.110:7006 [cluster]
停止节点 1 $ sudo /usr/local /redis-cluster/bin/redis-cli -c -h 192.168.33.110 -p 7001 shutdown
节点加入集群 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 $ sudo cp ~/redis-4.0.9/src/redis-trib.rb /usr/local /redis-cluster/bin/redis-trib $ sudo chmod a+x /usr/local /redis-cluster/bin/redis-trib $ sudo /usr/local /redis-cluster/bin/redis-trib create --replicas 1 \ > 192.168.33.110:7001 \ > 192.168.33.110:7002 \ > 192.168.33.110:7003 \ > 192.168.33.110:7004 \ > 192.168.33.110:7005 \ > 192.168.33.110:7006 \ >>> Creating cluster >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 Adding replica 127.0.0.1:7005 to 127.0.0.1:7001 Adding replica 127.0.0.1:7006 to 127.0.0.1:7002 Adding replica 127.0.0.1:7004 to 127.0.0.1:7003 >>> Trying to optimize slaves allocation for anti-affinity [WARNING] Some slaves are in the same host as their master M: 4ed7d8f54311fb883431c1c5a4715c1ae59d5215 127.0.0.1:7001 slots:0-5460 (5461 slots) master M: 31030e4c35659d5de1146941a2d1f2c1524b53e9 127.0.0.1:7002 slots:5461-10922 (5462 slots) master M: 573752b8b0747bc2c7bb3e4a1edb0e467cfdef0e 127.0.0.1:7003 slots:10923-16383 (5461 slots) master S: 11f577639b081c1514f28a74764c9330d819dcb8 127.0.0.1:7004 replicates 573752b8b0747bc2c7bb3e4a1edb0e467cfdef0e S: ff6d7ca3ea29036385c89a5dc93f807f91307871 127.0.0.1:7005 replicates 4ed7d8f54311fb883431c1c5a4715c1ae59d5215 S: a6f79f4d1de8d7bef4911e934b802829ed29a2d3 127.0.0.1:7006 replicates 31030e4c35659d5de1146941a2d1f2c1524b53e9 Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join.. >>> Performing Cluster Check (using node 127.0.0.1:7001) M: 4ed7d8f54311fb883431c1c5a4715c1ae59d5215 127.0.0.1:7001 slots:0-5460 (5461 slots) master 1 additional replica(s) S: 11f577639b081c1514f28a74764c9330d819dcb8 127.0.0.1:7004 slots: (0 slots) slave replicates 573752b8b0747bc2c7bb3e4a1edb0e467cfdef0e S: a6f79f4d1de8d7bef4911e934b802829ed29a2d3 127.0.0.1:7006 slots: (0 slots) slave replicates 31030e4c35659d5de1146941a2d1f2c1524b53e9 S: ff6d7ca3ea29036385c89a5dc93f807f91307871 127.0.0.1:7005 slots: (0 slots) slave replicates 4ed7d8f54311fb883431c1c5a4715c1ae59d5215 M: 31030e4c35659d5de1146941a2d1f2c1524b53e9 127.0.0.1:7002 slots:5461-10922 (5462 slots) master 1 additional replica(s) M: 573752b8b0747bc2c7bb3e4a1edb0e467cfdef0e 127.0.0.1:7003 slots:10923-16383 (5461 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
可以看到,创建集群这个过程比较繁琐,可以创建脚本来完成这些操作
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 $ sudo vim /usr/local /redis-cluster/bin/start.sh RedisServerPath=/usr/local /redis-cluster/bin RedisConfPath=/usr/local /redis-cluster/etc for i in 1 2 3 4 5 6; do $RedisServerPath /redis-server $RedisConfPath /redis-700$i .conf done /usr/local /redis-cluster/bin/redis-trib.rb create --replicas 1 \ 192.168.33.110:7001 \ 192.168.33.110:7002 \ 192.168.33.110:7003 \ 192.168.33.110:7004 \ 192.168.33.110:7005 \ 192.168.33.110:7006 \ $ sudo chmod a+x /usr/local /redis-cluster/bin/start.sh
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 $ sudo vim /usr/local /redis-cluster/bin/stop.sh RedisClusterPath=/usr/local /redis-cluster RedisCliPath=/usr/local /redis-cluster/bin ip=192.168.33.110 for i in 1 2 3 4 5 6; do $RedisCliPath /redis-cli -c -h $ip -p 700$i shutdown done rm -rf $RedisClusterPath /log /* rm -rf $RedisClusterPath /run/* rm -rf $RedisClusterPath /var/* $ sudo chmod a+x /usr/local /redis-cluster/bin/stop.sh
查看信息 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 $ /usr/local /redis-cluster/bin/redis-cli -c -h 192.168.33.110 -p 7001 192.168.33.110:7001> cluster nodes 5521b06a46910c873e7ee8a274f2cb9fc41ffed1 192.168.33.110:7003@17003 master - 0 1526681994008 3 connected 10923-16383 8153a1b35e0fab1e48f0171653f6cffd82cbc5a6 192.168.33.110:7004@17004 slave d7e0395152496f41fe73fbbdabc8499c79f6632e 0 1526681996033 4 connected a2ff29661185f6ad17c99232d0490b3814ba0164 192.168.33.110:7006@17006 slave affc237bf177fd4ac591314736a1358e16545d0c 0 1526681995020 6 connected d7e0395152496f41fe73fbbdabc8499c79f6632e 192.168.33.110:7002@17002 master - 0 1526681993000 2 connected 5461-10922 8b613e4b5abbdf3c2486dfcb8bc1e9dde1897820 192.168.33.110:7005@17005 slave 5521b06a46910c873e7ee8a274f2cb9fc41ffed1 0 1526681993000 5 connected affc237bf177fd4ac591314736a1358e16545d0c 192.168.33.110:7001@17001 myself,master - 0 1526681994000 1 connected 0-5460 192.168.33.110:7001> cluster info cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:1 cluster_stats_messages_ping_sent:59501 cluster_stats_messages_pong_sent:61755 cluster_stats_messages_sent:121256 cluster_stats_messages_ping_received:61750 cluster_stats_messages_pong_received:59501 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:121256
相关命令 命令 说明 CLUSTER INFO
打印集群信息 CLUSTER NODES
列出集群当前已知的所有节点信息 CLUSTER MEET <ip> <port>
将 ip
和 port
所指定的节点添加到集群当中 CLUSTER FORGET <node_id>
从集群中移除 node_id
指定的节点 CLUSTER REPLICATE <node_id>
将当前节点设置为 node_id
指定的节点的从节点 CLUSTER SAVECONFIG
将节点的配置文件保存到硬盘里面 CLUSTER ADDSLOTS <slot> [slot ...]
将一个或多个槽(slot
)指派(assign
)给当前节点 CLUSTER DELSLOTS <slot> [slot ...]
移除一个或多个槽对当前节点的指派 CLUSTER FLUSHSLOTS
移除指派给当前节点的所有槽,让当前节点变成一个没有指派任何槽的节点 CLUSTER SETSLOT <slot> NODE <node_id>
将槽 slot
指派给 node_id
指定的节点,如果槽已经指派给另一个节点,那么先让另一个节点删除该槽,然后再进行指派 CLUSTER SETSLOT <slot> MIGRATING <node_id>
将本节点的槽 slot
迁移到 node_id
指定的节点中 CLUSTER SETSLOT <slot> IMPORTING <node_id>
从 node_id
指定的节点中导入槽 slot
到本节点 CLUSTER SETSLOT <slot> STABLE
取消对槽 slot
的导入(import
)或者迁移(migrate
) CLUSTER KEYSLOT <key>
计算键 key
应该被放置在哪个槽上 CLUSTER COUNTKEYSINSLOT <slot>
返回槽 slot
目前包含的键值对数量 CLUSTER GETKEYSINSLOT <slot> <count>
返回 count
个 slot
槽中的键
实例操作 客户端操作 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 // 从某个节点设置值后,去不同的节点获取设置的值 $ /usr/local /redis-cluster/bin/redis-cli -c -h 192.168.33.110 -p 7001 192.168.33.110:7001> set foo bar -> Redirected to slot [12182] located at 192.168.33.110:7003 OK $ /usr/local /redis-cluster/bin/redis-cli -c -h 192.168.33.110 -p 7001 192.168.33.110:7001> get foo -> Redirected to slot [12182] located at 192.168.33.110:7003 "bar" $ /usr/local /redis-cluster/bin/redis-cli -c -h 192.168.33.110 -p 7002 192.168.33.110:7002> get foo -> Redirected to slot [12182] located at 192.168.33.110:7003 "bar" $ /usr/local /redis-cluster/bin/redis-cli -c -h 192.168.33.110 -p 7003 192.168.33.110:7003> get foo "bar" $ /usr/local /redis-cluster/bin/redis-cli -c -h 192.168.33.110 -p 7004 192.168.33.110:7004> get foo -> Redirected to slot [12182] located at 192.168.33.110:7003 "bar" $ /usr/local /redis-cluster/bin/redis-cli -c -h 192.168.33.110 -p 7005 192.168.33.110:7005> get foo -> Redirected to slot [12182] located at 192.168.33.110:7003 "bar" $ /usr/local /redis-cluster/bin/redis-cli -c -h 192.168.33.110 -p 7006 192.168.33.110:7006> get foo -> Redirected to slot [12182] located at 192.168.33.110:7003 "bar"
程序操作 下载安装 1 2 3 4 5 6 $ cd ~ $ mkdir test $ cd test $ composer init $ composer install $ composer require predis/predis
测试代码 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 <?php require __DIR__ . '/vendor/autoload.php' ;for ($i = 0 ; $i < 6 ; $i ++) { $parameters [] = [ 'host' => '192.168.33.110' , 'port' => '700' . $i , 'password' => null , 'database' => 0 , 'timeout' => 5 , 'read_write_timeout' => 5 , ]; } $options = ['cluster' => 'redis' ];try { $cluster = new \Predis\Client($parameters , $options ); } catch (\Predis\PredisException $e ) { echo $e ->getMessage(); exit (); } for ($i = 1 ; $i < 100 ; $i ++) { $format = 'No.%s is a RedisCluster test.' ; $cluster ->set("$i " , sprintf($format , $i )); echo $cluster ->get("$i " ) . PHP_EOL; }
测试结果 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 $ php cluster.php No.1 is a Redis cluster test . No.2 is a Redis cluster test . No.3 is a Redis cluster test . ... No.99 is a Redis cluster test . $ /usr/local /redis-cluster/bin/redis-cli -c -h 192.168.33.110 -p 7001 192.168.33.110:7001> get 11 "No.11 is a RedisCluster test." 192.168.33.110:7001> get 23 -> Redirected to slot [9671] located at 192.168.33.110:7002 "No.23 is a RedisCluster test." 192.168.33.110:7002> get 33 "No.33 is a RedisCluster test." 192.168.33.110:7002> get 67 "No.67 is a RedisCluster test." 192.168.33.110:7002> get 88 -> Redirected to slot [15207] located at 192.168.33.110:7003 "No.88 is a RedisCluster test." 192.168.33.110:7003> get 99 -> Redirected to slot [6263] located at 192.168.33.110:7002 "No.99 is a RedisCluster test." 192.168.33.110:7002> get 100 -> Redirected to slot [339] located at 192.168.33.110:7001 (nil) 192.168.33.110:7001> get 17 -> Redirected to slot [12304] located at 192.168.33.110:7003 "No.17 is a RedisCluster test." 192.168.33.110:7003> get 42 -> Redirected to slot [8000] located at 192.168.33.110:7002 "No.42 is a RedisCluster test." 192.168.33.110:7002> get 56 -> Redirected to slot [11509] located at 192.168.33.110:7003 "No.56 is a RedisCluster test." 192.168.33.110:7003> get 61 -> Redirected to slot [2369] located at 192.168.33.110:7001 "No.61 is a RedisCluster test."