logo 开发者
首页 码农运势 代码片段 大模型
关于

Redis手把手集群搭建

Redis集群搭建

环境准备

  • 一台Linux服务器
  • 编译好的Redis二进制文件，版本6.+

这里搭建的Redis集群仅demo级别，供学习用途，如需生产使用还需多服务器部署。

搭建

准备Redis.conf

port 7000
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

以上为最简配置，如需其他可以自加

新建六个文件夹（7001~7006)，也可以是六个服务器

mkdir redis
cd redis
mkdir 7001 7002 7003 7004 7005 7006

将以上配置中 port 7000依次修改为7001、7002、7003、7004、7005、7006

# 这是其中一个Redis，一共6个
port 7001  # 这个不相同，其他配置不变
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
daemonize yes

将配置完的redis.conf移入对应文件夹

root@localhost redis % ll
drwxr-xr-x  6 liujun  staff  192  2 23 14:01 7001
drwxr-xr-x  6 liujun  staff  192  2 23 14:01 7002
drwxr-xr-x  6 liujun  staff  192  2 23 14:01 7003
drwxr-xr-x  6 liujun  staff  192  2 23 14:01 7004
drwxr-xr-x  6 liujun  staff  192  2 23 14:01 7005
drwxr-xr-x  6 liujun  staff  192  2 23 14:01 7006

启动各个Redis实例

root@localhost redis % cd 7001
root@localhost 7001 % redis-server redis.conf

需进入每个文件夹中执行redis启动命令，这里直接使用 redis-server 命令启动，是配置了全局环境变量，所以可以直接使用，若没有配置，则需要使用全路径启动，如 /usr/sbin/redis/redis-server redis.conf

六个实例启动完成，如下

root@localhost redis % ps -ef | grep redis
  501 26466     1   0  5:36下午 ??         0:02.85 redis-server *:7001 [cluster] 
  501 26551     1   0  5:41下午 ??         0:00.06 redis-server *:7002 [cluster] 
  501 26553     1   0  5:41下午 ??         0:00.05 redis-server *:7003 [cluster] 
  501 26555     1   0  5:41下午 ??         0:00.05 redis-server *:7004 [cluster] 
  501 26557     1   0  5:41下午 ??         0:00.06 redis-server *:7005 [cluster] 
  501 26559     1   0  5:41下午 ??         0:00.05 redis-server *:7006 [cluster] 
  501 26575 26403   0  5:41下午 ttys000    0:00.00 grep redis

建立集群

这里使用命令仅适用于Redis 5 或更高版本，若是以前更老版本，请使用redis-trib

Redis 5+

redis-cli --cluster create 127.0.0.1:7001 \
127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \
127.0.0.1:7006 --cluster-replicas 1

Redis 5 以前

redis-trib.rb create --replicas 1 127.0.0.1:7001 \
127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006

执行如下

root@localhost redis % redis-cli --cluster create 127.0.0.1:7001 \
127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \
127.0.0.1:7006 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:7005 to 127.0.0.1:7001
Adding replica 127.0.0.1:7006 to 127.0.0.1:7002
Adding replica 127.0.0.1:7004 to 127.0.0.1:7003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 6aaaff204289440db6dbe0be589fb46522ba015c 127.0.0.1:7001
   slots:[0-5460] (5461 slots) master
M: 299403c911514294d69cae39dc621c08bf1af184 127.0.0.1:7002
   slots:[5461-10922] (5462 slots) master
M: faf57276fcd0e44c09472beb13ed64a972db3d79 127.0.0.1:7003
   slots:[10923-16383] (5461 slots) master
S: 8bc190edfeafc9bdd71d68faca062e1f491b83ed 127.0.0.1:7004
   replicates faf57276fcd0e44c09472beb13ed64a972db3d79
S: 68323fc50d406d860467bf40c1f26ed39d5f9410 127.0.0.1:7005
   replicates 6aaaff204289440db6dbe0be589fb46522ba015c
S: bdf81d7c75edfd7d7778af82573f36990adc13e5 127.0.0.1:7006
   replicates 299403c911514294d69cae39dc621c08bf1af184
Can I set the above configuration? (type 'yes' to accept):

这里输出集群配置，及其每个节点信息和主从配置，输入 yes ， 确认建立集群

Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join

>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: 6aaaff204289440db6dbe0be589fb46522ba015c 127.0.0.1:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 68323fc50d406d860467bf40c1f26ed39d5f9410 127.0.0.1:7005
   slots: (0 slots) slave
   replicates 6aaaff204289440db6dbe0be589fb46522ba015c
M: 299403c911514294d69cae39dc621c08bf1af184 127.0.0.1:7002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: bdf81d7c75edfd7d7778af82573f36990adc13e5 127.0.0.1:7006
   slots: (0 slots) slave
   replicates 299403c911514294d69cae39dc621c08bf1af184
S: 8bc190edfeafc9bdd71d68faca062e1f491b83ed 127.0.0.1:7004
   slots: (0 slots) slave
   replicates faf57276fcd0e44c09472beb13ed64a972db3d79
M: faf57276fcd0e44c09472beb13ed64a972db3d79 127.0.0.1:7003
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

集群建立完毕，可以看到已经输出每个主节点和从节点对应关系，至此Redis集群建立完毕。

添加节点

添加从节点

添加从节点，则为其中一个主节点添加相应从节点，哈希槽不变

启动一个Redis实例，使用以上配置文件，修改下port参数，避免端口冲突，这里使用7007

root@localhost redis % mkdir 7007
root@localhost redis % cp 7006/redis.conf 7007/
root@localhost redis % cd 7007
root@localhost 7007 % ll
# 这里的Redis.conf port 已修改 7007
-rw-r--r--  1 liujun  staff  130  2 25 18:24 redis.conf
root@localhost 7007 % redis-server redis.conf

将新的节点加入某个主节点的从节点

redis-cli --cluster add-node <new_host>:<new_port> <cluster_host>:<cluster_port> --cluster-slave --cluster-master-id <master_node_id>
# new_host: 新节点地址
# new_port: 新节点端口
# cluster_host: 集群内随机节点地址
# cluster_port: 集群内随机节点端口
# master_node_id: 主节点ID

root@localhost ~ % redis-cli --cluster add-node 127.0.0.1:7007 127.0.0.1:7001 --cluster-slave --cluster-master-id 6aaaff204289440db6dbe0be589fb46522ba015c
>>> Adding node 127.0.0.1:7007 to cluster 127.0.0.1:7001
>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: 6aaaff204289440db6dbe0be589fb46522ba015c 127.0.0.1:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 68323fc50d406d860467bf40c1f26ed39d5f9410 127.0.0.1:7005
   slots: (0 slots) slave
   replicates 6aaaff204289440db6dbe0be589fb46522ba015c
M: 299403c911514294d69cae39dc621c08bf1af184 127.0.0.1:7002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: bdf81d7c75edfd7d7778af82573f36990adc13e5 127.0.0.1:7006
   slots: (0 slots) slave
   replicates 299403c911514294d69cae39dc621c08bf1af184
S: 8bc190edfeafc9bdd71d68faca062e1f491b83ed 127.0.0.1:7004
   slots: (0 slots) slave
   replicates faf57276fcd0e44c09472beb13ed64a972db3d79
M: faf57276fcd0e44c09472beb13ed64a972db3d79 127.0.0.1:7003
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:7007 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of 127.0.0.1:7001.
[OK] New node added correctly.

添加为从节点可以有多种方式，这里只介绍了一种

添加主节点

添加主节点，正如我们之前所说，Redis KEY被划分为16384个哈希槽，多个主节点处理其中的一个或者一部分，现在添加一个主节点就需要处理一部分哈希槽，就需要从现有的主节点中抽取一部分哈希槽来给新的主节点处理，那么怎么处理呢，基本一下几步：

  • 新建一个Redis实例作为新的主节点
  • 使用reshard命令为新节点分配哈希槽
  • 等待哈希槽转移完毕

1.新建一个Redis实例

root@localhost redis % mkdir 7008 
root@localhost redis % cp 7007/redis.conf 7008/
root@localhost redis % cd 7008
root@localhost 7008 % vim redis.conf 
root@localhost 7008 % redis-server redis.conf

2.添加新节点为空节点

redis-cli --cluster add-node <new_host>:<new_port> <cluster_host>:<cluster_port>

root@localhost 7008 % redis-cli --cluster add-node 127.0.0.1:7008 127.0.0.1:7001
>>> Adding node 127.0.0.1:7008 to cluster 127.0.0.1:7001
>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: 6aaaff204289440db6dbe0be589fb46522ba015c 127.0.0.1:7001
   slots:[0-5460] (5461 slots) master
   2 additional replica(s)
S: 68323fc50d406d860467bf40c1f26ed39d5f9410 127.0.0.1:7005
   slots: (0 slots) slave
   replicates 6aaaff204289440db6dbe0be589fb46522ba015c
S: c4dc9e656c66c870836686e455244f742d300911 127.0.0.1:7007
   slots: (0 slots) slave
   replicates 6aaaff204289440db6dbe0be589fb46522ba015c
M: 299403c911514294d69cae39dc621c08bf1af184 127.0.0.1:7002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: bdf81d7c75edfd7d7778af82573f36990adc13e5 127.0.0.1:7006
   slots: (0 slots) slave
   replicates 299403c911514294d69cae39dc621c08bf1af184
S: 8bc190edfeafc9bdd71d68faca062e1f491b83ed 127.0.0.1:7004
   slots: (0 slots) slave
   replicates faf57276fcd0e44c09472beb13ed64a972db3d79
M: faf57276fcd0e44c09472beb13ed64a972db3d79 127.0.0.1:7003
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:7008 to make it join the cluster.
[OK] New node added correctly.

5.分配哈希槽

redis-cli --cluster reshard <cluster_host>:<cluster_port>
# cluster_host: 集群内随机节点地址
# cluster_port: 集群内随机节点端口

root@localhost 7008 % redis-cli --cluster reshard 127.0.0.1:7001
>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: 6aaaff204289440db6dbe0be589fb46522ba015c 127.0.0.1:7001
   slots:[0-5460] (5461 slots) master
   2 additional replica(s)
S: 68323fc50d406d860467bf40c1f26ed39d5f9410 127.0.0.1:7005
   slots: (0 slots) slave
   replicates 6aaaff204289440db6dbe0be589fb46522ba015c
M: 3f3abba12b3acbea3eb2e45e8ca3ec1493f864a0 127.0.0.1:7008
   slots: (0 slots) master
S: c4dc9e656c66c870836686e455244f742d300911 127.0.0.1:7007
   slots: (0 slots) slave
   replicates 6aaaff204289440db6dbe0be589fb46522ba015c
M: 299403c911514294d69cae39dc621c08bf1af184 127.0.0.1:7002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: bdf81d7c75edfd7d7778af82573f36990adc13e5 127.0.0.1:7006
   slots: (0 slots) slave
   replicates 299403c911514294d69cae39dc621c08bf1af184
S: 8bc190edfeafc9bdd71d68faca062e1f491b83ed 127.0.0.1:7004
   slots: (0 slots) slave
   replicates faf57276fcd0e44c09472beb13ed64a972db3d79
M: faf57276fcd0e44c09472beb13ed64a972db3d79 127.0.0.1:7003
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
# 这里输入想要转移多少哈希槽
How many slots do you want to move (from 1 to 16384)? 2000
# 这里输入转移哈希槽的目的地NodeId
What is the receiving node ID? 3f3abba12b3acbea3eb2e45e8ca3ec1493f864a0
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
# 哈希槽来源节点
# all: 从所有节点抽取
Source node #1: all

Ready to move 2000 slots.
  Source nodes:
    M: 6aaaff204289440db6dbe0be589fb46522ba015c 127.0.0.1:7001
       slots:[0-5460] (5461 slots) master
       2 additional replica(s)
    M: 299403c911514294d69cae39dc621c08bf1af184 127.0.0.1:7002
       slots:[5461-10922] (5462 slots) master
       1 additional replica(s)
    M: faf57276fcd0e44c09472beb13ed64a972db3d79 127.0.0.1:7003
       slots:[10923-16383] (5461 slots) master
       1 additional replica(s)
  Destination node:
    M: 3f3abba12b3acbea3eb2e45e8ca3ec1493f864a0 127.0.0.1:7008
       slots: (0 slots) master
  Resharding plan:
  Moving slot 5461 from 299403c911514294d69cae39dc621c08bf1af184
    Moving slot 5462 from 299403c911514294d69cae39dc621c08bf1af184
    Moving slot 5463 from 299403c911514294d69cae39dc621c08bf1af184
    Moving slot 5464 from 299403c911514294d69cae39dc621c08bf1af184
    Moving slot 5465 from 299403c911514294d69cae39dc621c08bf1af184
    Moving slot 5466 from 299403c911514294d69cae39dc621c08bf1af184
    Moving slot 5467 from 299403c911514294d69cae39dc621c08bf1af184
    Moving slot 5468 from 299403c911514294d69cae39dc621c08bf1af184
    Moving slot 5469 from 299403c911514294d69cae39dc621c08bf1af184
    Moving slot 5470 from 299403c911514294d69cae39dc621c08bf1af184
    Moving slot 5471 from 299403c911514294d69cae39dc621c08bf1af184
    Moving slot 5472 from 299403c911514294d69cae39dc621c08bf1af184
    ······
    Moving slot 11579 from 127.0.0.1:7003 to 127.0.0.1:7008: 
		Moving slot 11580 from 127.0.0.1:7003 to 127.0.0.1:7008: 
		Moving slot 11581 from 127.0.0.1:7003 to 127.0.0.1:7008: 
		Moving slot 11582 from 127.0.0.1:7003 to 127.0.0.1:7008: 
		Moving slot 11583 from 127.0.0.1:7003 to 127.0.0.1:7008: 
		Moving slot 11584 from 127.0.0.1:7003 to 127.0.0.1:7008: 
		Moving slot 11585 from 127.0.0.1:7003 to 127.0.0.1:7008: 
		Moving slot 11586 from 127.0.0.1:7003 to 127.0.0.1:7008: 
		Moving slot 11587 from 127.0.0.1:7003 to 127.0.0.1:7008: 
		Moving slot 11588 from 127.0.0.1:7003 to 127.0.0.1:7008: 
# master节点添加成功

删除节点

redis-cli --cluster del-node <node_host>:<node_port> <node_id> 
# node_host: node地址
# node_port: node端口
# node_id: node ID

root@localhost 7008 % redis-cli --cluster del-node 127.0.0.1:7007 c4dc9e656c66c870836686e455244f742d300911 
>>> Removing node c4dc9e656c66c870836686e455244f742d300911 from cluster 127.0.0.1:7007
>>> Sending CLUSTER FORGET messages to the cluster...
>>> Sending CLUSTER RESET SOFT to the deleted node.

参考资料

https://redis.io/topics/cluster-tutorial

*建站不易，转发请注明来源