Redis Cluster

Introduction

This tutorial going to be a quick one, we are going to setup a redis cluster with master slave replication.

I am going to use Docker to setup the cluster, but you can setup the cluster on your local machine or on a server.

For TL;DR, you can skip to the TLDR section.

Prerequisites

Ubuntu (probably works on most linux distros)
Redis
Python3

Redis Cluster

Redis cluster is a distributed implementation of redis. It uses sharding to distribute data across multiple nodes.

Redis cluster has the following properties¹

There has to be at least 3 master nodes.
There are 16384 hash slots in redis cluster.
Every node in the cluster is responsible for a subset of hash slots.
Every node in the cluster is connected to every other node in the cluster.
Every node in the cluster knows which node is responsible for which hash slot.
When a node receives a command, it checks if it is responsible for the hash slot of the key.
If the node is responsible for the hash slot, it executes the command.
If the node is not responsible for the hash slot, it forwards the command to the node that is responsible for the hash slot.
Redis clusters are not strongly consistent, so it is possible that some nodes have different values for the same key.

Master Slave Responsiblities

Master node is the node that accepts write commands and slave nodes are the nodes that accept read commands.
When a master node receives a write command, it executes the command and forwards the command to the slave nodes.
When a slave node receives a read command, it executes the command.

Setup

We are going to setup a cluster with 6 nodes. 3 master nodes and 3 slave nodes. We will use the ports 7000 to 7005 for the nodes they are going to be on the same machine. You can use different ports and ip addresses if you want to setup the cluster on different machines.

Create a folder and redis.conf file for each node.

mkdir 7000
cd 7000
vim redis.conf

Add the following configuration to the redis.conf file

port 7000
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

Repeat the same steps for the other nodes. (don't forget to change the port number)

Then run the following command on each folder to start the nodes

redis-server redis.conf

Now we have 6 nodes running on the ports 7000 to 7005.

Initialize Cluster

We are going to use the redis-cli to initialize the cluster.

redis-cli --cluster create \
  127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 \
  127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \
  --cluster-replicas 1

Then you should see something like this, type yes and hit enter

... // some logs
...
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
...
... // some logs

Python Client

We are going to use python redis client to connect to the cluster.

from redis.cluster import RedisCluster as Redis
 
rc = Redis(host="localhost", port=7000) # any node will do
print(rc.get_nodes())

Run the python script

python3 test.py

You should see something like this

[
  [host=127.0.0.1,port=7000,name=127.0.0.1:7000,server_type=primary,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=7000,db=0>>>],
  [host=127.0.0.1,port=7003,name=127.0.0.1:7003,server_type=replica,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=7003,db=0>>>],
  [host=127.0.0.1,port=7001,name=127.0.0.1:7001,server_type=primary,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=7001,db=0>>>],
  [host=127.0.0.1,port=7004,name=127.0.0.1:7004,server_type=replica,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=7004,db=0>>>],
  [host=127.0.0.1,port=7002,name=127.0.0.1:7002,server_type=primary,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=7002,db=0>>>],
  [host=127.0.0.1,port=7005,name=127.0.0.1:7005,server_type=replica,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=7005,db=0>>>]
]

Test

rc.set("foo", "bar")
print('foo:', rc.get("foo"))  # foo: b'bar'

🔥 Now you can add Redis to your resume 🔥

TLDR With Docker

If you want to try without any hassle, you can use the following script to setup the cluster.

Pull and run the latest ubuntu image

docker pull ubuntu:latest
docker run -it ubuntu:latest

Create a folder and cd into it

mkdir redis-cluster
cd redis-cluster

Create a file called setup.sh

nano setup.sh

then paste the following code

#!/bin/bash
 
apt update
apt install vim redis python3 python3-pip
 
pip3 install redis
 
# Create folders 7000 to 7005
for i in {7000..7005}; do
  folder_name="$i"
  mkdir "$folder_name"
  cd "$folder_name"
 
  # Create redis.conf file
  redis_conf="./redis.conf"
  echo "port $folder_name" > "$redis_conf"
  echo "cluster-enabled yes" >> "$redis_conf"
  echo "cluster-config-file nodes.conf" >> "$redis_conf"
  echo "cluster-node-timeout 5000" >> "$redis_conf"
  echo "appendonly yes" >> "$redis_conf"
 
  cd ..
done
 
for i in */ ; do
  echo $i
  cd $i
  redis-server redis.conf &
  cd ..
done
 
# Wait for Redis servers to start
sleep 3
 
# Initialize Redis cluster
redis-cli --cluster create \
  127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 \
  127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \
  --cluster-replicas 1

Run the setup.sh script

bash setup.sh

You should see something like this Type yes and hit enter

... // some logs
...
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
...
... // some logs

Now we can test the cluster

from redis.cluster import RedisCluster as Redis
 
rc = Redis(host="localhost", port=7000) # any node will do
print(rc.get_nodes())
 
rc.set("foo", "bar")
print('foo:', rc.get("foo"))

Run the python script

python3 test.py

You should see something like this

root@5940b81bbc3a:/redis-cluster# python3 main.py
[
  [host=127.0.0.1,port=7000,name=127.0.0.1:7000,server_type=primary,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=7000,db=0>>>],
  [host=127.0.0.1,port=7003,name=127.0.0.1:7003,server_type=replica,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=7003,db=0>>>],
  [host=127.0.0.1,port=7001,name=127.0.0.1:7001,server_type=primary,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=7001,db=0>>>],
  [host=127.0.0.1,port=7004,name=127.0.0.1:7004,server_type=replica,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=7004,db=0>>>],
  [host=127.0.0.1,port=7002,name=127.0.0.1:7002,server_type=primary,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=7002,db=0>>>],
  [host=127.0.0.1,port=7005,name=127.0.0.1:7005,server_type=replica,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=7005,db=0>>>]
]
foo: b'bar'

Voila! Now we have a redis cluster with master slave replication.

7000, 7001, 7002 are the master nodes and 7003, 7004, 7005 are the slave nodes.

Final Notes

If any of the master nodes goes down, the slave node will take over the master node. Because we are using the RedisCluster client to connect to the cluster, we don't have to worry about which node is the master node and which node is the slave node. The client will automatically connect to available nodes.

Different Machine Setup

If you want to setup the cluster on multiple machines, you just have to change the host and port of the nodes. Rest of the steps are the same.

To Idiots

Redis is not a database

¹ You can learn more here about redis clusters