Click here to Skip to main content
13,900,891 members
Click here to Skip to main content
Add your own
alternative version

Tagged as


3 bookmarked
Posted 22 Sep 2013
Licenced CPOL

Redis-Sentinel TwemProxy Agent

, 22 Sep 2013
Rate this:
Please Sign up or sign in to vote.
A nodejs agent to connect to Redis-Sentinel and update TwemProxy on master node changes


I was recently introduced to Redis by a colleague. For those of you who don't know what it is, it is essentially an incredibly fast key-value store.

We had decided to adopt this technology at work, so we began to think about Performance, Redundancy, etc., and after a day or two of playing around, discovered that there isn't really an established solution for what we wanted to do.

As a result, I ended up combining a few different technologies to get the setup we required, to give us redundancy, automatic failover, and horizontal scaling.


The whole solution utilizes the following technologies:

Redis Configuration

The purpose of this article isn't to tell you how to set each of these technologies up individually, it is to show you how I link them together, and more specifically, the Redis-TwemProxy Agent that does the linking of Redis-Sentinel and TwemProxy.

This diagram best explains it:


Redis slaves in each shard are read-only copies of their master.

Redis-Sentinel constantly monitors your master nodes in each shard, should a master node go down, a slave node is promoted to the master, and the other slaves are reconfigured to be slaves of this new master.

The Redis-TwemProxy agent script is subscribed to Redis-Sentinel, and detects the master-change event, when this happens, the TwemProxy config is updated and the TwemProxy service is restarted.

Redis-TwemProxy AGent

This was the missing link for me. Redis Sentinel worked perfectly, and promoted my slaves to masters when the master went down, however TwemProxy was still directing traffic to the old, now dead, slave.

The script I have written over on GitHub, is this glue. It is a simple nodejs script that is run permanently on boot on the TwemProxy server, and keeps the TwemProxy config up to date.

The Result

If you tail your TwemProxy Sentinel log, you should see something like this:

[11:31:04] Loading TwemProxy config
[11:31:04] Redis Sentinel TwemProxy Agent Started on: Fri Sep 20 2013 11:31:04 GMT+0000 (UTC)
[11:31:04] Subscribing to sentinel.
[11:31:04] Connection to Redis Sentinel established.
[11:31:04] Getting latest list of masters...
[11:31:04] Master received: redis1
[11:31:04] Updating Master redis1 to
[11:31:04] Master received: redis2
[11:31:04] Updating Master redis2 to
[11:31:04] Saving TwemProxy config
[11:31:04] TwemProxy restarted with output:
[11:31:04] Stopping ...
TwemProxy stopped
Starting TwemProxy server...
[11:09:40] Received switch-master: { details:
   { 'master-name': 'redis2',
     'old-ip': '',
     'old-port': '6379',
     'new-ip': '',
     'new-port': '6380' } }
[11:09:40] Nutcracker config updated with new content:
[11:09:40] twem1:
  hash: fnv1a_64
  distribution: modula
  auto_eject_hosts: false
  redis: true
  server_retry_timeout: 100
  server_failure_limit: 500
   - redis1
   - redis2

[11:09:40] Nutcracker restarted with output:
[11:09:40] Stopping ...
TwemProxy stopped
Starting TwemProxy server...

Script Configuration

The script configuration is simple really, edit the /lib/cli.js script and set the rather self explanatory configuration settings.

I hope this helps those of you who are tasked with setting up and configuring Redis!


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

Karl Stoney
Architect Hewlett Packard Enterprise Security Services
United Kingdom United Kingdom
Technical Architect for Hewlett-Packard Enterprise Security Service.

Please take the time to visit my site

You may also be interested in...

Comments and Discussions

QuestionHow to start TwemProxy AGent ! Pin
phamtienthinh14-Jan-14 16:58
memberphamtienthinh14-Jan-14 16:58 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Cookies | Terms of Use | Mobile
Web05 | 2.8.190306.1 | Last Updated 22 Sep 2013
Article Copyright 2013 by Karl Stoney
Everything else Copyright © CodeProject, 1999-2019
Layout: fixed | fluid