constantly get into failover-abort-not-elected state for the sentinel

[dennis bing leng]

Hi All

I have configured the our redis server as below:
<Loading Image...>
The way I have configured is that master and slave are physically
separated, I have two sentinels, one is on the master box, the other one is
on the slave box. the current version of redis on these boxes are 2.8.6.
the failover works in the our DEV environment. But when I use it in the QA
environment with the same configuration. it did not work as i am keep
getting failover-abort-not-elected state during the failover process (e.g.
I reboot the master). the current quorum I use is 1 on both sentinels.

the current master ip is 100.0.0.2, the slave is 100.0.0.1

when I reboot master (100.0.0.2), here is the log file from sentinel on
100.0.0.1

[1515] 14 Feb 02:17:32.176 # +sdown master mymaster 100.0.0.2 6381
[1515] 14 Feb 02:17:32.177 # +odown master mymaster 100.0.0.2 6381 #quorum
1/1
[1515] 14 Feb 02:17:32.177 # +new-epoch 5
[1515] 14 Feb 02:17:32.177 # +try-failover master mymaster 100.0.0.2 6381
[1515] 14 Feb 02:17:32.177 # +vote-for-leader
a665c1457fcd575418ae52a965d5ec389fb9b835 5
[1515] 14 Feb 02:17:42.217 # -failover-abort-not-elected master mymaster
100.0.0.2 6381
[1515] 14 Feb 02:17:45.691 # +sdown sentinel 100.0.0.2:26381 100.0.0.2
26381 @ mymaster 100.0.0.2 6381

I was expecting that the sentinel on slave will get elected to be the
leader then trigger the failover process. Did i misconfigure it?

Thank you so very much.

Bing

[Aleksey Didik]

Hello Bing,

Did you actually find the answer?
I have the same problem now.

Aleksey.

Post by dennis bing leng
Hi All
<https://lh4.googleusercontent.com/-_oRVF-fvpEI/Uv566PutcXI/AAAAAAAAAXc/DVy6vAiFEVI/s1600/Sentinel.jpg>
The way I have configured is that master and slave are physically
separated, I have two sentinels, one is on the master box, the other one is
on the slave box. the current version of redis on these boxes are 2.8.6.
the failover works in the our DEV environment. But when I use it in the QA
environment with the same configuration. it did not work as i am keep
getting failover-abort-not-elected state during the failover process (e.g.
I reboot the master). the current quorum I use is 1 on both sentinels.
the current master ip is 100.0.0.2, the slave is 100.0.0.1
when I reboot master (100.0.0.2), here is the log file from sentinel on
100.0.0.1
[1515] 14 Feb 02:17:32.176 # +sdown master mymaster 100.0.0.2 6381
[1515] 14 Feb 02:17:32.177 # +odown master mymaster 100.0.0.2 6381 #quorum
1/1
[1515] 14 Feb 02:17:32.177 # +new-epoch 5
[1515] 14 Feb 02:17:32.177 # +try-failover master mymaster 100.0.0.2 6381
[1515] 14 Feb 02:17:32.177 # +vote-for-leader
a665c1457fcd575418ae52a965d5ec389fb9b835 5
[1515] 14 Feb 02:17:42.217 # -failover-abort-not-elected master mymaster
100.0.0.2 6381
[1515] 14 Feb 02:17:45.691 # +sdown sentinel 100.0.0.2:26381 100.0.0.2
I was expecting that the sentinel on slave will get elected to be the
leader then trigger the failover process. Did i misconfigure it?
Thank you so very much.
Bing

[Matt Stancliff]

The way I have configured is that master and slave are physically separated, I have two sentinels, one is on the master box, the other one is on the slave box.

First note: Sentinel is a distributed system driven by consensus. It’s a good idea to have more Sentinel servers than Redis servers (you can even run Sentinel processes on servers without Redis—the more Sentinels the better!)

it did not work as i am keep getting failover-abort-not-elected state during the failover process (e.g. I reboot the master). the current quorum I use is 1 on both sentinels.

You _can_ use a quorum of 1 (not recommended), but you must have a _majority_ of Sentinels voting in the election. A majority is defined as half your Sentinels plus one more (50% + 1).

If you only have two Sentinels and one goes down, you no longer have a majority, so nothing can be elected. (50% of 2 = 1. 1 + 1 = 2. No majority possible.)

You can set quorum to 0 (also not recommended), but then any Sentinel server can decide actions for your Redis setup without consulting other Sentinel servers.

Also see the “Quorum” section of http://redis.io/topics/sentinel



-Matt