1 Topic by djbon2112 (edited by djbon2112 2014-07-29 13:33:18)

  • djbon2112
  • Member
  • Offline
  • Registered: 2014-05-04
  • Posts: 5

Topic: Advice for running a master-master iRedMail cluster

Hello everyone!

I love iRedMail, and have been using it since April last year to host my personal e-mail. I *could* set up the whole thing myself from scratch, but iRedMail saves me an immense amount of time and energy! My single-node has been running fine, but my physical server setup features two machines, and I want to increase my reliability.

I'm interested in setting up a full high-availability cluster of two iRedMail instances, on Debian Wheezy VMs, which will operate in a master-master configuration. I wish to use LDAP as the authentication backend, ruling out all of the guides for this I've seen so far. This will be behind a Linux IPVS load balancer setup so that all connections on as many protocols as possible are distributed evenly between the two VMs, and a failed VM is automatically taken out of rotation (should I need to take down a physical node for maintenance, or due to a hardware or software failure of either node).

I've identified the following elements which need to be shared, and my solutions in [square brackets] for those that I already have:

vmail directories [use a shared NFS mountpoint containing Maildirs]
MySQL databases [use a separate MySQL cluster with a single load-balanced connection IP]
LDAP databases []
Dovecot configuration []

Everything else should work on either VM on its own and not need any explicit synchronization as far as I can tell.

I've been working first on LDAP syncrepl master-master setups, but I can't seem to get the standard LDAP HA guides to work with the iRedMail LDAP configuration. I haven't tried anything with Dovecot yet since LDAP is my first priority. I have a plan of attack to implement the first two with the solutions provided, but I have not actually begun yet.

Has anyone set something like this up, and are able to offer any advice? Is there anything I've missed?

Once I get this working I plan to post an in-depth guide on how I did it, so hopefully I can make it work!

==== Required information ====
- iRedMail version: latest
- Store mail accounts in which backend (LDAP/MySQL/PGSQL):  LDAP
- Linux/BSD distribution name and version: Debian Wheezy amd64
- Related log if you're reporting an issue: N/A
====

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: Advice for running a master-master iRedMail cluster

djbon2112 wrote:

MySQL databases [use a separate MySQL cluster with a single load-balanced connection IP]
...
I've been working first on LDAP syncrepl master-master setups, but I can't seem to get the standard LDAP HA guides to work with the iRedMail LDAP configuration.

How about running OpenLDAP on that MySQL cluster server? then setup your iRedMail server to run OpenLDAP as slave. One master, multiple slaves.

3 Reply by djbon2112

  • djbon2112
  • Member
  • Offline
  • Registered: 2014-05-04
  • Posts: 5

Re: Advice for running a master-master iRedMail cluster

I suppose I could do so, but I still end up with a master-master setup there (the two MySQL hosts are actually a MariaDB Galera cluster) and then adding iRedMail's LDAP as a slave just makes it more complex. I suppose I can continue messing around with the current LDAP implementation to see if I can get it working right. I'm in a testing phase anyways so I can mess around and not worry about losing anything!

ZhangHuangbin wrote:
djbon2112 wrote:

MySQL databases [use a separate MySQL cluster with a single load-balanced connection IP]
...
I've been working first on LDAP syncrepl master-master setups, but I can't seem to get the standard LDAP HA guides to work with the iRedMail LDAP configuration.

How about running OpenLDAP on that MySQL cluster server? then setup your iRedMail server to run OpenLDAP as slave. One master, multiple slaves.

4 Reply by djbon2112

  • djbon2112
  • Member
  • Offline
  • Registered: 2014-05-04
  • Posts: 5

Re: Advice for running a master-master iRedMail cluster

Hello again:

I think I got this solved, at least for the backend. All the SQL is on the cluster, LDAP is a multi-master sync between mail1 and mail2, and webmail seems to be working. iRedAdmin, however, does not: it's just returning "Error: Session expired, please re-login." each time I try to log in with my "postmaster" admin account. This happens on both servers. I've enabled 'debug = True' in iredadmin/settings.py, but I don't see anything different in my Apache2 logs (Debian). Any suggestions?

5 Reply by djbon2112 (edited by djbon2112 2014-08-18 07:43:28)

  • djbon2112
  • Member
  • Offline
  • Registered: 2014-05-04
  • Posts: 5

Re: Advice for running a master-master iRedMail cluster

Turns out this was because I was using the wrong (read: default) load-balancing method for my Database servers in IPVS; Roundcube seemed not to mind but iRedAdmin did. I changed it out to a better one (from wrr to sh) and magically everything is working. I'm going to do a bit more testing, but things look promising! And once I get everything migrated I'll write up a complete guide for anyone else looking to do this (a little crazy) super-HA iRedMail cluster.

djbon2112 wrote:

Hello again:

I think I got this solved, at least for the backend. All the SQL is on the cluster, LDAP is a multi-master sync between mail1 and mail2, and webmail seems to be working. iRedAdmin, however, does not: it's just returning "Error: Session expired, please re-login." each time I try to log in with my "postmaster" admin account. This happens on both servers. I've enabled 'debug = True' in iredadmin/settings.py, but I don't see anything different in my Apache2 logs (Debian). Any suggestions?

6 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: Advice for running a master-master iRedMail cluster

About iRedAdmin issue, you can try to change below value in file libs/iredbase.py:

web.config.session_parameters['ignore_change_ip'] = True

Looking forward to your complete guide. smile

7 Reply by djbon2112 (edited by djbon2112 2014-09-21 01:16:04)

  • djbon2112
  • Member
  • Offline
  • Registered: 2014-05-04
  • Posts: 5

Re: Advice for running a master-master iRedMail cluster

ZhangHuangbin wrote:

About iRedAdmin issue, you can try to change below value in file libs/iredbase.py:

web.config.session_parameters['ignore_change_ip'] = True

Looking forward to your complete guide. smile

That setting ended up not helping, but I was able to reproduce the problem on-demand by changing the IPVS scheduler for MySQL traffic, so I'm convinced that was the problem.

I've finished the cluster, did a few weeks of testing, and am now a full week into using it for all my domains. No problems so far! As promised, here's the guide in plain-text. Hopefully this will help others, and please feel free to post it as desired (it's under the GNU FDL).

http://pastebin.com/JcYeQBrX

8 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: Advice for running a master-master iRedMail cluster

I added your article in iRedMail web site:
http://www.iredmail.org/doc.html#integration

== UPDATE ==

Announced your tutorial in Twitter (@iredmail) and our mailing list: https://groups.google.com/forum/#!forum/iredmail

9 Reply by Kali (edited by Kali 2014-09-21 20:45:09)

  • Kali
  • Member
  • Offline
  • Registered: 2014-09-21
  • Posts: 2

Re: Advice for running a master-master iRedMail cluster

nice reading, but i dont like the handling of ARP

one of the major problem of DR is the problem of ARP Poisoning
http://www.austintek.com/LVS/LVS-HOWTO/ … oblem.html

In the proposed configuration balanced server do not announce the MAC address of the loopback interface to prevent the ARP Poisoning
Instead of setup the VIP on a loopback alias and hide the MAC address, ill prefer to use linux firewall on the balanced server to accept packet for the VIP (with a NAT) in that way the ARP problem is completly prevented at cost of small cpu increase (unnoticeable)