1 Topic by Radapompa

  • Radapompa
  • Member
  • Offline
  • Registered: 2016-05-18
  • Posts: 20

Topic: Local dovecot when using shared mailbox

============ Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.5-1
- Linux/BSD distribution name and version: Debian 8
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): LDAP
- Web server (Apache or Nginx): Nginx
- Manage mail accounts with iRedAdmin-Pro? No
- Related log if you're reporting an issue:

I've been having issue with users getting corrupted dovecot indexes. Now I've found that you should put the dovecot index on the local machine instead of in the network share. So I found an old post describing how to do this.

So I've created the folder /var/lib/dovecot/indexes and set the permissions for vmail:vmail to the folder with 700 as permissions. I've changed the mail_location to following in /etc/dovecot/dovecot.conf

mail_location = maildir:%Lh/Maildir/:INDEX=/var/lib/dovecot/indexes/%Ld/%Lu/

But I'm curios about the namespace called shared. Today the location is:

location = maildir:%%Lh/Maildir/:INDEX=%%Lh/Maildir/Shared/%%u

So I assume that this also should be changed. So would the following be correct?

location = maildir:%%Lh/Maildir/:INDEX=/var/lib/dovecot/indexes/%%Ld/%%Lu/Shared/%%u

Also, wouldn't it be good have this as default in iRedMail? Then it wouldn't be an hassle for those that chooses to have an shared storage.

Thanks for an great project!

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: Local dovecot when using shared mailbox

*) The global "mail_location" doesn't require change.
*) iRedMail-0.9.5-1 has "location =" setting for shared folders like this:

namespace {
    type = shared
    separator = /
    prefix = Shared/%%u/
    location = maildir:%%Lh/Maildir/:INDEX=%%Lh/Maildir/Shared/%%u

    # This is setting used in the latest development edition. (Shared/<domain>/<user>)
    #location = maildir:%%Lh/Maildir/:INDEX=%%Lh/Maildir/Shared/%%Ld/%%Ln

    # this namespace should handle its own subscriptions or not.
    subscriptions = yes
    list = children
}

As you can see, indexes for shared folders are placed under ".../Maildir/Shared/<email>"

For user's own mailbox, index are placed under the mailbox with below (global) setting:

mail_location = maildir:%Lh/Maildir/:INDEX=%Lh/Maildir/

3 Reply by Radapompa

  • Radapompa
  • Member
  • Offline
  • Registered: 2016-05-18
  • Posts: 20

Re: Local dovecot when using shared mailbox

Thanks,
But let me rephrase the question.

I have a clustered environment with three mail servers. The /var/vmail folder is in a gluster share which is replicated between these three mail servers. Now since dovecot index exists in /var/vmail/vmail1/.../Maildir, it gets corrupted since all three mail servers wants to write in the same index file.
I've read that you should then move the index to the local machine instead of having it in the gluster share. As explained in this forum post: http://www.iredmail.org/forum/topic3560 … orage.html

But the above forum post dosen't address the namespace called shared. Which is why I'm asking if below would be correct

location = maildir:%%Lh/Maildir/:INDEX=/var/lib/dovecot/indexes/%%Ld/%%Lu/Shared/%%u

If I use following for default mail_location

mail_location = maildir:%Lh/Maildir/:INDEX=/var/lib/dovecot/indexes/%Ld/%Lu/

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: Local dovecot when using shared mailbox

I think you're right, we should stored index for shared mailboxes locally. Will fix it in iRedMail immediately.

5 Reply by Radapompa

  • Radapompa
  • Member
  • Offline
  • Registered: 2016-05-18
  • Posts: 20

Re: Local dovecot when using shared mailbox

Nice!
But, I've tried to change mail_location directive. And I can't see the new index. It seems that it still uses the index in /var/vmail/vmail1/.../Maildir. So is it using the dovecot-ldap.conf to define where it should put the index?
If I understood correctly from Dovecot documentation. The userdb directive overrides the mail_location directive (see http://wiki.dovecot.org/MailLocation). So in my case, it uses the dovecot-ldap.conf.
But I can't find any directive here for index. So not sure how this works.

6 Reply by Radapompa

  • Radapompa
  • Member
  • Offline
  • Registered: 2016-05-18
  • Posts: 20

Re: Local dovecot when using shared mailbox

Radapompa wrote:

Nice!
But, I've tried to change mail_location directive. And I can't see the new index. It seems that it still uses the index in /var/vmail/vmail1/.../Maildir. So is it using the dovecot-ldap.conf to define where it should put the index?
If I understood correctly from Dovecot documentation. The userdb directive overrides the mail_location directive (see http://wiki.dovecot.org/MailLocation). So in my case, it uses the dovecot-ldap.conf.
But I can't find any directive here for index. So not sure how this works.

Answered my question myself. Yes, you need to update dovecot-ldap.conf. Below is the new user_attrs, if you use the same path for index as I did

user_attrs      = mail=master_user,mail=user,homeDirectory=home,=mail=maildir:~/Maildir/:INDEX=/var/lib/dovecot/indexes/%Ld/%Lu/,mailQuota=quota_rule=*:bytes=%$

7 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: Local dovecot when using shared mailbox

I don't think you need to update /etc/dovecot/dovecot-ldap.conf (or dovecot-mysql.conf).

*) For INDEX of user's mailbox, it's defined in "mail_location =" in dovecot.conf globally.
*) For INDEX of shared mailbox, it's defined in "location =" in "namespace {}" (for shared folder) in dovecot.conf.

Just modifying those 2 should be enough.

8 Reply by Radapompa (edited by Radapompa 2016-10-27 19:13:29)

  • Radapompa
  • Member
  • Offline
  • Registered: 2016-05-18
  • Posts: 20

Re: Local dovecot when using shared mailbox

ZhangHuangbin wrote:

I don't think you need to update /etc/dovecot/dovecot-ldap.conf (or dovecot-mysql.conf).

*) For INDEX of user's mailbox, it's defined in "mail_location =" in dovecot.conf globally.
*) For INDEX of shared mailbox, it's defined in "location =" in "namespace {}" (for shared folder) in dovecot.conf.

Just modifying those 2 should be enough.

It didn't work when I only updated these two settings. According to Dovecot documentation (http://wiki.dovecot.org/MailLocation), the mail userdb field overrides mail_location. Since userdb points to dovecot-ldap.conf, and you have the location set in user_attrs, it will take the settings there prior to mail_location.
So for the future, the dovecot-ldap.conf needs to be changed. Either by adding INDEX as in my previous post. Or remove =mail=maildir:~/Maildir/ from the configuration (not tested).

9 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: Local dovecot when using shared mailbox

Radapompa wrote:

the mail userdb field overrides mail_location

That's true, but we don't define INDEX in userdb. And it works during my testing last night. Maybe there's something wrong with my or your testing?

10 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: Local dovecot when using shared mailbox

UPDATE: Confirmed that no need to update dovecot-{ldap,mysql,pgsql}.conf.

My test settings:

mail_location = maildir:%Lh/Maildir/:INDEX=/var/vmail-indexes/%Ld/%Ln/

namespace {
    type = shared
    location = maildir:%%Lh/Maildir/:INDEX=/var/vmail-indexes/%Ld/%Ln/Shared/%%Ld/%%Ln
    ...
}

11 Reply by MattHo

  • MattHo
  • Member
  • Offline
  • Registered: 2017-06-06
  • Posts: 34

Re: Local dovecot when using shared mailbox

==== Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.7
- Linux/BSD distribution name and version: CentOS 7 64Bit
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Web server (Apache or Nginx): Nginx
- Manage mail accounts with iRedAdmin-Pro? Yes
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

Hi Zhang,

I got the same problem, I build a storage cluster which running Glusterfs and the dovecot index crashed always.

I found your message above (#4) said "..we should stored index for shared mailboxes locally. Will fix it in iRedMail immediately.". May I know v0.9.7 fixed?

If not, may I have your advise which config file I need to amend to change the index file store locally which I'm running MySQL.

Many thanks.

Matt

12 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: Local dovecot when using shared mailbox

Hi Matt,

We better move to a new forum topic.

1) If you use only one Dovecot server, storing index on shared storage should be fine.
2) If you have multiple Dovecot  servers, and all of them may access same mailbox, then the index must be stored in a different place to avoid updating by multiple server, otherwise the index files will be a mess.

Which situation do you have? 1 or 2?

In iRedMail load-balance cluster, we store indexes locally under /var/vmail-indexes, but with a standalone iRedMail installation, it's stored in mailbox.

TIP: If you can define the index directory while installing iRedMail like this:

MAILBOX_INDEX_DIR=/var/vmail-indexes bash iRedMail.sh

Then iRedMail will use /var/vmail-indexes in Dovecot config files.

13 Reply by MattHo

  • MattHo
  • Member
  • Offline
  • Registered: 2017-06-06
  • Posts: 34

Re: Local dovecot when using shared mailbox

Hi Zhang,

I should create another new post or you can help move it?

BTW, we are situation 2.
As our system is already installed and running, could you please advise how can we change the index location?

Many thanks.

Matt

14 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: Local dovecot when using shared mailbox

The solution is mentioned in #10 reply of current thread. big_smile (path in "INDEX=")
https://forum.iredmail.org/post52296.html#p52296

BTW, would you mind sharing some detailed technical info of your iRedMail cluster?

15 Reply by MattHo

  • MattHo
  • Member
  • Offline
  • Registered: 2017-06-06
  • Posts: 34

Re: Local dovecot when using shared mailbox

I updated the config file and now all index files are store locally, the error message is gone.
Many thanks Zhang.

Sure, I'm happy to share our cluster details. Our cluster is simple, just 2 servers right now.

We are follow the structure as below but we are running MySQL.
https://docs.iredmail.org/haproxy.keepa … terfs.html

We skipped to setup HA-Proxy as we were make all connections to our 1st server, 2nd server is just a backup for any reason the 1st server is down.
We created 2 MX record, 1st & 2nd server. If 1st server is down, we can just switch the IP to 2nd Server and no mail will be lose.

The cluster ran for a month ready, only few errors occurred (SOGO connection, dovecot connection and index error).

If anyone have questions or comment, please let me know.

Regards,
Matt

16 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: Local dovecot when using shared mailbox

Am i right that you're running setup like below?

- Running GlusterFS on 2 servers as shared mailbox + data storage.
- MySQL master-slave replication. Or, master-master?

What data do you store on GlusterFS?

17 Reply by MattHo

  • MattHo
  • Member
  • Offline
  • Registered: 2017-06-06
  • Posts: 34

Re: Local dovecot when using shared mailbox

Sorry for my late reply.

Yes Zhang, we are running 3 GlusterFS on 3 Servers (A, B & C) which A & B servers are doing the mail server, C server act as a file storage backup. These 3 servers just cluster the vmail folder.
For the MySQL, we are master-master on A & B server only. The structure like below:

A Server - Installed iRedmail with MySQL & Gluster, master mail server
B Server - Installed iRedmail with MySQL & Gluster, slave mail server. If A server is down then we will switch to this server.
C Server - Just installed Gluster, this server is doing a file backup batch jobs or other log analysis process.

A & B are 2 CPU power server with 16G Ram, C just a workstation with large HDD Space.

18 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: Local dovecot when using shared mailbox

MattHo wrote:

For the MySQL, we are master-master on A & B server only. The structure like below:

With only 2 servers, it's not a trustable master-master cluster. You need at least 3 MySQL servers. Search "mysql cluster split brain" with Google.

You already have 3 servers for GlusterFS, why not run MySQL on all of them to get a solid MySQL master-master cluster?

By the way, we have a working load-balance iRedMail cluster solution in production for few months, all setup done with Ansible. We will announce it soon. If you're interested in this solution, feel free to contact us. big_smile

19 Reply by MattHo

  • MattHo
  • Member
  • Offline
  • Registered: 2017-06-06
  • Posts: 34

Re: Local dovecot when using shared mailbox

Yes, first of all we only have 2 servers for setup a master-master cluster. That 3rd server is added couple week later.
BTW, we are now thinking do we need to move the MySQL to the cloud server (Single point) or create another MySQL Cluster (3 workstations) to handle it although this master-master working well. We still looking for the online documents to find out which is the best way.

BTW, yes I'm interest for your cluster solution. May I know more information?

20 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: Local dovecot when using shared mailbox

MattHo wrote:

BTW, we are now thinking do we need to move the MySQL to the cloud server (Single point) or create another MySQL Cluster (3 workstations) to handle it although this master-master working well.

PERSONALLY, i think single SQL server will be easier to handle than your 2-server (so called) master-master cluster. if this master-master got 'split-brain' issue, it will be painful and take more time to recover it, and of course your mail services will be completely down because Postfix/Dovecot cannot query SQL server.

Brief introduction of the architecture of our load-balance solution: it requires at least 5 servers:

- 2 frontend servers used to balance all incoming mail traffic.
- 3 servers for MariaDB master-master cluster and iRedMail servers. If you want to use existing MySQL/MariaDB cluster or single server, it's ok too. For example, a remote Amazon RDS database.
- If you don’t have NAS/SAN/ISCSI remote storage as mailbox, we can setup 1 server to run NFS service as remote mailbox storage.
- If you’re on a tight budget, we can deploy iRedMail on MariaDB cluster nodes, but it’s recommended to keep MariaDB servers separated for easier maintenance and troubleshooting. Also separate smtp, pop3/imap, webmail services to different servers.

Currently it supports Amazon AWS EC2 (with Elastic IP support, backend servers in different Availability Zones but in same data center), and physical servers, or of course the virtual machines built with VMware/VirtualBox/Xen/KVM/... (all cluster nodes in same network).

For server OS, it works with CentOS 7, Debian 9, Ubuntu 16.04 LTS. We tested more with Ubuntu till now, but all 3 linux distributions are working fine.

We have this load-balance cluster solution deployed in production for few months, it's working fine, and we're still actively improving it - just like iRedMail.

Feel free to contact me (zhb _at_ iredmail _dot_ org) if you want to discuss more details, even if you have no plan to use our solution. smile