1 (edited by Clouseau 2012-03-18 19:53:15)

Topic: Upgrade from Debian Lenny to Squeeze

Hello,

I have to migrate one production server from Lenny to Squeeze next week. The version of iRedMail that is on the server is 0.6.1. I have a dillema on the procedure because from 0.6.1 was a lot of newer versions... So what is better to do:

1) upgrade first iRedMail to the latest version by following the procedures in wiki and then upgrade to Squeeze
2) upgrade to Squeeze and then upgrade iRedmail to latest version

Also when I upgrade iRedMail in option 1) or 2) can I just deploy the latest 0.7.4 package or I need to upgrade every single version: 0.6.1 ->0.7.0 -> 0.7.1->0.7.2->0.7.3 -> 0.7.4-rc3->0.7.4 ?

Thank you for your answer and advice smile

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2

Re: Upgrade from Debian Lenny to Squeeze

Clouseau wrote:

1) upgrade first iRedMail to the latest version by following the procedures in wiki and then upgrade to Squeeze
2) upgrade to Squeeze and then upgrade iRedmail to latest version

My personal suggestion is:
1) BACKUP BACKUP BACKUP. Backup all data first.
2) Upgrade Lenny to Squeeze, and make sure all mail services are working as expected.
3) Upgrade iRedMail.

Clouseau wrote:

Also when I upgrade iRedMail in option 1) or 2) can I just deploy the latest 0.7.4 package or I need to upgrade every single version: 0.6.1 ->0.7.0 -> 0.7.1->0.7.2->0.7.3 -> 0.7.4-rc3->0.7.4 ?

You HAVE TO upgrade every single version. 0.6.1 -> 0.7.0 -> 0.7.1 -> ...

3

Re: Upgrade from Debian Lenny to Squeeze

My personal suggestion is:
1) BACKUP BACKUP BACKUP. Backup all data first.
2) Upgrade Lenny to Squeeze, and make sure all mail services are working as expected.
3) Upgrade iRedMail.

The server is backuping on daily basis(Tivoli) smile

Great tnx smile

P.S after I posted, I found that the upgrades are not actually installing every newer version of package of iRedMail, I have to just do things in upgrade tutorials. I was lucky to read that because I was on my way on downloading the 0.7.0 version and starting to install it. Would be a bitter suprise to found that all of my data was overwriten : ))

Best regards,
Clouseau

4

Re: Upgrade from Debian Lenny to Squeeze

Ok, i have encoutered a problem. i have menaged to get it all work but the ireadmin 0.1.3 panel doesn't read settings.ini file corectly.

I have copied the settings.ini file from iRedAdmin 0.1.3 to iReadAdmin 0.1.6. and here is the problem when I create user:

- with 0.1.3. it gives this:

storageBaseDirectory: /var/vmail
mailMessageStore: vmail01/hidden.hr/p/pe/per/perostari-2012.03.24.12.20.50/
homeDirectory: /var/vmail/vmail01/hidden.hr/p/pe/per/perostari-2012.03.24.12.20.5

- with 0.1.6.
storageBaseDirectory: /var
mailMessageStore: vmail/hidden.hr/p/e/r/pero-2012.03.23.23.02.38/
homeDirectory: /var/vmail/hidden.hr/p/e/r/pero-2012.03.23.23.02.38/

This is my settings.ini file (I have left the less important parts):

[general]
# Site webmaster's mail address.

# Database backend: ldap.
backend = ldap

# Base directory used to store all mail data.
# You can find it via command:
#   $ sudo postconf virtual_mailbox_base
storage_base_directory = /var/vmail

# Storage node. Example: vmail1, vmail2, vmail3.
# Useful to serve large amount of accounts, store mailboxes on multiple
# hard disks.
# Full mailbox base directory will be:
#   ${storage_base_directory}/${storage_node}
storage_node = vmail01

# Mailbox type: maildir, mbox.
# maildir is recommend for better performance.
mailbox_type = maildir

# Hashed maildir: True, False.
# Example:
#   - hashed: domain.ltd/u/us/use/username-2009.09.04.12.05.33/
#   - non-hashed: domain.ltd/username-2009.09.04.12.05.33/
# hashed maildir is RECOMMEND for better performance.
hashed_maildir = True

# Default quota size (in MB). 0 means unlimited.
default_quota = 1024

# Default mta transport.
# Warning: Please DO use 'dovecot' as transport for iRedMail.
mtaTransport = 3306 - in settings.ini.ldap.sample here it says dovecot, should I replace this line with dovecot?

# Show user login date instead of created date: True, False.
# Note: iRedAdmin open source edition doesn't support this feature now.
show_login_date = False

[ldap]
# Settings used for LDAP backend.

# LDAP server uri.
# Warning: LDAPS is not supported yet.
uri = ldap://127.0.0.1:489/

# LDAP protocol version. 3 (LDAP v3) is recommend.
protocol_version = 3

# Trace level of LDAP operations: 0, 2, [...], 255.
# 0 is recommend for production use.
trace_level = 0

# LDAP suffix.
# basedn: dn which contains virtual domains.
# domainadmin_dn: dn which contains virtual domain admins.
basedn = o=domains,dc=hidden,dc=hr
domainadmin_dn = o=domainAdmins,dc=hidden,dc=hr

Somehow the new iRedAdmin 0.1.6 ignores the setting storage_node = vmail0, and just reads storage_base_directory = /var/vmail.

This is my postfix virtual_mailbox_base = /var/vmail

This is my entry in dovecot-ldap.conf
hosts           = 127.0.0.1:489
ldap_version    = 3
auth_bind       = yes
dn              = cn=vmail,dc=hidden,dc=hr
dnpass          = Lw8PlLzq1z4BVUivvuHD15eU1mzgjz
base            = o=domains,dc=agr,dc=hr
scope           = subtree
deref           = never
user_filter     = (&(objectClass=mailUser)(accountStatus=active)(enabledService=mail)(enabledService=%Ls%Lc)(|(mail=%u)(&(enabledService=shadowaddress)(shadowAddress=%u))))
pass_filter     = (&(objectClass=mailUser)(accountStatus=active)(enabledService=mail)(enabledService=%Ls%Lc)(|(mail=%u)(&(enabledService=shadowaddress)(shadowAddress=%u))))
pass_attrs      = mail=user,userPassword=password
default_pass_scheme = CRYPT
user_attrs      = mail=user,homeDirectory=home,mailMessageStore=mail=maildir:/var/vmail/%$/Maildir/,mailQuota=quota_rule=*:bytes=%$

I have gonne through tutoruials few times just to check if I have missed something but I did all well so maybe there is a bug in iRedAdmin 0.1.6? Or the storage_node is gone in new version?

I know I can fix it by:

1) just adding vmail01 to storage_base_directory = /var/vmail

2) writting a script that changes all the entries in LDAP by changing all the users I can see by slapcat -f /etc/ldap/slapd-iredmail.conf so that everyone have this:
storageBaseDirectory: /var
mailMessageStore: vmail/hidden.hr/etc.
homeDirectory: /var/vmail/hidden.hr/etc.
And moving users maildirs from /var/vmail/vmail01/hidden.hr/etc. to /var/vmail/hidden.hr/etc.

If storage_node gets ignored I'm gladely to do option 2) to have all the data unified and in the same format. I presume it will work with postfix, dovecot and roundcube and I don't even have to create symlinks for old location in example ln -s /var/vmail/hidden.hr /var/vmail/vmail01/hidden.hr?

Thanks a lot for your answer smile)

5

Re: Upgrade from Debian Lenny to Squeeze

With 0.1.6, there's no "storage_node = " anymore, so your "storage_base_directory =" must set to '/var/vmail/vmail01'.

6

Re: Upgrade from Debian Lenny to Squeeze

ZhangHuangbin wrote:

With 0.1.6, there's no "storage_node = " anymore, so your "storage_base_directory =" must set to '/var/vmail/vmail01'.

Ok, I have set that but the problem is that in settings.ini now is storage_base_directory = /var/vmail/vmail01

but when I add user slapcat says:

storageBaseDirectory: /var/vmail
mailMessageStore: vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47/
homeDirectory: /var/vmail/vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47/

So it doesn't equals the setting in settings.ini. I thought that those two settings have identical meaning but aparently they don't? That is a little confusing because now the storage_node is missing. My postfix conf is still virtual_mailbox_base = /var/vmail.

Now I have a lot of old users, which was created with old ireadmin, in my ldap that have:
storageBaseDirectory: /var/vmail/vmail01
mailMessageStore: agr.hr/j/jm/jmi/jmik-2010.05.18.23.55.58/
homeDirectory: /var/vmail/vmail01/agr.hr/j/jm/jmi/jmik-2010.05.18.23.55.

and the new ones:
storageBaseDirectory: /var/vmail
mailMessageStore: vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47/
homeDirectory: /var/vmail/vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47

It appears that droping storage_node setting has done some damage to the system in new iredadmin. And that was not written anywhere in the upgrades... So I don't know what to do. I have "fixed" it all with symlinks in /var/vmail/ but that is a dirty fix...

Am I missing something?

7 (edited by Clouseau 2012-04-14 21:05:12)

Re: Upgrade from Debian Lenny to Squeeze

Also the mailMessage store is also different in dovecot-ldap.conf than it is in LDAP

user_attrs      = mail=user,homeDirectory=home,mailMessageStore=mail=maildir:/var/vmail/%$/Maildir/,mailQuota=quota_rule=*:bytes=%$

LDAP - mailMessageStore: vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47/

So this %S in dovecot-ldap.conf is the mailMessageStore from LDAP and than is added in full string... That is very odd, nameing different things the same, it's confusing... Am I wrong or there is a bug?

What I think -  If the setting.ini should in example storage_base_directory = /var/vmail. Then the nevely created user should have this:

storageBaseDirectory: /var/vmail
mailMessageStore: agr.hr/t/e/s/test2-2012.04.14.14.40.06/
homeDirectory: /var/vmail/agr.hr/t/e/s/test2-2012.04.14.14.40.06/

but it has this:

storageBaseDirectory: /var
mailMessageStore: vmail/agr.hr/t/e/s/test2-2012.04.14.14.40.06/
homeDirectory: /var/vmail/agr.hr/t/e/s/test2-2012.04.14.14.40.06/

So this to me looks like a bug because storageBaseDirectory differs from the one that is in settings.ini...

8

Re: Upgrade from Debian Lenny to Squeeze

Clouseau wrote:

Ok, I have set that but the problem is that in settings.ini now is storage_base_directory = /var/vmail/vmail01

but when I add user slapcat says:

storageBaseDirectory: /var/vmail
mailMessageStore: vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47/
homeDirectory: /var/vmail/vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47/

So it doesn't equals the setting in settings.ini. I thought that those two settings have identical meaning but aparently they don't? That is a little confusing because now the storage_node is missing. My postfix conf is still virtual_mailbox_base = /var/vmail.

Actually, iRedAdmin splits '/var/vmail/vmail01' to two parts:
- /var/vmail: it's the same as storage_base_directory on old iRedAdmin.
- vmail01: it's the same as storage_node on old iRedAdmin.

Clouseau wrote:

Now I have a lot of old users, which was created with old ireadmin, in my ldap that have:
storageBaseDirectory: /var/vmail/vmail01
mailMessageStore: agr.hr/j/jm/jmi/jmik-2010.05.18.23.55.58/
homeDirectory: /var/vmail/vmail01/agr.hr/j/jm/jmi/jmik-2010.05.18.23.55.

and the new ones:
storageBaseDirectory: /var/vmail
mailMessageStore: vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47/
homeDirectory: /var/vmail/vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47

You can try below setting in dovecot-ldap.conf:

user_attrs      = mail=user,homeDirectory=home,=mail=maildir:~/Maildir/,mailQuota=quota_rule=*:bytes=%$

It will use value of home (actually, homeDirectory) as maildir path.

9

Re: Upgrade from Debian Lenny to Squeeze

ZhangHuangbin wrote:

Actually, iRedAdmin splits '/var/vmail/vmail01' to two parts:
- /var/vmail: it's the same as storage_base_directory on old iRedAdmin.
- vmail01: it's the same as storage_node on old iRedAdmin.

Yes, I saw that before, just the naming scheme is a little bit odd smile

ZhangHuangbin wrote:

You can try below setting in dovecot-ldap.conf:
user_attrs      = mail=user,homeDirectory=home,=mail=maildir:~/Maildir/,mailQuota=quota_rule=*:bytes=%$
It will use value of home (actually, homeDirectory) as maildir path.

I did a slapcat to ldif, changed all the values to this form:

storageBaseDirectory: /var/vmail
mailMessageStore: vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47/
homeDirectory: /var/vmail/vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47

and slpadd it. Now everything is uniformed and new users that are added through the latest iRedAdmin panel, also have that entries.

Also, there is something left in backup/restore tutorial for Openldap. I didn't run backup.sh script, did it on my own but restore procedure has some important part tha should be mentioned:

http://www.iredmail.org/wiki/index.php? … _LDIF_file

Yes, delete all the files in /var/lib/ldap/iredmail.org but leave DB_CONFIG because when you do a restore of ldif it will complain that it can't find DB_CONFIG and the restore procedure will give a poor performance (it will last very long) smile So, leave DB_CONFIG and the resore procedure  will be quicker (553 users and 25000 lines restored in 8min) smile

This is just a suggestion, I think that the way users are created should be (and the entry in settings.ini file should be  identical to main.cf postfix virtual_mailbox_base entry - storage_base_directory = /var/vmail/)

storageBaseDirectory: /var/vmail/
mailMessageStore: /var/vmail/agr.hr/t/e/s/test-2012.04.14.14.38.47/MailDir
homeDirectory: /var/vmail/vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47

And the dovecot-ldap.conf shoul then be:
user_attrs      = mail=user,homeDirectory=home,mailMessageStore=mail=maildir:%$/,mailQuota=quota_rule=*:bytes=%$

But I don't know how this would have effect on rest of the system, problably nothing, those are just entries in ldap and the dovecot should get the same values after these change smile It's just becase the nameing scheme in iredmail LDAP and in dovecot/postfix are a little bit odd and nameing is the same for different things... smile

Thank you for your support smile
Clouseau

10

Re: Upgrade from Debian Lenny to Squeeze

Clouseau wrote:

delete all the files in /var/lib/ldap/iredmail.org but leave DB_CONFIG

Updated in wiki tutorial, thanks for your feedback. smile

Clouseau wrote:

But I don't know how this would have effect on rest of the system, problably nothing, those are just entries in ldap and the dovecot should get the same values after these change smile It's just becase the nameing scheme in iredmail LDAP and in dovecot/postfix are a little bit odd and nameing is the same for different things... smile

It's not obvious, but i'm glad that you asked. smile

The short answer is, if you use only one Postfix transport, it should be fine with either setting. But if you have multiple transports, and they query maildir path with different LDAP filter which return different value, it will be messy: Mails will be delivered to two mailboxes, and user can see only one mailbox.

Currently, iRedMail uses Dovecot program 'deliver' as LDA (Local Deliver Agent) to deliver emails to mailboxes. Dovecot queres LDAP server (with /etc/dovecot/dovecot-ldap.conf) to get absolute path of mailbox, then delivers emails to mailbox, it will keep Dovecot index files (which will speed up IMAP access performence) up to date, and check mailbox quota, etc.

With 'mailMessageStore=mail=maildir:/var/vmail/%$/Maildir/' in dovecot-ldap.conf, and 'mailMessageStore=vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47/' in LDAP server, the final maildir path is: /var/vmail/vmail01/agr.hr/t/e/s/test-2012.04.14.14.38.47/.

If you want to use other LDAP, e.g. Postfix built-in transport, 'virtual', according to Postfix settings, virtual_mailbox_base=/var/vmail and virtual_mailbox_maps (it will return mailMessageStore=/var/vmail/agr.hr/t/e/s/test-2012.04.14.14.38.47/Maildir, as you suggested), you will get maildir path: /var/vmail/var/vmail/agr.hr/t/e/s/test-2012.04.14.14.38.47/Maildir with duplicate /var/vmail.

So, please keep mailMessageStore a relative path instead of an absolute path.

About storage_node, 'vmail1', in maildir path, my original purpose is, if your hard disk/storage is nearly full, you can mount another volume to store emails for new mail domains, e.g. create /var/vmail/vmail2, mount new volumes under /var/vmail/vmail2, change storage_base_directory to /var/vmail/vmail2, then all newly created mail users will has maildir path '/var/vmail/vmail2/xxxxx'.

P.S. 'virtual' has better performance while delivering emails, because, for example, it doesn't maintain mailbox indexes and check mailbox quota. But IMAP server (Dovecot) has to index mailbox for user when user accessing mailbox, and index mailbox each time there's new email arrived/delivered, deleted, moved, etc. It's kind of balance. 'virtual' will be useful if you just want to "archive" emails without accessing mailbox.

P.S. I found my English is poor each time i want to explain something with some more words... forgive me if you didn't understand above reply, and ask again. smile

11

Re: Upgrade from Debian Lenny to Squeeze

ZhangHuangbin wrote:

If you want to use other LDAP, e.g. Postfix built-in transport, 'virtual', according to Postfix settings, virtual_mailbox_base=/var/vmail and virtual_mailbox_maps (it will return mailMessageStore=/var/vmail/agr.hr/t/e/s/test-2012.04.14.14.38.47/Maildir, as you suggested), you will get maildir path: /var/vmail/var/vmail/agr.hr/t/e/s/test-2012.04.14.14.38.47/Maildir with duplicate /var/vmail.

So, please keep mailMessageStore a relative path instead of an absolute path.

Thanks a lot for your detailed explanation, now I get it.

Btw /etc/postfix/ldap_virtual_mailbox_maps.cf
#

server_host     = 127.0.0.1
server_port     = 489
....
result_attribute= mailMessageStore

That would made the dobule /var/vmail setting that you mentioned...

Also, thanks for a side tip, I will soon definitely need to add extra storage and this nice feature that you given in example is very helpful, just mount an change settings.ini for new users.

Great, keep up the good work smile