1 (edited by ekzsolt 2015-01-23 21:55:48)

Topic: Catchall and plus addressing

======== Required information ====
- iRedMail version: 0.9.0
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MariaDB
- Linux/BSD distribution name and version: Ubuntu 14.04 x64
- Related log if you're reporting an issue:
====

I have a catchall domain with many mailbox and a working catchall account. This is working fine. But some of my users wants to use plus-addressing. And the email sent to validmailbox+ext@domain address, delivered to the catchall mailbox always.

The plus-addressing is working on those domains, that do not have a catchall account. How do I use the two funcions together?

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2

Re: Catchall and plus addressing

Do you have below lines in /etc/postfix/master.cf? The most important setting in your case is '-m ${extension}'.

# Use dovecot deliver program as LDA.
dovecot unix    -       n       n       -       -      pipe
    flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${domain} -m ${extension}

Also, do you have 'recipient_delimiter = +' in /etc/postfix/main.cf? Show us output of below command please:

# postconf recipient_delimiter

3

Re: Catchall and plus addressing

Yes, both lines have. These are in the default config. The '-m ${extension}' parameter is just responsible for the delivery within user's mailbox to the corresponding folder. But its seems, that the catchall resolution takes effect first.

4

Re: Catchall and plus addressing

Does it work if you replace 'WHERE mailbox.username='%s' AND ...' by 'WHERE mailbox.username='%u@%d' AND ...' in /etc/postfix/mysql/virtual_mailbox_maps.cf?

Note: for testing purpose, you don't need to restart Postfix after changed this file, just test it with command:

# postmap -q validmailbox+ext@domain.com mysql:/etc/postfix/mysql/virtual_mailbox_maps.cf

5

Re: Catchall and plus addressing

No, it doesn't help. This postmap output is NULL with '%s' and ''%u@%d' too. The situation is the same with a domain, that doesn't have catchall setting.

Maybe this log can be interesting:

postfix/qmgr[5929]: 824D81C70B: from=<validmailbox@domain.com>, size=1413, nrcpt=1 (queue active)
amavis[1206]: (01206-01) Passed CLEAN {RelayedInternal}, MYNETS/MYUSERS LOCAL [192.168.122.1]:46326 [192.168.122.1] <validmailbox@domain.com> -> <validmailbox+ext@domain.com>, Queue-ID: 1DFE61C705, mail_id: QOqwdCjP8JFt, Hits: -0.86, size: 405, queued_as: 824D81C70B, dkim_new=dkim:domain.com, 332 ms
postfix/smtp[6130]: 1DFE61C705: to=<validmailbox+ext@domain.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.7, delays=1.3/0.02/0.01/0.4, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 824D81C70B)
postfix/qmgr[5929]: 1DFE61C705: removed
postfix/pipe[6138]: 824D81C70B: to=<catchall@domain.com>, relay=dovecot, delay=0.38, delays=0.06/0.01/0/0.3, dsn=2.0.0, status=sent (delivered via dovecot service)
postfix/qmgr[5929]: 824D81C70B: removed

In the end, the dovecot delivery gets the catchall address, but the validmailbox+ext would be the proper address.

6

Re: Catchall and plus addressing

Bug solved. Please try below steps (backup files before you modify them):

*) Open file /etc/postfix/mysql/virtual_alias_maps.cf, find 'query =' line like below:

query       = SELECT ... WHERE alias.address='%s' AND ...

*) Update it to below one:

query       = SELECT ... WHERE alias.address=CONCAT(SUBSTRING_INDEX('%u', '+', 1), '@%d') AND ...

*) Save your change and restart Postfix service.

7

Re: Catchall and plus addressing

Thank you, it works! The validmailbox+ext@domain.com style addressed email arrive into the validmailbox, not the catchall mailbox.

But the dovecot LDA/LMTP gets the goto address from the alias table, so the destination mailbox delivery (dovecot -m parameter) or the lmtp_save_to_detail_mailbox feature does not work. Even if the domain don't have a catchall address.

8

Re: Catchall and plus addressing

Dovecot doesn't query "mailbox.alias" table at all, why it gets the goto address?
Does it work if you replace mailbox.username='%u' in /etc/dovecot/dovecot-mysql.conf like my previous reply? (use %n for username part of email address)

9

Re: Catchall and plus addressing

According to Postfix document, virtual(5):

ADDRESS EXTENSION
       When a mail address localpart contains the optional recipient delimiter
       (e.g., user+foo@domain), the  lookup  order  becomes:  user+foo@domain,
       user@domain, user+foo, user, and @domain.

       The   propagate_unmatched_extensions   parameter  controls  whether  an
       unmatched address extension (+foo) is propagated to the result of table
       lookup.

So Postfix will query 'user+foo@domain' first. If you have catch-all account, it will return catchall address directly without trying 'user@domain'. Still no idea how to force Postfix to query 'user@domain' instead.

10

Re: Catchall and plus addressing

I just see that in the default iredmail 0.9.0 install, without catchall setting, the validmailbox+ext addressed email arrives to the ext imap folder. See the last line in the log (to=<validmailbox+ext@domain.com>):

postfix/qmgr[3259]: BB77A1CB18: from=<validmailbox@domain.com>, size=1421, nrcpt=1 (queue active)
amavis[1213]: (01213-01) Passed CLEAN {RelayedInternal}, MYNETS/MYUSERS LOCAL [192.168.122.1]:47838 [192.168.122.1] <validmailbox@domain.com> -> <validmailbox+ext@domain.com>, Queue-ID: 438D61CB12, mail_id: 5sJ7cZRKC0HB, Hits: -0.86, size: 409, queued_as: BB77A1CB18, dkim_new=dkim:domain.com, 389 ms
postfix/smtp[3597]: 438D61CB12: to=<validmailbox+ext@domain.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=2.3, delays=1.7/0.03/0.02/0.52, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as BB77A1CB18)
postfix/pipe[3605]: BB77A1CB18: to=<validmailbox+ext@domain.com>, relay=dovecot, delay=0.45, delays=0.07/0.01/0/0.37, dsn=2.0.0, status=sent (delivered via dovecot service)

But if I modify the virtual_alias_maps.cf file according to your suggestion (in reply 6), the email arrives to the Inbox, and the log's last line says dovecot gets only the validmailbox@domain.com address without +ext:

postfix/qmgr[17833]: 5A72F1CB5A: from=<validmailbox@domain.com>, size=1421, nrcpt=1 (queue active)
amavis[1214]: (01214-01) Passed CLEAN {RelayedInternal}, MYNETS/MYUSERS LOCAL [192.168.122.1]:47872 [192.168.122.1] <validmailbox@domain.com> -> <validmailbox+ext@domain.com>, Queue-ID: D65041CB54, mail_id: coocSQMmiOHC, Hits: -0.86, size: 409, queued_as: 5A72F1CB5A, dkim_new=dkim:domain.com, 412 ms
postfix/smtp[17934]: D65041CB54: to=<validmailbox+ext@domain.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.8, delays=1.3/0.01/0.02/0.46, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 5A72F1CB5A)
postfix/pipe[17944]: 5A72F1CB5A: to=<validmailbox@domain.com>, relay=dovecot, delay=0.35, delays=0.05/0.01/0/0.28, dsn=2.0.0, status=sent (delivered via dovecot service)

So the dovecot automatic detail mailbox delivery doesn't work.

11

Re: Catchall and plus addressing

Still no idea which files we should update to fix this issue. Sorry.
I will do some more testing to try it again, and don't wait for me, do some testing yourself too. smile

12

Re: Catchall and plus addressing

Where can I find documentation about postfix and dovecot variables (%s, %u, %d, etc.) which are I can use in SQL lookups?

13

Re: Catchall and plus addressing

Postfix:

- http://www.postfix.org/postconf.5.html# … alias_maps
- http://www.postfix.org/virtual.5.html

Dovecot: http://wiki2.dovecot.org/Variables

14

Re: Catchall and plus addressing

Hi,

Did anybody get any success with this? I've just had a client point out that this isn't working so it'd be great to have a fix.

Thanks,
Steve.

15

Re: Catchall and plus addressing

Yes, I found a possible solution, but it is not perfect. In main.cf I have deleted the "proxy:mysql:/etc/postfix/mysql/catchall_maps.cf" and "proxy:mysql:/etc/postfix/mysql/domain_alias_catchall_maps.cf" from virtual_alias_maps parameters, so this looks like now:

virtual_alias_maps = proxy:mysql:/etc/postfix/mysql/virtual_alias_maps.cf, proxy:mysql:/etc/postfix/mysql/domain_alias_maps.cf

And in the mysql vmail.alias table I modified the catchall record like this: address: @domain.com, goto: catchall@domain.com. Note to @ sign in front of the address filed. This two modifications solved my original problem. But it is not compatible with iredadmin. And if you have alias domains, on those catchall feature doesn't work, but for me it isn't problem.

16

Re: Catchall and plus addressing

Ah, thanks for the info.

I'm using LDAP so that complicates matters somewhat further.

17

Re: Catchall and plus addressing

For SQL backends, it should be possible if we modify SQL query to skip '+extension' part in email address (username+extension@domain.com), but for LDAP, looks like mission impossible.

18

Re: Catchall and plus addressing

Dear all, here's the final and PERFECT solution to solve this issue, it's applicable to both MySQL/MariaDB and PostgreSQL backends:

1) Open file /etc/postfix/mysql/catchall_maps.cf, find below line:

query       = ... WHERE alias.address='%d' ...

2) Add one more statement:

query       = ... WHERE alias.address='%d' AND '%u' NOT LIKE '%%+%%' ...

3) Restart Postfix.

That's all.

19

Re: Catchall and plus addressing

What about those with LDAP?

Thanks

20

Re: Catchall and plus addressing

fbifido wrote:

What about those with LDAP?

Unfortunately, no solution for LDAP.

We can tune SQL commands to avoid '+' in query string, but cannot do it in LDAP query filter. So, currently, not possible for LDAP backend, unless we change how we implement catchall account in LDAP.

21

Re: Catchall and plus addressing

I've had a look at this myself today.

As was said above, the problem on LDAP is that:

1. The "+" cannot be stripped from LDAP lookups easily, so the catchall eventually succeeds.
2. Postfix will try user+extension@domain on the lookups first, then user@domain after that, but before the catchall for user+extension@domain succeeds, it never tries the second pass.

Possible solutions include stripping out extensions before doing the lookup - but that's probably going to obfuscate delivery, as the headers might get rewritten to hide the extension. Alternatively, if one could change the virtual_alias_maps list between iterations of lookup, that'd solve it (first pass without catchall, second pass including catchall). However, I don't think that's possible either.

Actually, a final option would be not to include a catchall lookup at all. Postfix, will hit the alias_maps in the following order normally:

user@domain, user, @domain

and with extension:

user+ext@domain, user@domain, user+ext, user, @domain

So surely if the LDAP entries for catchalls had similar service enablement and object classes to aliases, the normal postfix lookup process would work for plus addressing and for catchall?

The key differences I spot are, for catchall:

query_filter    = (&(objectClass=mailUser)(accountStatus=active)(|(mail=@%d)(shadowAddress=@%d)))
result_attribute= mailForwardingAddress

and for alias:

query_filter    = (&(|(mail=%s)(shadowAddress=%s))(accountStatus=active)(enabledService=mail)(enabledService=deliver)(|(objectClass=mailAlias)(&(objectClass=mailUser)(enabledService=forward))))
result_attribute= mailForwardingAddress

So some careful re-wording of that alias filter might be required. Or maybe there's a different map lookup that I'm missing, that would be a better candidate?

22

Re: Catchall and plus addressing

Hi stevekez,

Thanks for helping. Could you please point out which LDAP query files need to be modified? A diff/patch is recommended.

23

Re: Catchall and plus addressing

I haven't had any chance to test this myself yet, and would need to spin up a development VM in which to do it. It'll probably be a few days before I can do that.

24

Re: Catchall and plus addressing

OK, just ping me if you need some help.

25 (edited by stevekez 2015-05-27 02:21:41)

Re: Catchall and plus addressing

OK, in a small scale test on a custom VM it appears to work. I didn't install iRedAdmin Pro, but it shouldn't make a difference. I made an assumption about LDAP that may not be correct, but seemed like the least invasive solution. Details below.

====
Catch-all addresses in LDAP.

I've added "enabledService: catchall" allowing me to distinguish between this special service and accounts that are also mail accounts but need to have "deliver" etc enabled for the lookup to succeed. I hope this is a reasonable thing to add.
====
Virtual alias maps now have a different lookup:

$ diff virtual_alias_maps-old.cf virtual_alias_maps.cf 
10c10
< query_filter    = (&(|(mail=%s)(shadowAddress=%s))(accountStatus=active)(enabledService=mail)(enabledService=deliver)(|(objectClass=mailAlias)(&(objectClass=mailUser)(enabledService=forward))))
---
> query_filter    = (&(|(mail=%s)(shadowAddress=%s))(accountStatus=active)(|(enabledService=catchall)(&(enabledService=mail)(enabledService=deliver)(|(objectClass=mailAlias)(&(objectClass=mailUser)(enabledService=forward))))))

====
Catchall is now superfluous, so not used:

$ diff main-old.cf main.cf
83c83
< virtual_alias_maps = proxy:ldap:/etc/postfix/ldap/virtual_alias_maps.cf, proxy:ldap:/etc/postfix/ldap/virtual_group_maps.cf, proxy:ldap:/etc/postfix/ldap/virtual_group_members_maps.cf, proxy:ldap:/etc/postfix/ldap/catchall_maps.cf
---
> virtual_alias_maps = proxy:ldap:/etc/postfix/ldap/virtual_alias_maps.cf, proxy:ldap:/etc/postfix/ldap/virtual_group_maps.cf, proxy:ldap:/etc/postfix/ldap/virtual_group_members_maps.cf

====

Similar changes may need to be applied to group maps in order for this to work for all cases.

I did a *small* test that verified delivery to catchalltarget@ randomaddressforcatchall@ and username+1234@ all delivered correctly.

Please note that this post is for the benefit of ZhangHuangbin, and not a recommended configuration change yet!