1 (edited by dswartz 2015-11-17 00:39:40)

Topic: External emails treated as internal?

==== Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.2
- Linux/BSD distribution name and version: CentOS 6.6
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): LDAP
- Web server (Apache or Nginx): Apache
- Manage mail accounts with iRedAdmin-Pro? No
- Related log if you're reporting an issue: /var/log/maillog
====

Spamassassin and amavisd were working just fine (emails being scored and such).  I wanted to bypass spam checking for internal hosts, so I set 'bypass_spam_checks_maps => [1],' in /etc/amavisd/amavisd.conf, but now, no messages are being scored, even from external senders.  This is in the section that says:

# Apply to mails which coming from internal networks or authenticated users.   
# mail supposedly originating from our users                                   
$policy_bank{'MYUSERS'} = {

Looking at a test email I sent from my work address:

Nov 16 11:28:58 iredmail postfix/qmgr[32761]: 29BE62C0750: from=<XXXXX>, size=6821, nrcpt=1 (queue active)
Nov 16 11:28:58 iredmail postfix/smtpd[26759]: disconnect from iredmail.druber.com[127.0.0.1]
Nov 16 11:28:58 iredmail amavis[26732]: (26732-01) Passed CLEAN {RelayedInternal}, MYUSERS LOCAL [208.53.48.218]:29713 [157.56.110.146] <XXX> -> <dswartz@druber.com>, Queue-ID: CD9A02C06EA, Message-ID: <BY1PR0801MB0903FEA66981D3D0776F0166921E0@BY1PR0801MB0903.namprd08.prod.outlook.com>, mail_id: U8YUPz1f_nix, Hits: -, size: 6391, queued_as: 29BE62C0750, 124 ms

You can see 'Hits: -' indicating scoring disabled.  What I don't understand is the 'Passed CLEAN {RelayedInternal}, MYUSERS LOCAL'.  My iredmail server is sitting behind a NAT firewall, is this related?  I don't remember having made any changes that would affect this, but I could be wrong.  Any thoughts appreciated...

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2

Re: External emails treated as internal?

Everything I've seen/read online seems to say that this policy bank is for a local sender, not a local recipient. It's almost like amavis is confusing the sender with the recipient.

3

Re: External emails treated as internal?

Interesting, I changed this line:

@local_domains_maps = 1;

to this:
                                                     
@local_domains_maps = ['tld.com'];

and now it works?  e.g. an outside email says:

Hits: 0.974

whereas an email sent from an inside host says:

Hits: -

4

Re: External emails treated as internal?

This is a known bug, and will be fixed in upcoming iRedMail-0.9.3.
Here's the tutorial to fix it:
http://www.iredmail.org/docs/upgrade.ir … ernal-user
Note: This upgrade tutorial for iRedMail-0.9.3 is still a draft, do not apply other steps mentioned in above page.

5

Re: External emails treated as internal?

Ah, thanks!

6

Re: External emails treated as internal?

Okay, I made the change.  I then did three emails:

1. one using the local mail client, mailing to a recipient in my domain.
2. one to an outside recipient.
3. one from my outside (work) email to my home email.

All 3 show as 'Passed CLEAN {RelayedInbound}'.  I suspect #1 and #2 are because the 'mail' command ends up using port 25 and not port 587?  Seems less than optimal.  Also, removal of the MYUSERS policy bank makes it impossible (?) to disable spam/antivirus for local users sending email?

7

Re: External emails treated as internal?

*) Could you please show us full log (in /var/log/maillog) of these 3 emails?
*) Without MYUSERS, we can disable spam/virus scanning with policy bank: ORIGINATING and MYNETS.

8

Re: External emails treated as internal?

ZhangHuangbin wrote:

*) Could you please show us full log (in /var/log/maillog) of these 3 emails?
*) Without MYUSERS, we can disable spam/virus scanning with policy bank: ORIGINATING and MYNETS.

Mail from root on iredmail to a local address:

Nov 18 09:45:35 iredmail postfix/qmgr[4173]: 7B8572C06D6: from=<root@druber.com>, size=865, nrcpt=1 (queue active)
Nov 18 09:45:35 iredmail postfix/smtpd[12977]: disconnect from iredmail.druber.com[127.0.0.1]
Nov 18 09:45:35 iredmail amavis[11734]: (11734-04) Passed CLEAN {RelayedInbound}, <root@druber.com> -> <dswartz@druber.com>, Message-ID: <20151118144534.DEAB42C0791@iredmail.druber.com>, mail_id: zlq7cOS-ZyIs, Hits: -0.001, size: 438, queued_as: 7B8572C06D6, 601 ms
Nov 18 09:45:35 iredmail postfix/smtp[12972]: DEAB42C0791: to=<dswartz@druber.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=0.63, delays=0.02/0/0/0.61, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B8572C06D6)
Nov 18 09:45:35 iredmail postfix/qmgr[4173]: DEAB42C0791: removed
Nov 18 09:45:35 iredmail postfix/pipe[12980]: 7B8572C06D6: to=<dswartz@druber.com>, relay=dovecot, delay=0.05, delays=0/0/0/0.04, dsn=2.0.0, status=sent (delivered via dovecot service)
Nov 18 09:45:35 iredmail postfix/qmgr[4173]: 7B8572C06D6: removed

Mail from root on iredmail to outside address:

Nov 18 09:47:10 iredmail postfix/pickup[12775]: 90CA42C0791: uid=0 from=<root>
Nov 18 09:47:10 iredmail postfix/cleanup[12969]: 90CA42C0791: message-id=<20151118144710.90CA42C0791@iredmail.druber.com>
Nov 18 09:47:10 iredmail postfix/qmgr[4173]: 90CA42C0791: from=<root@druber.com>, size=450, nrcpt=1 (queue active)
Nov 18 09:47:15 iredmail postfix/smtpd[13068]: connect from iredmail.druber.com[127.0.0.1]
Nov 18 09:47:15 iredmail postfix/smtpd[13068]: CD3EB2C06D6: client=iredmail.druber.com[127.0.0.1]
Nov 18 09:47:15 iredmail postfix/cleanup[12969]: CD3EB2C06D6: message-id=<20151118144710.90CA42C0791@iredmail.druber.com>
Nov 18 09:47:15 iredmail postfix/qmgr[4173]: CD3EB2C06D6: from=<root@druber.com>, size=901, nrcpt=1 (queue active)
Nov 18 09:47:15 iredmail postfix/smtpd[13068]: disconnect from iredmail.druber.com[127.0.0.1]
Nov 18 09:47:15 iredmail amavis[11735]: (11735-04) Passed CLEAN {RelayedInbound}, <root@druber.com> -> <xxxxxxxxxxxx>, Message-ID: <20151118144710.90CA42C0791@iredmail.druber.com>, mail_id: F1jkPd9OzdpY, Hits: 0, size: 450, queued_as: CD3EB2C06D6, 5254 ms
Nov 18 09:47:15 iredmail postfix/smtp[12972]: 90CA42C0791: to=<xxxxxxxxxx>, relay=127.0.0.1[127.0.0.1]:10024, delay=5.3, delays=0.02/0/0/5.3, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as CD3EB2C06D6)
Nov 18 09:47:15 iredmail postfix/qmgr[4173]: 90CA42C0791: removed
Nov 18 09:47:16 iredmail postfix/smtp[13070]: Host offered STARTTLS: [smtpcorp.com]
Nov 18 09:47:17 iredmail postfix/smtp[13070]: CD3EB2C06D6: to=<xxxxxxxxxxxx>, relay=smtpcorp.com[216.22.15.245]:2525, delay=1.6, delays=0.01/0.04/0.6/1, dsn=2.0.0, status=sent (250 OK id=1Zz40z-NRKEsy-Dv)
Nov 18 09:47:17 iredmail postfix/qmgr[4173]: CD3EB2C06D6: removed

Mail from outside host to local address:

Nov 18 09:49:56 iredmail postfix/smtpd[13080]: connect from c7-b.mxthunder.net[208.53.48.218]
Nov 18 09:49:57 iredmail postfix/smtpd[13080]: 2D0CC2C06D6: client=c7-b.mxthunder.net[208.53.48.218]
Nov 18 09:49:57 iredmail postfix/cleanup[13089]: 2D0CC2C06D6: message-id=<BY1PR0801MB09036FA045ADA39D296C75D7921C0@BY1PR0801MB0903.namprd08.prod.outlook.com>
Nov 18 09:49:57 iredmail postfix/qmgr[4173]: 2D0CC2C06D6: from=<xxxxxx>, size=6399, nrcpt=1 (queue active)
Nov 18 09:49:57 iredmail postfix/smtpd[13080]: disconnect from c7-b.mxthunder.net[208.53.48.218]
Nov 18 09:49:58 iredmail postfix/smtpd[13094]: connect from iredmail.druber.com[127.0.0.1]
Nov 18 09:49:58 iredmail postfix/smtpd[13094]: 9F3BE2C0791: client=iredmail.druber.com[127.0.0.1]
Nov 18 09:49:58 iredmail postfix/cleanup[13089]: 9F3BE2C0791: message-id=<BY1PR0801MB09036FA045ADA39D296C75D7921C0@BY1PR0801MB0903.namprd08.prod.outlook.com>
Nov 18 09:49:58 iredmail postfix/qmgr[4173]: 9F3BE2C0791: from=<xxxxx>, size=6826, nrcpt=1 (queue active)
Nov 18 09:49:58 iredmail postfix/smtpd[13094]: disconnect from iredmail.druber.com[127.0.0.1]
Nov 18 09:49:58 iredmail amavis[11737]: (11737-04) Passed CLEAN {RelayedInbound}, [208.53.48.218]:61839 [157.56.111.108] <xxxxxx> -> <dswartz@druber.com>, Queue-ID: 2D0CC2C06D6, Message-ID: <BY1PR0801MB09036FA045ADA39D296C75D7921C0@BY1PR0801MB0903.namprd08.prod.outlook.com>, mail_id: fDbGgrt1Q0ou, Hits: 0.975, size: 6396, queued_as: 9F3BE2C0791, 1251 ms
Nov 18 09:49:58 iredmail postfix/smtp[13091]: 2D0CC2C06D6: to=<dswartz@druber.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.7, delays=0.37/0.03/0/1.3, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 9F3BE2C0791)
Nov 18 09:49:58 iredmail postfix/qmgr[4173]: 2D0CC2C06D6: removed
Nov 18 09:49:58 iredmail postfix/pipe[13096]: 9F3BE2C0791: to=<dswartz@druber.com>, relay=dovecot, delay=0.07, delays=0.01/0.04/0/0.03, dsn=2.0.0, status=sent (delivered via dovecot service)
Nov 18 09:49:58 iredmail postfix/qmgr[4173]: 9F3BE2C0791: removed

9

Re: External emails treated as internal?

Interestingly, when my daughter sent me an email (local to local) via webmail, it looked like this:

Nov 18 08:21:55 iredmail postfix/smtpd[12468]: connect from iredmail.druber.com[127.0.0.1]
Nov 18 08:21:55 iredmail postfix/smtpd[12468]: 8BBAB2C0791: client=iredmail.druber.com[127.0.0.1]
Nov 18 08:21:55 iredmail postfix/cleanup[12463]: 8BBAB2C0791: message-id=<7D8923DF-4F8B-425A-8F99-7BDEF9CE6529@druber.com>
Nov 18 08:21:55 iredmail postfix/qmgr[4173]: 8BBAB2C0791: from=<xxx@druber.com>, size=2308, nrcpt=1 (queue active)
Nov 18 08:21:55 iredmail postfix/smtpd[12468]: disconnect from iredmail.druber.com[127.0.0.1]
Nov 18 08:21:55 iredmail amavis[11736]: (11736-01) Passed CLEAN {RelayedInternal}, ORIGINATING LOCAL [108.26.149.237]:53892 [108.26.149.237] <xxx@druber.com> -> <dswartz@druber.com>, Queue-ID: DD6132C06CF, Message-ID: <7D8923DF-4F8B-425A-8F99-7BDEF9CE6529@druber.com>, mail_id: QpL2CG0CamfY, Hits: -0.999, size: 1064, queued_as: 8BBAB2C0791, dkim_new=dkim:druber.com, 629 ms
Nov 18 08:21:55 iredmail postfix/smtp[12465]: DD6132C06CF: to=<dswartz@druber.com>, relay=127.0.0.1[127.0.0.1]:10026, delay=0.74, delays=0.08/0.02/0.01/0.64, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 8BBAB2C0791)
Nov 18 08:21:55 iredmail postfix/qmgr[4173]: DD6132C06CF: removed
Nov 18 08:21:55 iredmail postfix/pipe[12470]: 8BBAB2C0791: to=<dswartz@druber.com>, relay=dovecot, delay=0.07, delays=0.01/0.04/0/0.02, dsn=2.0.0, status=sent (delivered via dovecot service)
Nov 18 08:21:55 iredmail postfix/qmgr[4173]: 8BBAB2C0791: removed

10

Re: External emails treated as internal?

So, what confuses you is the "Relayed{XXX}" tag? Quote from Amavisd-new release notes (http://www.ijs.si/software/amavisd/release-notes.txt )

- added a macro 'actions_performed', which expands into a comma-separated
  list of words: Accepted, Relayed(Untagged), RelayedTagged, Discarded,
  Rejected, Bounced, NoBounce or TempFailed, followed by a mail flow
  direction word: Inbound, Internal, Outbound or OpenRelay. For brevity
  the 'RelayedUntagged' status appears in this list as 'Relayed'.
  Additionally, the list may include words Quarantined and Archived.
  For multirecipient messages it is possible that the list includes
  more than one combination.

  The purpose of this macro is to augment the bare-bones 'Passed CLEAN'
  or 'Blocked SPAM' in the main log entry. For this purpose the default
  log template now includes this macro call. If the additional information
  is not desired in the log, please assign a customized template to the
  $log_templ configuration variable.

  Some examples of the new log entries:

    Passed CLEAN {RelayedOutbound}, ...
    Passed CLEAN {RelayedInbound}, ...
    Passed CLEAN {RelayedInternal,RelayedOutbound}, ...
    Passed SPAMMY {RelayedTaggedInbound}, ...
    Blocked SPAM {RejectedInbound,Quarantined}, ...
    Blocked INFECTED (Mal/BredoZp-B) {DiscardedInbound,Quarantined}, ...

  Semantics of entries in the 'actions_performed' list corresponds
  to the newly added SNMP variables 1.3.6.1.4.1.15312.2.1.1.19 - .26
  (with the exception that 'RelayedUntagged' counter is abbreviated
  in this macro as 'Relayed'). Please see their detailed description
  in a file AMAVIS-MIB.txt .

Excuse me, what's the real issue now?

11

Re: External emails treated as internal?

It seemed confusing to me that sending mail to an outside address shows as RelayedInbound.  Also wasn't sure why email from iredmail itself to a virtual domain address shows as RelayedInbound rather than RelayedInternet (which is what happens in the loopback/webmail case.)  Neither of these show as ORIGINATING either?

12

Re: External emails treated as internal?

I checked Amavisd source code (/usr/sbin/amavisd-new) to understand how it defines 'Relayed[XXX]':

      my $orig = $msginfo->originating;
      ...
      my $islocal = $r->recip_is_local;
        if ($orig) {
          if ($islocal) { $which_counts{$which.'Internal'}++ }
          else          { $which_counts{$which.'Outbound'}++ }
          $which_counts{$which.'Originating'}++;
        } else {
          if ($islocal) { $which_counts{$which.'Inbound'}++ }
          else          { $which_counts{$which.'OpenRelay'}++ }
        }
      ...

*) Mail was sent by an authenticated user, if recipient is a local user, it will be either RelayedInternal. otherwise RelayedOutbound.
*) Mail was sent by an external user (not authenticated), if recipient is a local user, it will be RelayedInbound. Otherwise RelayedOpenRelay.

Hope it helps.

13

Re: External emails treated as internal?

Great thanks.  Still one nit: why when root@druber.com => SOMEUSER@druber.com does it say 'RelayedInbound'?  'ORIGINATING LOCAL' is not set in this case.  Interestingly, if I telnet to port 25 on localhost and do an SMTP transaction manually, it shows up as '{RELAYED INTERNAL} MYNETS LOCAL', whereas 'mail dswartz@druber.com' is {RELAYED INBOUND} even though the connect is to localhost.  One difference is that in the mail client case, I am seeing:

Nov 19 09:39:56 iredmail postfix/pickup[24126]: 649B12C0791: uid=0 from=<root>
Nov 19 09:39:56 iredmail postfix/cleanup[24700]: 649B12C0791: message-id=<20151119143956.649B12C0791@iredmail.druber.com>

So it looks like the message is being injected differently, which seems to be fooling amavis?  Not sure this is a big deal - just want to make sure something isn't broken here...

14

Re: External emails treated as internal?

If you have smtp authentication, it will be applied with both 'ORIGINATING' and MYNETS policy banks. if no smtp authentication, sending from 127.0.0.1 is 'MYNETS'.

I'm afraid that i may be not able to deeply/clearly explain how Amavisd works and how it detects 'internal', 'inbound', 'outbound', i suggest you ask in Amavisd mailing list to make it clear if it still confused you. Sorry about this.

15

Re: External emails treated as internal?

Okay, I don't want to take up more of your time.  I think you misunderstood my motivation, though.  I don't want to become an amavis expert - I just wanted to know why this (fairly complex) system is apparently mis-categorizing some types of emails.  All I have done is install iredmail, and then apply a knowledge base workaround for a known bug, so I don't think it was unreasonable to wonder why this doesn't seem to be working 100%.  But whatever...

16

Re: External emails treated as internal?

I understand this is a reasonable question, i just mean i'm not good enough to answer this question, so i suggest you ask in Amavisd mailing list to get support from Amavisd developers.

Don't go mad.