1

Topic: Not all junk being filtered

==== Provide basic information to help troubleshoot and get quick answer ====
- iRedMail version: 0.7.3
- Linux/BSD distribution name and version: Red Hat Enterprise Linux 5 - 64 bit
- Any related log? Log is helpful for troubleshooting.
====

Hi

I've enabled the junk mail filtering into the junk folder by renaming the dovecot.sieve file. It seems to be working but not all of the junk is being filtered. All junk is being flagged with ***SPAM*** in the subject, but only some of the junk goes into the junk folder.

Here's the bit I've edited in my amavisd.conf file:

$sa_tag_level_deflt  = -100;  # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 5;  # add 'spam detected' headers at that level
$sa_kill_level_deflt = 5;  # triggers spam evasive actions (e.g. blocks mail)
$sa_dsn_cutoff_level = 8;   # spam level beyond which a DSN is not sent
$sa_crediblefrom_dsn_cutoff_level = 18; # likewise, but for a likely valid From
# $sa_quarantine_cutoff_level = 25; # spam level beyond which quarantine is off
$penpals_bonus_score = 8;    # (no effect without a @storage_sql_dsn database)
$penpals_threshold_high = $sa_kill_level_deflt;  # don't waste time on hi spam
$bounce_killer_score = 100;  # spam score points to add for joe-jobbed bounces


This is the header from an email that *did* get moved to the junk folder:

Return-Path: <xcmkyurzvsyai@faqserv.com>
Delivered-To: matt.xxx@xxx.net
Received: from localhost (localhost [127.0.0.1])
    by xxx.xxx.net (Postfix) with ESMTP id DE4309582DD
    for <matt.xxx@xxx.net>; Tue,  6 Dec 2011 14:02:09 +0000 (GMT)
X-Quarantine-ID: <pa0WVw6VioML>
X-Virus-Scanned: amavisd-new at xxx.xxx.net
X-Amavis-Alert: BAD HEADER SECTION, MIME error: error: unexpected end of
    header
X-Spam-Flag: YES
X-Spam-Score: 21.783
X-Spam-Level: *********************
X-Spam-Status: Yes, score=21.783 tagged_above=-100 required=5
    tests=[EMPTY_MESSAGE=0.607, INVALID_MSGID=2.603, MISSING_HB_SEP=2.5,
    MISSING_SUBJECT=1.285, MSGID_SHORT=0.232, RCVD_NUMERIC_HELO=2.599,
    RDNS_NONE=0.1, SPF_FAIL=5, TT_MSGID_TRUNC=1.874,
    TVD_SPACE_RATIO=2.899, UNCLOSED_BRACKET=2.083,
    UNPARSEABLE_RELAY=0.001] autolearn=spam
Subject: ***SPAM***
Received: from xxx.xxx.net ([127.0.0.1])
    by localhost (xxx.xxx.net [127.0.0.1]) (amavisd-new, port 10024)
    with ESMTP id pa0WVw6VioML for <matt.xxx@xxx.net>;
    Tue,  6 Dec 2011 14:02:09 +0000 (GMT)



This is the header from an email that *did not* get moved to the junk folder but did have the subject marked as ***spam***:

Return-Path: <c48scshito@ezweb.ne.jp>
Delivered-To: matt.xxx@xxx.net
Received: from localhost (localhost [127.0.0.1])
    by xxx.xxx.net (Postfix) with ESMTP id CC2CD9582DD
    for <matt.xxx@xxx.net>; Tue,  6 Dec 2011 16:33:02 +0000 (GMT)
X-Quarantine-ID: <XEaWjhS0AlxI>
X-Virus-Scanned: amavisd-new at xxx.xxx.net
X-Amavis-Alert: BAD HEADER SECTION, Non-encoded 8-bit data (char 90 hex):
    From: "\220\274\223\207 \216u\225\344\227\242" <hy[...]
X-Spam-Flag: YES
X-Spam-Score: 9.993
X-Spam-Level: *********
X-Spam-Status: Yes, score=9.993 tagged_above=-100 required=5
    tests=[FROM_ILLEGAL_CHARS=3.999, RCVD_IN_BL_SPAMCOP_NET=2.188,
    RDNS_NONE=0.1, SPF_SOFTFAIL=0.654, TVD_SPACE_RATIO=2.899,
    URIBL_WS_SURBL=0.1533] autolearn=no
Received: from xxx.xxx.net ([127.0.0.1])
    by localhost (xxx.xxx.net [127.0.0.1]) (amavisd-new, port 10024)
    with ESMTP id XEaWjhS0AlxI for <matt.xxx@xxx.net>;
    Tue,  6 Dec 2011 16:32:58 +0000 (GMT)


Any ideas?

Thanks

Matt

2

Re: Not all junk being filtered

No idea yet, sorry.
Could you please show us the sieve rules used to move SPAM to Junk folder?

3

Re: Not all junk being filtered

This is my /var/vmail/sieve/dovecot.sieve file:

require "fileinto";

# -------------------------------------------------
# --------------- Global sieve rules --------------
# -------------------------------------------------

# rule:[Move Spam to Junk Folder]
if header :matches ["X-Spam-Flag"] ["YES"] {

    # Keep this mail in INBOX.
    #keep;

    # If you ensure it is really a spam, drop it to 'Junk', and stop
    # here so that we do not reply to spammers.
    fileinto "Junk";

    # Do not waste resource on spam mail.
    stop;

    # If you ensure they are spam, you can discard it.
    #discard;
}


Thanks

Matt

4

Re: Not all junk being filtered

In fact after leaving the account over night, I can see that nothing is being filtered at all. Spam is being marked up as ***SPAM*** and the X-Spam-Flag is being set to yes in the header, but it is not filing into the Junk folder.

Matt

5

Re: Not all junk being filtered

I might have figured out what's causing this, but I'm not sure what the solution should be.

When I create a filter rule using RoundCube to filter certain emails into a folder, it stops the global junk mail filter from working.

When a personal sieve file is created, does that prevent the global sieve file from being used?

Obviously I could create my own junk mail rule in RoundCube when I add other rules, but this isn't ideal when dealing with a large user base.

Matt

6

Re: Not all junk being filtered

Oops, Dovecot-1.x works as you described, if there's a personal sieve rule file, global sieve rules will be ignored.
In Dovecot-2.x, it can force to execute a global sieve rule file before or after personal sieve rules with settings:

plugin {
    ...
    sieve_before = /path/to/a/sieve/file
    sieve_after = /path/to/another/sieve/file
}

Reference: http://wiki2.dovecot.org/Pigeonhole/Sie … quentially

On RHEL/CentOS/Scientific Linux 5.x, iRedMail will install Dovecot-1.2 by default. But installs Dovecot-2 on RHEL/CentOS/Scientific Linux 6.x.

7

Re: Not all junk being filtered

Is there a way to update to Dovecot-2 on RHEL5 without breaking iRedMail?

8

Re: Not all junk being filtered

We don't have tutorial for upgrading Dovecot-1.x to 2.x now. But here's our sample config file for Dovecot-2.x:
https://bitbucket.org/zhb/iredmail/src/ … ecot2.conf

As you can see, there're many words start with 'PH_', they're placeholders, and will be configured during iRedMail installation, you can use it as a sample config file, then sync it with Dovecot-1.x config file.

Also, Dovecot provides tutorial for upgrading 1.x to 2.x: http://wiki2.dovecot.org/Upgrading/2.0

Post any issues here, we will try our best to help upgrade. smile