1

Topic: greylisting by domain name

==== Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.5-1
- Linux/BSD distribution name and version: ubuntu 16.04 64
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): LDAP
- Web server (Apache or Nginx):nginx
- Manage mail accounts with iRedAdmin-Pro? yes
- Related log if you're reporting an issue:
====

Hallo, many email servers have more smtp server than one with different IP's , Your greylist script works only with IP's. Sometimes i wait for new mail more than 1-2 hours.

Example:

Nov 22 20:30:56 mail postfix/smtpd[42409]: NOQUEUE: reject: RCPT from 17.mo3.mail-out.ovh.net[87.98.178.58]:
Nov 22 20:37:31 mail postfix/smtpd[42409]: NOQUEUE: reject: RCPT from 15.mo3.mail-out.ovh.net[87.98.150.177]:
Nov 22 20:47:25 mail postfix/smtpd[42409]: NOQUEUE: reject: RCPT from 8.mo3.mail-out.ovh.net[87.98.172.249]:
Nov 22 21:07:24 mail postfix/smtpd[43538]: NOQUEUE: reject: RCPT from 11.mo3.mail-out.ovh.net[87.98.184.158]:
Nov 22 21:47:37 mail amavis[40663]: (40663-15) Passed CLEAN {RelayedInbound}, [87.98.184.158]-


I'd want to have ability to grelist by domain name mask , such as *.ovh.net in this example. I could use other greylisting scripts with this ability , but i will lose greylisting control via iredamin pro.

Could you make this in your scripts , pls?

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2

Re: greylisting by domain name

Solutions:

*) Whitelist this sender domain for greylisting service:

cd /opt/iredapd/tools/
python spf_to_greylist_whitelists.py --submit <domain1> <domain2> ...

Note: script spf_to_greylist_whitelists.py will store those mail domain names in sql table, and query its SPF, MX, A records, store all IP addresses or networks in those DNS records as whitelisted sender servers.

*) If sender domain doesn't have proper SPF/MX records, whitelist their mail domain name directly:

cd /opt/iredapd/tools/
python greylisting_admin.py --disable --from '@<domain>'

Note: '@' is required if you want to whitelist entire domain. And it just checks sender email address, not sender server IP addresses.

Check more details and samples in tutorial below:
http://www.iredmail.org/docs/manage.iredapd.html

BTW, of course you can use either one or both two, but first one is recommended - if sender domain has proper SPF DNS  record.

3

Re: greylisting by domain name

Actually whitelist is not a solution. In this case i need to look logs by myself every day and  add smtp servers with multiply ip's to whitelist. And this will not help with new emails from new servers.

I'd like to have possibility to choose way for grelylisting. I want to store and check domain names by mask *.domain.com
instead of IP's.

Before i did't use Pro version and i always delete default iredmail greylisting and used other scripts for this, but now i'd like to keep all Pro version possibilities in web interface.

ZhangHuangbin wrote:

Solutions:

*) Whitelist this sender domain for greylisting service:

cd /opt/iredapd/tools/
python spf_to_greylist_whitelists.py --submit <domain1> <domain2> ...

Note: script spf_to_greylist_whitelists.py will store those mail domain names in sql table, and query its SPF, MX, A records, store all IP addresses or networks in those DNS records as whitelisted sender servers.

*) If sender domain doesn't have proper SPF/MX records, whitelist their mail domain name directly:

cd /opt/iredapd/tools/
python greylisting_admin.py --disable --from '@<domain>'

Note: '@' is required if you want to whitelist entire domain. And it just checks sender email address, not sender server IP addresses.

Check more details and samples in tutorial below:
http://www.iredmail.org/docs/manage.iredapd.html

BTW, of course you can use either one or both two, but first one is recommended - if sender domain has proper SPF DNS  record.

4

Re: greylisting by domain name

Other way is to give ability to check IP's by /24 instead of individual IP's

5

Re: greylisting by domain name

Constin wrote:

Actually whitelist is not a solution. In this case i need to look logs by myself every day and  add smtp servers with multiply ip's to whitelist. And this will not help with new emails from new servers.
I'd like to have possibility to choose way for grelylisting. I want to store and check domain names by mask *.domain.com
instead of IP's.

I'm confused, iRedAPD tool "spf_to_greylist_whitelists.py" with option "--submit" does exactly what you want.

You submit a mail domain name, "spf_to_greylist_whitelists.py" stores this domain name in SQL table, and query its SPF/XM records to get the server IP addresses/networks and whitelist them. There's a cron job to query those stored mail domain names regularly, and update the whitelisted IP addresses/networks.

6

Re: greylisting by domain name

Hmm , maybe we don't understand each other. I don't want  to add domains to whitelist manually, because we have many new mails from new domains.  But i want greylist to do it automatically by domain name OR by IP/24 .

For my example:
Firs connection from 17.mo3.mail-out.ovh.net[87.98.178.58]: should be greylisted
Second connection from 15.mo3.mail-out.ovh.net[87.98.150.177]]: should be autowhitelisted by 87.98.150.* mask

like it made in postfix-gld


#
# Shall we use lightgrey option ? (0=No,1=Yes) (default is 0)
#
# The lightgrey option, mask the last octet of IP addresses
# and thus we greylist only C classes (/24) instead of individual IPs.
#
LIGHTGREY=0




#
# Shall we use the mxgrey algorithm ? (0=No,>0=Yes) (default is 0)
#
# the mxgrey algorithm is a variation of the greylist algorithm.
# When this is enabled, we allow all incoming mails from an IP address
# whatever source/destination email as long as this IP has been greylisted
# at least X time and succeded the mail resend .
#
# Example:
# The IP 1.2.3.4 sends an email from src@domain.com to user@yourdomain.com
# We greylist this mail as this IP is not yet in database and send a 450 SMTP code
# After some time, the IP re-send the mail from src@domain.com to user@yourdomain.com
# We update the db.
# Some time after the ip 1.2.3.4 sends an email from john@domain.com to fred@yourdomain.com
# We will accept this mail without any greylisting, as this ip already succeded a greylist test
# and thus seems to be a valid smtp server and not a spammer .
#
# The advantage of this method, is that it reduce the re-send time due to greylisting to
# x mail per server instead of one mail per destination .
#
# The value you provide in MXGREY is the minimum number of succesful greylists
# before accepting all mails from this MX. higher the number is, harder is to get in.
#
# This algortihm replace the old LIGHTGREYDOMAIN which was available prior version 1.6
#
MXGREY=0


ZhangHuangbin wrote:
Constin wrote:

Actually whitelist is not a solution. In this case i need to look logs by myself every day and  add smtp servers with multiply ip's to whitelist. And this will not help with new emails from new servers.
I'd like to have possibility to choose way for grelylisting. I want to store and check domain names by mask *.domain.com
instead of IP's.

I'm confused, iRedAPD tool "spf_to_greylist_whitelists.py" with option "--submit" does exactly what you want.

You submit a mail domain name, "spf_to_greylist_whitelists.py" stores this domain name in SQL table, and query its SPF/XM records to get the server IP addresses/networks and whitelist them. There's a cron job to query those stored mail domain names regularly, and update the whitelisted IP addresses/networks.

7

Re: greylisting by domain name

So if IP(v4) address '1.1.1.1' passed the greylisting service, we simply whitelist '1.1.1.0/24' (or '1.1.1.*'), and expire in X days?

8

Re: greylisting by domain name

Not certainly in that way. We don't wait 1.1.1.1 to connect twice.

First connection from 1.1.1.1 - greylisted, start waiting new connection from 1.1.1.0/24 to pass.
Second connection from 1.1.1.2  - PASSED . Add 1.1.1.0/24 to whitelist with expire in X days.

9

Re: greylisting by domain name

I'm afraid that this is not a good idea, and not how greylisting works.

*) We track connections from 1.1.1.1 and 1.1.1.2 before any of them passed greylisting.
*) We use 'sender + recipient + sender IP' as a triplet to track sender.
*) When one sender/triplet passed greylisting, you can decide whether whitelist a single IP address, or a C class network.

In old design, when one triplet passed, we simply whitelist this triplet (that means only SAME sender + SAME recipient + SAME sender IP). In current iRedAPD design, we whitelist this sender IP directly, so all emails sent from this IP will pass greylisting service. And your request is whitelisting its C class network.

10

Re: greylisting by domain name

In this way first delay will be more then 1 hour always( if sender have many IP's). Look my log with mail-out.ovh.net , it used 87.98.184.158 again only after 1 hour.
It would be perfect to give users(admins) flexible  greylisting tool with many options. You can use your strong rules by default, but  i'd like to have ability to configure it as i want.
For me "sender + recipient + sender IP/24" is the best option. But with /24 at start, i don't want to wait triplet with single IP.

Now business tell me "ok, we don't want to wait 1-2 hours for first email from new client after call".  And i have only 2 ways
1) switch your greylist scripts to other (postfix-gld has this ability) and lost it in iredadmin.
2) disable greylist and get spam.

11

Re: greylisting by domain name

" *) We track connections from 1.1.1.1 and 1.1.1.2 before any of them passed greylisting.
*) We use 'sender + recipient + sender IP' as a triplet to track sender.
*) When one sender/triplet passed greylisting, you can decide whether whitelist a single IP address, or a C class network."


This is not bad too. for me better than now )

12

Re: greylisting by domain name

I checked iRedAPD source code, it's very easy to whitelist "1.1.1.*" while sender passed greylisting, from your experience, do you think it's enough?

13

Re: greylisting by domain name

Hallo ZhangHuangbin!
Thank you.


I think this is good solution, but here is not possible to make one configuration for all)
It’s depends  of many factors, uses admin fail2ban or not, uses admin   reject_unknown_helo_hostname  or reject_non_fqdn_hostname in smtpd_helo_restrictions or not etc. That’s why i ask about it as option. Or i could hardcode it only for my iRedAPD.

Am I correct in assuming that :

First connection: a@domain1.com  IP 1.1.1.1 --> x@domain2.com - greylisted
Second connection a@domain1.com  -->IP 1.1.1.1 x@domain2.com  -PASSED  and  1.1.1.0/24 temporarily (X days) whitelisted.
Third connection b@domain1.com  IP 1.1.1.2 -->  z@domain2.com  - PASSED because from 1.1.1.0/24 and  we don't check sender and receiver for this range more.

Or you check sender and receiver in third connection? What would be great to give an option to choose.

another option:

First connection: a@domain1.com  IP 1.1.1.1 --> x@domain2.com - greylisted
Second connection a@domain1.com  -->IP 1.1.1.1 x@domain2.com  -PASSED  and  all senders with all IP’s from domain1.com
temporarily  (X days) whitelisted =  domain1.com  whitelisted. I think it’s not complicated because you already have whitelistcheck for  domain names.

14

Re: greylisting by domain name

i thought again  and think now , that 1.1.1.0/24 is enough)

15

Re: greylisting by domain name

Is it possible to do it this way:
- email from 1.1.1.1 comes and gets greylisted
- another try to deliver same email comes from 1.1.1.2 and it passes as it's 1.1.1.*

16

Re: greylisting by domain name

any news?

17

Re: greylisting by domain name

ZhangHuangbin wrote:

I checked iRedAPD source code, it's very easy to whitelist "1.1.1.*" while sender passed greylisting, from your experience, do you think it's enough?

Could you please print diff? i'l make it only for me.

18

Re: greylisting by domain name

You guys are digging a half-year old thread... sad

Currently, if one sender passed greylisting, it's IP will be whitelisted for few days (depends on the setting in iRedAPD config file, default is 30 days). If another sender sends email from same server (same IP), it's considered as it already passed greylisting, so no more greylisted.

Greylisting depends on whether sender server retries mail delivery, if it passed, it sure will pass for all emails sent by hosted users, so no reason to apply greylisting to this sender server anymore. But if it passed, we still don't pass '1.1.1.*' automatically.

19

Re: greylisting by domain name

ZhangHuangbin wrote:

You guys are digging a half-year old thread... sad

Currently, if one sender passed greylisting, it's IP will be whitelisted for few days (depends on the setting in iRedAPD config file, default is 30 days). If another sender sends email from same server (same IP), it's considered as it already passed greylisting, so no more greylisted.

Greylisting depends on whether sender server retries mail delivery, if it passed, it sure will pass for all emails sent by hosted users, so no reason to apply greylisting to this sender server anymore. But if it passed, we still don't pass '1.1.1.*' automatically.

i don't digging) i just paused this question while waiting next releases)   This is big problem, i couldn't use default greylisting without 1.1.1.*

20

Re: greylisting by domain name

How about use CIDR network instead?

21

Re: greylisting by domain name

ZhangHuangbin wrote:

How about use CIDR network instead?

no problem , 1.1.1.0/24 ))

1/2 year ago you wrote , that its very easy to make changes for this ability, could you tell me code diff, please? i don't know python..

22

Re: greylisting by domain name

Constin wrote:

1/2 year ago you wrote , that its very easy to make changes for this ability, could you tell me code diff, please? i don't know python..

It's just a quick check without actually writing some code to achieve it, so i don't have any diff. sorry.