1 Topic by chc-pr (edited by chc-pr 2017-06-06 03:48:47)

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Topic: fail2ban is failing to ban

==== Required information ====
- iRedMail version (check /etc/iredmail-release): v0.9.6
- Linux/BSD distribution name and version: Ubuntu 16.04.02 ?xenial
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Web server (Apache or Nginx): Apache
- Manage mail accounts with iRedAdmin-Pro? No
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

As per the title really, fail2ban is failing to ban.
This is a fresh install of iredmail with no other significant installations. I used a minimal Ubuntu 16.04.02 server as the base and installed a copy of iredmail onto that. Letsencrypt was then added to gain a public certified certificate.

The only other additions (other than apt-get upgrade and updates) have been an alternative webmail service and a calendar service for backward compatability for some users. SOGo and Roundcube have been kept untouched. I have not as yet added any fail2ban entries for these two services.

Everything other than fail2ban is working perfectly.

As can be seen from the iptables -L -n outputs, the various iptables chains specified in the fail2ban.local config file do not seem to be being activated other than [sshd]

I modified the ignoreip parameter so that only one local net ip address is ignored so I could both test fail2ban and ensure I did not lock myself out (as I noted in the comment ahead of the modified line)

I have included below, copies of the following

iptables -L -n outputs, both
- prior to deliberate mal logons
- post deliberate mal logons
fail2ban.local config file
fail2ban.conf config file
fail2ban/filter.d/sogo-auth.conf config file (and full path just in case that matters)
sogo log

Note that Roundcube and every other service I tested is also NOT banning. ONLY sshd seems to be working.

This is the only thing stopping me putting this into service now, so any help in getting to the bottom of it would be very welcome. I have searched the net high and low, and while I have found a great deal of information on fail2ban and similar issues, I cannot seem to locate anything quite close enough to help me get to the bottom of the issues.

Thank you.

================================
prior to deliberate mal logons
(looks identical after 7+ failed logons for any service other than sshd)
================================
root@mail2:~# iptables -L -n
Chain INPUT (policy DROP)
target     prot opt source               destination
f2b-sshd   tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0            icmptype 8
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:22
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:80
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:443
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:25
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:587
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:110
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:995
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:143
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:993

Chain FORWARD (policy DROP)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain f2b-sshd (1 references)
target     prot opt source               destination
RETURN     all  --  0.0.0.0/0            0.0.0.0/0

================================
post deliberate mal logons on sshd
================================
root@mail2:~# iptables -L -n
Chain INPUT (policy DROP)
target     prot opt source               destination
f2b-sshd   tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0            icmptype 8
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:22
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:80
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:443
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:25
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:587
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:110
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:995
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:143
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:993

Chain FORWARD (policy DROP)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain f2b-sshd (1 references)
target     prot opt source               destination
REJECT     all  --  172.16.1.127         0.0.0.0/0            reject-with icmp-port-unreachable
RETURN     all  --  0.0.0.0/0            0.0.0.0/0

==============
fail2ban.local
==============
# Refer to /etc/fail2ban/jail.conf for more examples.
[DEFAULT]
# time is in seconds. 3600 = 1 hour, 86400 = 24 hours (1 day)
findtime    = 3600
bantime     = 3600
maxretry    = 5

# This is the default inoreip line
#ignoreip    = 127.0.0.1 127.0.0.0/8 172.16.0.0/12

# The following line is a modification of the above default line
# I am using it to test blocking by fail2ban. It only ignores one
# desktop test machine and loopback block.
# Every other IP should be blocked
# if the failure parameters are exceeded
ignoreip    = 127.0.0.1 127.0.0.0/8 172.16.1.124

[sshd]
enabled     = true
filter      = sshd
action      = iptables-multiport[name=sshd, port="22", protocol=tcp]
logpath     = /var/log/auth.log

[sshd-ddos]
enabled     = true
filter      = sshd-ddos
action      = iptables-multiport[name=sshd-ddos, port="22", protocol=tcp]
logpath     = /var/log/auth.log

[roundcube-iredmail]
enabled     = true
filter      = roundcube.iredmail
action      = iptables-multiport[name=roundcube, port="http,https,smtp,submission,pop3,pop3s,imap,imaps,sieve", prot$
logpath     = /var/log/mail.log
findtime    = 3600

[dovecot-iredmail]
enabled     = true
filter      = dovecot.iredmail
action      = iptables-multiport[name=dovecot, port="http,https,smtp,submission,pop3,pop3s,imap,imaps,sieve", protoc$
logpath     = /var/log/dovecot.log
                /var/log/dovecot-imap.log
                /var/log/dovecot-pop3.log
                /var/log/dovecot-sieve.log

[postfix-iredmail]
enabled     = true
filter      = postfix.iredmail
action      = iptables-multiport[name=postfix, port="http,https,smtp,submission,pop3,pop3s,imap,imaps,sieve", protoc$
logpath     = /var/log/mail.log

[postfix-sasl]
enabled     = true
filter      = postfix-sasl


action      = iptables-multiport[name=postfix, port="http,https,smtp,submission,pop3,pop3s,imap,imaps,sieve", protoc$
logpath     = /var/log/mail.log

[sogo-iredmail]
enabled     = true
filter      = sogo-auth
action      = iptables-multiport[name=sogo, port="http,https,smtp,submission,pop3,pop3s,imap,imaps,sieve", protocol=$
logpath     = /var/log/sogo/sogo.log

==============
fail2ban.conf
==============
# Fail2Ban main configuration file
#
# Comments: use '#' for comment lines and ';' (following a space) for inline comments
#
# Changes:  in most of the cases you should not modify this
#           file, but provide customizations in fail2ban.local file, e.g.:
#
# [Definition]
# loglevel = DEBUG
#

[Definition]

# Option: loglevel
# Notes.: Set the log level output.
#         CRITICAL
#         ERROR
#         WARNING
#         NOTICE
#         INFO
#         DEBUG
# Values: [ LEVEL ]  Default: ERROR
#
loglevel = INFO

# Option: logtarget
# Notes.: Set the log target. This could be a file, SYSLOG, STDERR or STDOUT.
#         Only one log target can be specified.
#         If you change logtarget from the default value and you are
#         using logrotate -- also adjust or disable rotation in the
#         corresponding configuration file
#         (e.g. /etc/logrotate.d/fail2ban on Debian systems)
# Values: [ STDOUT | STDERR | SYSLOG | FILE ]  Default: STDERR
#
logtarget = SYSLOG

# Option: syslogsocket
# Notes: Set the syslog socket file. Only used when logtarget is SYSLOG
#        auto uses platform.system() to determine predefined paths
# Values: [ auto | FILE ]  Default: auto
syslogsocket = auto

# Option: socket
# Notes.: Set the socket file. This is used to communicate with the daemon. Do
#         not remove this file when Fail2ban runs. It will not be possible to
#         communicate with the server afterwards.
# Values: [ FILE ]  Default: /var/run/fail2ban/fail2ban.sock
#
socket = /var/run/fail2ban/fail2ban.sock

# Option: pidfile
# Notes.: Set the PID file. This is used to store the process ID of the
#         fail2ban server.
# Values: [ FILE ]  Default: /var/run/fail2ban/fail2ban.pid
#
pidfile = /var/run/fail2ban/fail2ban.pid

# Options: dbfile
# Notes.: Set the file for the fail2ban persistent data to be stored.
#         A value of ":memory:" means database is only stored in memory
#         and data is lost when fail2ban is stopped.
#         A value of "None" disables the database.
# Values: [ None :memory: FILE ] Default: /var/lib/fail2ban/fail2ban.sqlite3
dbfile = /var/lib/fail2ban/fail2ban.sqlite3

# Options: dbpurgeage
# Notes.: Sets age at which bans should be purged from the database
# Values: [ SECONDS ] Default: 86400 (24hours)
dbpurgeage = 86400

=================================
fail2ban/filter.d/sogo-auth.conf
=================================
# Fail2ban filter for SOGo authentcation
#
# Log file usually in /var/log/sogo/sogo.log

[Definition]

failregex = ^ sogod \[\d+\]: SOGoRootPage Login from '<HOST>' for user '.*' might not have worked( - password policy: \d*  grace: -?\d*  expire: -?\d*  bound: -?\d*)?\s*$

ignoreregex =

#
# DEV Notes:
#
# The error log may contain multiple hosts, whereas the first one
# is the client and all others are poxys. We match the first one, only
#
# Author: Arnd Brandes

=========
sogo log
=========
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be11a1d60[WOWatchDogChild]> sending terminate signal to pid 1818
Jun 05 10:08:02 sogod [1620]: <0x0x555be11a1540[WOWatchDogChild]> sending terminate signal to pid 1817
Jun 05 10:08:02 sogod [1620]: <0x0x555be11a8c90[WOWatchDogChild]> sending terminate signal to pid 1816
Jun 05 10:08:02 sogod [1620]: <0x0x555be1198360[WOWatchDogChild]> sending terminate signal to pid 1815
Jun 05 10:08:02 sogod [1620]: <0x0x555be11972d0[WOWatchDogChild]> sending terminate signal to pid 1814
Jun 05 10:08:02 sogod [1620]: <0x0x555be1197090[WOWatchDogChild]> sending terminate signal to pid 1813
Jun 05 10:08:02 sogod [1620]: <0x0x555be11969a0[WOWatchDogChild]> sending terminate signal to pid 1812
Jun 05 10:08:02 sogod [1620]: <0x0x555be1196500[WOWatchDogChild]> sending terminate signal to pid 1811
Jun 05 10:08:02 sogod [1620]: <0x0x555be1196170[WOWatchDogChild]> sending terminate signal to pid 1810
Jun 05 10:08:02 sogod [1620]: <0x0x555be119d860[WOWatchDogChild]> sending terminate signal to pid 1809
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be11969a0[WOWatchDogChild]> child 1812 exited
Jun 05 10:08:02 sogod [1620]: <0x0x555be11a1d60[WOWatchDogChild]> child 1818 exited
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be1196170[WOWatchDogChild]> child 1810 exited
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be119d860[WOWatchDogChild]> child 1809 exited
Jun 05 10:08:02 sogod [1620]: <0x0x555be1196500[WOWatchDogChild]> child 1811 exited
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be11a8c90[WOWatchDogChild]> child 1816 exited
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be11972d0[WOWatchDogChild]> child 1814 exited
Jun 05 10:08:02 sogod [1620]: <0x0x555be11a1540[WOWatchDogChild]> child 1817 exited
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be1198360[WOWatchDogChild]> child 1815 exited
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> Terminating with SIGINT or SIGTERM
Jun 05 10:08:02 sogod [1620]: <0x0x555be1197090[WOWatchDogChild]> child 1813 exited
Jun 05 10:08:02 sogod [1620]: <0x0x555be0e92920[WOWatchDog]> all children exited. We now terminate.
Jun 05 10:08:38 sogod [1633]: version 3.2.9 (build @shiva.inverse 201706040704) -- starting
Jun 05 10:08:38 sogod [1633]: vmem size check enabled: shutting down app when vmem > 500 MB. Currently at 224 MB
Jun 05 10:08:43 sogod [1633]: <0x0x559a51e0e060[SOGoProductLoader]> SOGo products loaded from '/usr/lib/GNUstep/SOGo':
Jun 05 10:08:43 sogod [1633]: <0x0x559a51e0e060[SOGoProductLoader]>   Mailer.SOGo, ActiveSync.SOGo, MainUI.SOGo, CommonUI.SOGo, SchedulerUI.SOGo, Contacts.SOGo, MailerUI.SOGo, ContactsUI.SOGo, Administratio$
Jun 05 10:08:44 sogod [1633]: All products loaded - current memory usage at 278 MB
Jun 05 10:08:44 sogod [1633]: <0x0x559a51bfb920[WOWatchDog]> listening on 127.0.0.1:20000
Jun 05 10:08:44 sogod [1633]: <0x0x559a51bfb920[WOWatchDog]> watchdog process pid: 1633
Jun 05 10:08:44 sogod [1633]: <0x0x7feae7871100[WOWatchDogChild]> watchdog request timeout set to 60 minutes
Jun 05 10:08:44 sogod [1633]: <0x0x559a51bfb920[WOWatchDog]> preparing 10 children
Jun 05 10:08:44 sogod [1633]: <0x0x559a51bfb920[WOWatchDog]> child spawned with pid 1805
Jun 05 10:08:44 sogod [1633]: <0x0x559a51bfb920[WOWatchDog]> child spawned with pid 1806
Jun 05 10:08:44 sogod [1633]: <0x0x559a51bfb920[WOWatchDog]> child spawned with pid 1807
Jun 05 10:08:44 sogod [1633]: <0x0x559a51bfb920[WOWatchDog]> child spawned with pid 1808
Jun 05 10:08:44 sogod [1633]: <0x0x559a51bfb920[WOWatchDog]> child spawned with pid 1809
Jun 05 10:08:44 sogod [1633]: <0x0x559a51bfb920[WOWatchDog]> child spawned with pid 1810
Jun 05 10:08:44 sogod [1633]: <0x0x559a51bfb920[WOWatchDog]> child spawned with pid 1811
Jun 05 10:08:44 sogod [1633]: <0x0x559a51bfb920[WOWatchDog]> child spawned with pid 1812
Jun 05 10:08:44 sogod [1633]: <0x0x559a51bfb920[WOWatchDog]> child spawned with pid 1813
Jun 05 10:08:44 sogod [1633]: <0x0x559a51bfb920[WOWatchDog]> child spawned with pid 1814
Jun 05 10:08:45 sogod [1807]: <0x0x559a52133000[WOHttpAdaptor]> notified the watchdog that we are ready
Jun 05 10:08:45 sogod [1811]: <0x0x559a52133000[WOHttpAdaptor]> notified the watchdog that we are ready
Jun 05 10:08:45 sogod [1812]: <0x0x559a52133000[WOHttpAdaptor]> notified the watchdog that we are ready
Jun 05 10:08:45 sogod [1805]: <0x0x559a52133000[WOHttpAdaptor]> notified the watchdog that we are ready
Jun 05 10:08:45 sogod [1813]: <0x0x559a52133000[WOHttpAdaptor]> notified the watchdog that we are ready
Jun 05 10:08:45 sogod [1810]: <0x0x559a52133000[WOHttpAdaptor]> notified the watchdog that we are ready
Jun 05 10:08:45 sogod [1814]: <0x0x559a52133000[WOHttpAdaptor]> notified the watchdog that we are ready
Jun 05 10:08:45 sogod [1806]: <0x0x559a52133000[WOHttpAdaptor]> notified the watchdog that we are ready
Jun 05 10:08:45 sogod [1809]: <0x0x559a52133000[WOHttpAdaptor]> notified the watchdog that we are ready
Jun 05 10:08:45 sogod [1808]: <0x0x559a52133000[WOHttpAdaptor]> notified the watchdog that we are ready
Jun 05 10:18:43 sogod [1808]: <0x0x559a51dd8ea0[SOGoCache]> Cache cleanup interval set every 300.000000 seconds
Jun 05 10:18:43 sogod [1808]: <0x0x559a51dd8ea0[SOGoCache]> Using host(s) '127.0.0.1' as server(s)
Jun 05 10:18:44 sogod [1808]: [WARN] <0x0x7feae7892cc0[WOxElemBuilder]> could not locate builders: WOxExtElemBuilder,WOxExtElemBuilder
Jun 05 10:18:44 sogod [1808]: 172.16.1.127 "GET /SOGo/ HTTP/1.1" 200 6984/0 0.401 26015 73% 3M
Jun 05 10:18:59 sogod [1808]: SOGoRootPage Login from '172.16.1.127' for user 'chris' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0
Jun 05 10:18:59 sogod [1808]: 172.16.1.127 "POST /SOGo/connect HTTP/1.1" 403 34/74 0.041 - - 0
Jun 05 10:19:07 sogod [1808]: SOGoRootPage Login from '172.16.1.127' for user 'chris' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0
Jun 05 10:19:07 sogod [1808]: 172.16.1.127 "POST /SOGo/connect HTTP/1.1" 403 34/75 0.002 - - 0
Jun 05 10:19:17 sogod [1808]: SOGoRootPage Login from '172.16.1.127' for user 'chris' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0
Jun 05 10:19:17 sogod [1808]: 172.16.1.127 "POST /SOGo/connect HTTP/1.1" 403 34/76 0.002 - - 0
Jun 05 10:19:23 sogod [1808]: SOGoRootPage Login from '172.16.1.127' for user 'chris' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0
Jun 05 10:19:23 sogod [1808]: 172.16.1.127 "POST /SOGo/connect HTTP/1.1" 403 34/77 0.002 - - 0
Jun 05 10:19:32 sogod [1808]: SOGoRootPage Login from '172.16.1.127' for user 'chris' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0
Jun 05 10:19:32 sogod [1808]: 172.16.1.127 "POST /SOGo/connect HTTP/1.1" 403 34/77 0.002 - - 0
Jun 05 10:19:39 sogod [1808]: SOGoRootPage Login from '172.16.1.127' for user 'chris' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0
Jun 05 10:19:39 sogod [1808]: 172.16.1.127 "POST /SOGo/connect HTTP/1.1" 403 34/78 0.002 - - 0
Jun 05 10:19:46 sogod [1808]: SOGoRootPage Login from '172.16.1.127' for user 'chris' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0
Jun 05 10:19:46 sogod [1808]: 172.16.1.127 "POST /SOGo/connect HTTP/1.1" 403 34/79 0.002 - - 0
Jun 05 10:19:52 sogod [1808]: SOGoRootPage Login from '172.16.1.127' for user 'chris' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0
Jun 05 10:19:52 sogod [1808]: 172.16.1.127 "POST /SOGo/connect HTTP/1.1" 403 34/80 0.002 - - 0
Jun 05 10:19:58 sogod [1808]: SOGoRootPage Login from '172.16.1.127' for user 'chris' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0
Jun 05 10:19:58 sogod [1808]: 172.16.1.127 "POST /SOGo/connect HTTP/1.1" 403 34/81 0.003 - - 0

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: fail2ban is failing to ban

When you restart Fail2ban service, any error/warning message in /var/log/syslog?

3 Reply by chc-pr

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

ZhangHuangbin wrote:

When you restart Fail2ban service, any error/warning message in /var/log/syslog?

I stopped, started and then reloaded fail2ban.  Here is the log of that entire sequence.  There seems is some notice about sshd not being a filter, but that is all I can see ...


Jun  7 15:02:46 mail2 fail2ban.server[1569]: INFO Stopping all jails
Jun  7 15:02:47 mail2 fail2ban.jail[1569]: INFO Jail 'sshd' stopped
Jun  7 15:02:47 mail2 fail2ban.server[1569]: INFO Exiting Fail2ban
Jun  7 15:02:47 mail2 fail2ban-client[5802]: ERROR  Failed to access socket path: /var/run/fail2ban/fail2ban.sock. I$
Jun  7 15:02:47 mail2 systemd[1]: fail2ban.service: Control process exited, code=exited status=255
Jun  7 15:02:47 mail2 systemd[1]: fail2ban.service: Unit entered failed state.
Jun  7 15:02:47 mail2 systemd[1]: fail2ban.service: Failed with result 'exit-code'.
Jun  7 15:02:47 mail2 systemd[1]: fail2ban.service: Service hold-off time over, scheduling restart.
Jun  7 15:02:47 mail2 systemd[1]: Stopped Fail2Ban Service.
Jun  7 15:02:47 mail2 systemd[1]: Starting Fail2Ban Service...
Jun  7 15:02:48 mail2 fail2ban-client[5807]: 2017-06-07 15:02:48,072 fail2ban.server         [5809]: INFO    Startin$
Jun  7 15:02:48 mail2 fail2ban-client[5807]: 2017-06-07 15:02:48,072 fail2ban.server         [5809]: INFO    Startin$
Jun  7 15:02:48 mail2 fail2ban.server[5811]: INFO Changed logging target to SYSLOG (/dev/log) for Fail2ban v0.9.3
Jun  7 15:02:48 mail2 fail2ban.database[5811]: INFO Connected to fail2ban persistent database '/var/lib/fail2ban/fai$
Jun  7 15:02:48 mail2 fail2ban.jail[5811]: INFO Creating new jail 'sshd'
Jun  7 15:02:48 mail2 fail2ban.jail[5811]: INFO Jail 'sshd' uses pyinotify
Jun  7 15:02:48 mail2 fail2ban.filter[5811]: INFO Set jail log file encoding to UTF-8
Jun  7 15:02:48 mail2 fail2ban.jail[5811]: INFO Initiated 'pyinotify' backend
Jun  7 15:02:48 mail2 fail2ban.filter[5811]: INFO Set findtime = 600
Jun  7 15:02:48 mail2 fail2ban.filter[5811]: INFO Set maxRetry = 5
Jun  7 15:02:48 mail2 fail2ban.filter[5811]: INFO Added logfile = /var/log/auth.log
Jun  7 15:02:48 mail2 fail2ban.filter[5811]: INFO Set jail log file encoding to UTF-8
Jun  7 15:02:48 mail2 fail2ban.actions[5811]: INFO Set banTime = 600
Jun  7 15:02:48 mail2 fail2ban.filter[5811]: INFO Set maxlines = 10
Jun  7 15:02:48 mail2 fail2ban.server[5811]: INFO Jail sshd is not a JournalFilter instance
Jun  7 15:02:48 mail2 fail2ban.jail[5811]: INFO Jail 'sshd' started
Jun  7 15:02:48 mail2 systemd[1]: Started Fail2Ban Service.
Jun  7 15:03:05 mail2 fail2ban.server[5811]: INFO Stopping all jails
Jun  7 15:03:06 mail2 fail2ban.jail[5811]: INFO Jail 'sshd' stopped
Jun  7 15:03:06 mail2 fail2ban.server[5811]: INFO Changed logging target to SYSLOG (/dev/log) for Fail2ban v0.9.3
Jun  7 15:03:06 mail2 fail2ban.server[5811]: INFO Changed logging target to SYSLOG (/dev/log) for Fail2ban v0.9.3
Jun  7 15:03:06 mail2 fail2ban.jail[5811]: INFO Creating new jail 'sshd'
Jun  7 15:03:06 mail2 fail2ban.jail[5811]: INFO Jail 'sshd' uses pyinotify
Jun  7 15:03:06 mail2 fail2ban.filter[5811]: INFO Set jail log file encoding to UTF-8
Jun  7 15:03:06 mail2 fail2ban.jail[5811]: INFO Initiated 'pyinotify' backend
Jun  7 15:03:06 mail2 fail2ban.actions[5811]: INFO Set banTime = 600
Jun  7 15:03:06 mail2 fail2ban.filter[5811]: INFO Set maxRetry = 5
Jun  7 15:03:06 mail2 fail2ban.filter[5811]: INFO Added logfile = /var/log/auth.log
Jun  7 15:03:06 mail2 fail2ban.filter[5811]: INFO Set jail log file encoding to UTF-8
Jun  7 15:03:06 mail2 fail2ban.filter[5811]: INFO Set findtime = 600
Jun  7 15:03:06 mail2 fail2ban.filter[5811]: INFO Set maxlines = 10
Jun  7 15:03:06 mail2 fail2ban.server[5811]: INFO Jail sshd is not a JournalFilter instance
Jun  7 15:03:06 mail2 fail2ban.jail[5811]: INFO Jail 'sshd' started

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: fail2ban is failing to ban

chc-pr wrote:

Jun  7 15:02:47 mail2 fail2ban-client[5802]: ERROR  Failed to access socket path: /var/run/fail2ban/fail2ban.sock. I$

Seems Fail2ban cannot create a socket file "/var/run/fail2ban/fail2ban.sock"?

5 Reply by chc-pr

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

Thank you for your replies, it really is appreciated.

fail2ban.sock exists, and has srwx------root.root
I have not touched these settings so these are the original from the iRedmail installation.

There is one further file in that directory fail2ban.pid

Any clues as to what to do now?  Are the ownerships/access rights correct for that file?

ZhangHuangbin wrote:
chc-pr wrote:

Jun  7 15:02:47 mail2 fail2ban-client[5802]: ERROR  Failed to access socket path: /var/run/fail2ban/fail2ban.sock. I$

Seems Fail2ban cannot create a socket file "/var/run/fail2ban/fail2ban.sock"?

6 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: fail2ban is failing to ban

Wait a minute, you have all jails in "fail2ban.local"? it should be in "jail.local".

7 Reply by chc-pr

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

ZhangHuangbin wrote:

Wait a minute, you have all jails in "fail2ban.local"? it should be in "jail.local".

Thank you Zhang. This may be the problem. There is a jail.conf in /etc/fail2ban but no jail.local. I will cp fail2ban.local to jail.local and see what happens.

Just for the record, I did not create the files ... this was done by the installation script. Oddly, a clean install on a new VM with nothing else on it DOES have the jail.local but that too does not block the failed attempts ...

I will copy to jail.local and report back.  Thanks again.

8 Reply by chc-pr (edited by chc-pr 2017-06-10 21:11:53)

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

I have now run a few more tests to isolate the issues.
Let me start with the good news, I spun up another new server and installed iredmail again - this time using nginx as the webserver option and all the jails worked - so it is a problem related to the Apache configs.

I also spun up a new server and installed a virgin iredmail with Apache.  It had jail.local but no jails were being created.  When I moved the jail.local to fail2ban.local I got sshd jail coming up but nothing else.  I have to say I am really confused by this behaviour.

9 Reply by chc-pr (edited by chc-pr 2017-06-11 03:41:13)

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

Reading the roadmap sticky, it is clear that nginx is the preferred webserver anyway and as fail2ban seems to work with that installation I am going to move everything onto that.  I have built a test system for the additional services I need and all seems fine.  Need to do a bit more testing on that but overall it seems to be working.

Thank you Zhang, I really appreciate the time you gave me.

I have sent you a coffee :-)

10 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: fail2ban is failing to ban

Thank you for your coffee.

But this sounds like a bug of iRedMail installer, i need to test it.

11 Reply by chc-pr (edited by chc-pr 2017-06-11 05:29:01)

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

Hmmm, more testing has revealed that wrt the fail2ban issue Roundcube and sshd ban successfully, SOGo does NOT.  I have not checked postfix, dovecot or the ddos jails as I have not really worked out how to do that yet, but clearly something is less than healthy. The jails are all appearing now which they did not with the Apache install, but there is still not proper blocking.

The no working fail2ban issue has now been exhibited on a total of 5 different fresh server installs (all Ubuntu 16.04.02).  Is the problem Ubuntu, fail2ban itself or the installer/default configurations?

12 Reply by chc-pr

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

FYI, here is the iptables output for the nginx test server.  With the Apache install only the f2b-sshd was showing up.

Are there any other missing jails?

root@nginx1:/etc/fail2ban/filter.d# iptables -L -n -v
Chain INPUT (policy DROP 1661 packets, 264K bytes)
pkts bytes target     prot opt in     out     source               destination
  218 28580 f2b-sogo   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            multiport dports 80,443,25,587,110,995,143,993,4190
  218 28580 f2b-postfix  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            multiport dports 80,443,25,587,110,995,143,993,4190
  218 28580 f2b-dovecot  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            multiport dports 80,443,25,587,110,995,143,993,4190
  218 28580 f2b-roundcube  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            multiport dports 80,443,25,587,110,995,143,993,4190
  153 24492 f2b-postfix  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            multiport dports 80,443,25,587,110,995,143,993,4190
1306  108K f2b-sshd-ddos  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            multiport dports 22
1310  108K f2b-sshd   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            multiport dports 22
174K   30M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
  873 52380 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 8
    1    52 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80
  225 11420 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:25
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:587
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:110
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:995
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:143
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:993

Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 15998 packets, 1614K bytes)
pkts bytes target     prot opt in     out     source               destination

Chain f2b-dovecot (1 references)
pkts bytes target     prot opt in     out     source               destination
  218 28580 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain f2b-postfix (2 references)
pkts bytes target     prot opt in     out     source               destination
  371 53072 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain f2b-roundcube (1 references)
pkts bytes target     prot opt in     out     source               destination
  153 24492 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain f2b-sogo (1 references)
pkts bytes target     prot opt in     out     source               destination
  218 28580 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain f2b-sshd (1 references)
pkts bytes target     prot opt in     out     source               destination
1310  108K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain f2b-sshd-ddos (1 references)
pkts bytes target     prot opt in     out     source               destination
1306  108K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

13 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: fail2ban is failing to ban

chc-pr wrote:

Hmmm, more testing has revealed that wrt the fail2ban issue Roundcube and sshd ban successfully, SOGo does NOT.

How did you test the ban? And any log in fail2ban log file?

14 Reply by chc-pr (edited by chc-pr 2017-06-12 15:05:31)

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

ZhangHuangbin wrote:
chc-pr wrote:

Hmmm, more testing has revealed that wrt the fail2ban issue Roundcube and sshd ban successfully, SOGo does NOT.

How did you test the ban? And any log in fail2ban log file?

Zhang, here is what I did

1 - set the ignoreip to a single IP address on my network only (so I would not get locked out)
2 - set the ban time to 5 minutes and number of attempts to 3 so I could run repeated tests, but also check the iptables outputs
3 - opened a VM and opened the Roundcube and SOGo pages in turn
4 - for each  of the two webmail servers, I first made sure I was able to logon successfully. I then made several 'failed' login attempts using correct usernames but dummy passwords then checked the iptables results after 4 fails. Waited for the timeouts and redid the tests for the other webmail server.  Repeated this step twice to confirm the results.

I will clear the log files, redo the tests and send the results to you. I can also do both the Apache (which only blocks sshd) and Nginx installs (which blocks sshd and Roundcube but not SOGo) as I still have both VMservers set up. These are test installs so I can get it all tested and working before going live with it - so making any tests you need do not in any way impact my existing (hopefully soon to be replaced) server.

I am also looking at doing some pentesting of the SMTP and other modules, but mostly I just want to be sure that the servers are reasonably secure from spammers.

15 Reply by chc-pr

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

I was doing some more tests on the Apache install today and stopped and restarted fail2ban.  Got some interesting outputs which may be of help

root@mail2:/etc/fail2ban# fail2ban-client stop
ERROR  Failed to access socket path: /var/run/fail2ban/fail2ban.sock. Is fail2ban running?
root@mail2:/etc/fail2ban# fail2ban-client start
ERROR  Error in action definition iptables-multiport[name=postfix, port="http,https,smtp,submission,pop3,pop3s,imap,imaps,sieve", protoc$
ERROR  Errors in jail 'postfix-sasl'. Skipping...
ERROR  Error in action definition iptables-multiport[name=roundcube, port="http,https,smtp,submission,pop3,pop3s,imap,imaps,sieve", prot$
ERROR  Errors in jail 'roundcube-iredmail'. Skipping...
ERROR  Error in action definition iptables-multiport[name=dovecot, port="http,https,smtp,submission,pop3,pop3s,imap,imaps,sieve", protoc$
ERROR  Errors in jail 'dovecot-iredmail'. Skipping...
ERROR  Error in action definition iptables-multiport[name=postfix, port="http,https,smtp,submission,pop3,pop3s,imap,imaps,sieve", protoc$
ERROR  Errors in jail 'postfix-iredmail'. Skipping...
ERROR  Error in action definition iptables-multiport[name=sogo, port="http,https,smtp,submission,pop3,pop3s,imap,imaps,sieve", protocol=$
ERROR  Errors in jail 'sogo-iredmail'. Skipping...
root@mail2:/etc/fail2ban#

I think the probable cause of the non-working fail2ban is pretty clear, I just do not know how to go about fixing it. What I do know is that the issues are part of the default config - so it is likely that many others who think they have a working fail2ban setup probably do not.

16 Reply by chc-pr (edited by chc-pr 2017-06-14 17:50:45)

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

Well, I have removed the errors on starting by simply changing the protoc$ bits at the end to protocol=tcp] in all cases (I took this from the working sshd jail config above.

So, this looks like the reason these jails were not running because iptables -L -n now brings up all the jails. 

Thank you.

17 Reply by chc-pr

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

I have now identified that there appear to be two problems with the regex expression in sogo-auth.conf.
1 - the regex expression starts with sogod rather than the date expression.  adding \w{3} \d{2} \d{2}:\d{2}:\d{2} immediately after the start ^ works according to the regex tester I used.

2 - I think for some reason the '<HOST>' bit doesn't seem to pull out the host ip address.  When I use the regex tester it (not surprisingly) doesn't like the <HOST> variable.  Replacing this with \d+.\d+.\d+.\d+ gives a match against the log entry, so I see no reason why the <HOST> entry should not work except that the sogo jail never picks up any entries.

18 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,790

Re: fail2ban is failing to ban

sogo-auth.conf is shipped by Fail2ban, so you must report this issue to Fail2ban directly, it has an issue tracker:
https://github.com/fail2ban/fail2ban/issues

19 Reply by chc-pr

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

ZhangHuangbin wrote:

sogo-auth.conf is shipped by Fail2ban, so you must report this issue to Fail2ban directly, it has an issue tracker:
https://github.com/fail2ban/fail2ban/issues

Thanks Zhang. I will go do that. You have done such a brilliant job typing it all together I am never quite sure what is your work and what is the opensource package :-)

20 Reply by chc-pr

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

After a bit of too'ing and fro'ing with the fail2ban people, it looks like the problem is that the timestamp in the log file is 5 hours  different to the system time. Any ideas on how to correct that? And while I am at it, any reason for not updating fail2ban to a newer version? Apparently 0.9.3 is quite old .. current version is 1. something

21 Reply by chc-pr

  • chc-pr
  • Member
  • Offline
  • Registered: 2017-05-12
  • Posts: 60

Re: fail2ban is failing to ban

I am going to start a new thread as I think this one has reached the end of the first problem and maybe someone else will see the new thread and have an answer.