Background

So I was having an issue on some Plesk servers with customers being unable to send outgoing email at times, or worse, their web applications that use smtp to localhost having the same issue.  The problem was primarily occurring on servers set up for shared hosting that had a large number of IP addresses.  A little log analysis revealed that the issue was twofold:

1. Due to having a large number of IP’s, scriptkiddie connections to port 587 testing random short email usernames (which I always disable anyway) were really flooding the server since the idiots would test one IP after another not realizing they’re testing the same accounts on the same server repeatedly.
2. Just a ton of inbound RBL-listed spam attempts to port 25; on a server with 300 domains I was seeing an average of 600 to 1500 connections per minute from IP’s listed by either Spamcop or Spamhaus, my two go-to RBL’s.

I read about fail2ban as being a possible option to combat this; it sounded good on paper, i.e. watches logs, looks for repeated rejections from an IP, bans it for a defined amount of time.  Even easier, I found I could install it from EPEL:  http://fedoraproject.org/wiki/EPEL

Install

So basically my install steps were:

1. (CentOS 6-specific)  rpm -ivh http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm (works for 32-bit and 64-bit)
2. yum install fail2ban -y

I included the -y in the above only because I know what it’s going to install after doing it the right way first and you’re safe to have it auto install the pre-req’s; it only had two required packages not already on my system (python-inotify and gamin-python).

The fail2ban rpm package will store the config files under /etc/fail2ban.  Those files are:

fail2ban.conf – This file contains just the very general logging config and the location of the socket and pid files.  The fail2ban default is to log to a log file /var/log/fail2ban.log.  However, the EPEL package default is to log via syslog.  I left it as the default, but I mention this in case people are installing this after having used the source version in the past.  If you need to change things in here, copy the file to fail2ban.local and edit that copy instead.  That prevents your settings from being overridden by updates to the package that replace fail2ban.conf.  The .local equivalent overrides the .conf file.

jail.conf – This is going to need some edits as it is the config file that defines what all should be watched and what to do.  It defaults to having a huge array of things to look for, all disabled by default, but still present in the file.  You can choose to leave the ones that aren’t applicable to your environment, or strip them out.  I chose to mark everything in jail.conf as ‘enabled = false’, copied the file to jail.local, then edited that file and took out everything but what I wanted because I prefer a cleaner config file when I’m going in looking for something.  Here’s my changes:

• Changed usedns from warn to no.  By default it will warn which means when it sees a hostname in the log file, it does a lookup to get the IP for logging purposes.  I get far too many connections per minute to want to have time wasted doing DNS lookups.  I generally have DNS lookups turned off on the services that are doing the logging as well.
• Left only the following three service definitions in the new jail.local file to block FTP and SSH attempts for one day using a linux route since it’s more efficient than iptables and will block from all services, and still using iptables for email-specific blocking in case it’s a legit customer, with only a one hour block after 10 attempts:

[proftpd-iptables]

enabled  = true
filter   = proftpd
action   = iptables[name=ProFTPD, port=ftp, protocol=tcp]
logpath  = /var/log/secure
maxretry = 5
bantime = 86400

# Here we use blackhole routes for not requiring any additional kernel support
# to store large volumes of banned IPs

[ssh-route]

enabled = true
filter = sshd
action = route
logpath  = /var/log/secure
maxretry = 5
bantime = 86400

[postfix-iptables]

enabled  = true
filter   = postfix
action   = iptables-multiport[name=smtp, port="25,465,587,110,995,143,993", protocol=tcp]
logpath  = /usr/local/psa/var/log/maillog
maxretry = 10
bantime = 3600

The above is an edited down version after my initial setup.  For the initial attempts, I wanted a much better view of what was getting blocked to make sure my rules weren’t faulty, so under each action line I had a second entry:

sendmail-whois[name=FTPD, dest=me@alert.destination.com, sender=fail2ban@my.server.com]

That would send me alerts with each ban and ban removal.

action.d/route.conf – This file defines what the action taken is when the action of ‘route’ is specified in the jail.conf for a given monitor.  By default, it uses the command ‘ip route add unreachable ‘ to block someone.  I copied the file to route.local and changed unreachable to blackhole because I prefer to drop the packets with no ICMP response rather than generate unnecessary traffic.

filter.d/proftpd.conf – This file defines what to look for in the specified log file for entries related to ProFTPd that should result in a ban.  I had to modify this to match how Plesk’s version of ProFTPd in Plesk 11 logs things, which is the format:

Jul 17 02:03:02 server1 proftpd[9907]: 192.0.2.1 (120.33.241.46[120.33.241.46]) - USER admin: no such user found from 120.33.241.46 [120.33.241.46] to 192.0.2.1:21

So I copied the file to proftpd.local and replaced the failregex line and the three that follow it with just one entry:

failregex = .*proftpd\[\S+\]: \S+ \(\S+\[<HOST>\]\) - USER \S+: no such user found from .*$

filter.d/sshd.conf – This file defines what to look for in the specified log file for entries related to SSH that should result in a ban.  I had to modify this one as well; not sure if it’s Plesk’s doing or just OpenSSH on CentOS 6 differs from the default fail2ban filters, but none matched the format I see in my logs:

Jul 18 22:11:12 server1 sshd[23862]: Failed password for invalid user public from 61.240.36.1 port 58422 ssh2

I also don’t use allowusers/denyusers, group restrictions, etc. so I copied it to sshd.local and just got rid of all the default regex’s to cut down on processing time and went with just this one line replacement for the failregex:

failregex = ^%(__prefix_line)sFailed (?:password|publickey) for .* from <HOST>(?: port \d*)?(?: ssh\d*)?\s*$

filter.d/postfix.conf – This file defines what to look for in the specified log file for entries related to Postfix that should result in a ban.  I had to modify this a bit because the default doesn’t catch quite what I want; i.e. I don’t want to just block IP’s because of delivery failures but I do want to block them if they are blocked for being on an RBL and keep trying to deliver even after getting a permanent rejection notice.  I also want to block people trying to authenticate repeatedly via POP3/IMAP/SMTP.  So these were the log entries I based my patterns on:

Jul 19 09:54:31 server1 postfix/smtpd[7009]: NOQUEUE: reject: RCPT from unknown[41.198.201.132]: 554 5.7.1 Service unavailable; Client host [41.198.201.132] blocked using bl.spamcop.net; Blocked - see http://www.spamcop.net/bl.shtml?41.198.201.132; from=<hamlin.chatteloch@altrasystems.com> to=<user@domain.com> proto=ESMTP helo=<41-198-201-132.iwayafrica.net>

However, there were also some entries I did not want to block that you may be inclined to accidentally overlook if you’re running a hosting server.  In the environment I was working on, many customers were using third party anti-spam relay services such as Postini, MX Logic, Spam Soap, etc.  The way these types of services work, MX records point a given domain’s mail to the anti-spam vendor, they filter, then they deliver the mail to mail.domain.com via SMTP.  These types of vendors often operate in two ways; quick and easy or requiring configuration.  Quick and easy means they don’t have to be told which accounts actually exist because they’ll just try to deliver and learn on their own.  Most customers go that route and the problem you’ll run into is if you have fail2ban watching for log entries like the following:

Jul 19 09:53:41 server1 postfix/smtpd[16454]: NOQUEUE: reject: RCPT from na3sys009amx240.postini.com[74.125.149.124]: 554 5.7.1 <f0ce191@domain.com>: Relay access denied; from=<f0ce191yaxvr@altren.com> to=<f0ce191@domain.com> proto=SMTP helo=<psmtp.com>

You could end up blocking a legit Postini server from delivering email to your customers.  With this being the case, I decided to make a regex that only blocks addresses rejected for blacklisting when it comes to SMTP specifically.  I of course also had to deal with POP3/IMAP; those entries look like:

Jul 19 07:44:28 server1 pop3d: IMAP connect from @ [::ffff:208.90.197.194]checkmailpasswd: FAILED: matt - short names not allowed from @ [::ffff:208.90.197.194]DEBUG: Connection, ip=[::ffff:208.90.197.194]

Jul 19 08:26:06 server1 postfix/smtpd[21612]: warning: unknown[142.0.41.169]: SASL LOGIN authentication failed: authentication failure

Jul 19 08:15:31 server1 pop3d: LOGIN FAILED, ip=[::ffff:216.214.244.126]

Jul 19 08:15:31 server1 imapd-ssl: IMAP connect from @ [::ffff:173.162.145.213]IMAP connect from @ [::ffff:173.162.145.213]ERR: LOGIN FAILED, ip=[::ffff:173.162.145.213]

Jul 19 08:01:06 server1 smtp_auth: FAILED: doug - password incorrect from smtp169.abstractexponent.com [142.0.41.169]

Jul 19 08:00:29 server1 pop3d: IMAP connect from @ [::ffff:86.147.247.241]ERR: LOGIN FAILED, ip=[::ffff:86.147.247.241]

Jul 19 08:00:29 server1 pop3d: LOGIN FAILED, ip=[::ffff:86.147.247.241]

So to block those I copied the file to postfix.local and used the following regexes:

failregex = : LOGIN FAILED, ip=\[<HOST>\]
            (?i): warning: \S+\[<HOST>\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed: \w
            smtp_auth: FAILED: \S+ - password incorrect from .*\[<HOST>\]
            5\.7\.1 Service unavailable; Client host \[<HOST>\] blocked using

Implementation

Since we installed via rpm, all the init scripts were already created.  Just need to take two additional steps:

1. Set fail2ban to start with the system: 

   chkconfig --level 3456 fail2ban on

2. Go ahead and start it up: /etc/rc.d/init.d/fail2ban start
3. That should be about it.  If you have it set to email you when bans occur, you’ll start getting emails.
4. If you run ‘iptables -L’ and ‘netstat -nr’ you’ll start to see the IP’s it blocks building up.  If you stop the program using the initscript, it will clear out the bans.

It’s worth noting that Fail2ban currently does NOT support IPv6.

Problems

Only issue I’ve run into after implementation is that, in very rare circumstances,  some users with broken email clients that do not support DIGEST-MD5 authentication correctly will end up getting locked out.  Fix for that is here.