We run some webservers at work as a part of the services we offer our customers, we’re not a large host by a long shot, but we still get a large amount of brute force attacks against the WordPress installs we run.
I’ve been monitoring the servers and just manually dealing with them using a quick firewall block whenever I’ve seen these in the past, but unfortunately we’ve passed the point of this being a viable solution.
Enter fail2ban.

fail2ban lets you create custom regex rules to match addresses hitting server events within a given time period, and it will then automatically block access to predefined ports via IPTables for a given amount of time.
I did some reading and saw that this could be used against your webserver logs (in our case, apache2), so I set up all logs to also be written to a centralized location.
I’ve been adjusting our rules a little here and there for the past week, and I think I’ve found one that works relatively well, yet doesn’t lock out real users who are doing weird things (it happens, you’d be surprised)
For the fail2ban jail.conf file, I’ve gone with the following

[wordpress]

enabled  = true

bantime  = 900

port     = http,https

filter   = wordpress

logpath  = /var/log/apache2/other_vhost_access.log

findtime = 8

maxretry = 5

The gist of the…well… gist… is as follows;

• Enable the rule labeled wordpress
• If the rule is matched, ban the IP for 900 seconds (15 minutes)
• The ports to block should be http (80) and https (443)
• This rule applies to the filter named wordpress.conf
• The path to the centralized log file
• The seconds to retain a match
• The amount of entries within the retained time before the ban is issued

So basically, if your IP matches my regex 5 times within 8 seconds, you get banned for 15 minutes
Next up, our regex config file, located in filter.d/wordpress.conf

# Fail2Ban configuration file

#

# Author: Marius

#

[INCLUDES]

before = common.conf

[Definition]

# Option:  failregex

# Notes.:  regex to match the password failures messages in the logfile. The

#          host must be matched by a group named "host". The tag "<HOST>" can

#          be used for standard IP/hostname matching and is only an alias for

#          (?:::f{4,6}:)?(?P<host>\S+)

# Values:  TEXT

#

# Different events to match against

# 

# 64.34.173.227 - - [06/Nov/2014:08:39:41 +0100] "POST /wp-login.php HTTP/1.1" 200 1922 "http://www.clorith.net/wp-login.php" "[% tools.ua.random() %]"

# 37.57.231.204 - - [19/Nov/2014:13:43:41 +0100] "POST http://www.clorith.net/wp-login.php/ HTTP/1.1" 200 5761 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.0.2) Gecko/2008091620 Firefox/3.0.2"

failregex = ^ .+?POST.+?wp-login\.php.+?"

# Option:  ignoreregex

# Notes.:  regex to ignore. If this regex matches, the line is ignored.

# Values:  TEXT

#

ignoreregex =

Here we start of by including the common config with all the default data in it
I’ve then got a regex block that’s fairly simple (overly simplistic perhaps?), oh and yeah, this uses the default apache log format common.
Once they were running I started watching the logfile for a while (tail -f /var/log/apache2/other_vhost_access.log | grep wp-login) and I would see 5 hits against wp-login in a row, then nothing, a quick look at the IP tables (iptables -L) would then show the IP addresses that hit the login page listed under the fail2ban-wordpress chain.

Update – 2016-06-28

As the comment from Junior below mentions, this may cause problems for you if you are behind a proxy like CloudFlare.
Luckily, CloudFlare provides the tools to fix that, they have a handy module that translates the IPs for you so that on your server end you always have the legitimate IPs, so if you do use CloudFlare and want that extra layer of brute force protection, you really should have their module running as well.
https://www.cloudflare.com/resources-downloads/