Most people are familiar with the fail2ban package available for protecting against brute-force password attacks against your servers.
Fail2ban scans log files like /var/log/pwdfail or /var/log/apache/error_log and bans IP that makes too many password failures. It updates firewall rules to reject the IP address.
What you might not have experimented with is it's ability to execute custom actions instead (or as well as) blocking the source IP in iptables.