Catch SMTP Brute-Force on Port 587 with Fail2ban

Password-guessing against the submission port (587) is constant background noise on any internet-facing mail server. Each attempt shows up in your mail log as a SASL authentication failure, and the standard, boring, reliable way to act on a pattern of them is Fail2ban: it tails the log, counts failures per source IP, bans the IP in your firewall, and can email you when it does. This guide wires that up for the submission service specifically.

Before you run this

What this does: It configures Fail2ban to watch your Postfix mail log for repeated SASL authentication failures, ban the offending IP at the firewall after a threshold, and send you an email with a whois report when a ban happens. It does not change how Postfix accepts or authenticates mail — it only reacts to failures Postfix has already logged.

Privileges: You need root or sudo. Editing files under /etc/fail2ban, restarting the service, and inserting firewall rules all require it. Fail2ban runs as a root service.

Test first, and don't ban yourself. The dangerous mistake here is banning your own management or relay IPs. Before enabling anything, add your trusted networks to ignoreip (shown below), and keep a separate SSH session open while you test so a misconfigured ban can't lock you out of the box. Validate your filter against the real log with fail2ban-regex (shown at the end) before starting the jail.

What it changes, and how reversible it is: Fail2ban inserts firewall rules (iptables or nftables, depending on your distro's banaction) that drop traffic from banned IPs. Those bans are fully reversible — you unban with one command, and they expire on their own after bantime. The config edits are plain text files you can revert. Nothing here is irreversible, but a wrong ignoreip or an over-aggressive maxretry can block legitimate senders, so treat threshold changes with care.

Assumptions for this guide:

• Debian 12 or Ubuntu 22.04, systemd.
• Postfix handling submission on 587, with SASL auth provided by Dovecot (the common iRedMail-style setup).
• rsyslog is installed and Postfix logs to /var/log/mail.log. If your box logs only to the journal, see the backend = systemd note below.
• Fail2ban from the distro repos (0.11.x on these releases).

If you're on RHEL/Alma, the package and commands are the same, but the log file is /var/log/maillog and the default banaction differs — adjust logpath accordingly.

Confirm what Postfix is logging

A SASL failure on submission looks like this in /var/log/mail.log:

postfix/submission/smtpd[12345]: warning: unknown[203.0.113.50]: SASL LOGIN authentication failed: UGFzc3dvcmQ6

Check you're actually seeing these before building anything around them:

grep "SASL .* authentication failed" /var/log/mail.log | tail

If that returns nothing, either you haven't been probed yet or your SASL failures are logged elsewhere — fix that before continuing, because Fail2ban can only act on what's in the log.

Install Fail2ban

sudo apt update
sudo apt install fail2ban whois

whois is optional but needed for the whois-enriched email action we'll use. Fail2ban ships a postfix filter that understands SASL failures via its auth mode, and a matching postfix-sasl jail is defined (disabled) in the stock jail.conf. We won't edit the stock files — we override in jail.local.

Configure the jail

Create /etc/fail2ban/jail.local. Never edit jail.conf directly — package updates overwrite it, and jail.local always wins.

[DEFAULT]
# Your own IPs/networks that must never be banned.
# Replace with your real management and relay addresses.
ignoreip = 127.0.0.1/8 ::1 198.51.100.0/24

# Ban for 1 hour; count failures within a 10-minute window.
bantime  = 1h
findtime = 10m
maxretry = 5

# Email alerting
destemail = you@example.com
sender    = fail2ban@example.com
mta       = sendmail

# action_mwl = ban + email with whois report AND the matching log lines.
# Use action_mw (no log lines) or action_ (ban only, no mail) if you prefer.
action = %(action_mwl)s

[postfix-sasl]
enabled  = true
filter   = postfix[mode=auth]
port     = submission,smtp,465,imap,imaps,pop3,pop3s
logpath  = /var/log/mail.log
backend  = %(postfix_backend)s

Notes on the choices:

• filter = postfix[mode=auth] is the stock postfix filter run in authentication mode — this is what matches the SASL-failure lines. I'm using the mode parameter rather than inventing a filter name.
• The port list covers submission plus the other SASL-speaking services, since the same filter catches failures from all of them. If you want to ban only submission offenders, narrow port to submission.
• maxretry = 5 within findtime = 10m is a reasonable starting threshold. A real user fat-fingering a password a couple of times won't trip it; a bot hammering credentials will. Tune after you've watched it for a day.
• If your server logs mail only to the systemd journal (no /var/log/mail.log), remove the logpath line from the jail and set backend = systemd instead.

Keep the email settings honest: mta = sendmail assumes Postfix's sendmail binary is present, which it is on a Postfix host. If mail from the server isn't flowing, the alerts won't either — test that separately.

Start it and confirm it's live

# Validate config syntax before restarting
sudo fail2ban-client -t

sudo systemctl enable --now fail2ban
sudo systemctl restart fail2ban

fail2ban-client -t parses your config and reports errors without touching the running service — run it every time you edit jail.local.

Verify it works

First, confirm the jail is loaded and watching the log:

sudo fail2ban-client status
sudo fail2ban-client status postfix-sasl

The second command shows the file list being monitored, the number of matched failures, and the currently banned IPs. If Currently banned grows over time, it's working.

Second, test the filter against your actual log without banning anyone. fail2ban-regex dry-runs the regex and tells you how many lines matched:

sudo fail2ban-regex /var/log/mail.log "postfix[mode=auth]"

Look at the Lines: X lines, 0 ignored, N matched summary. If N is 0 but grep earlier found failures, your log format or path doesn't match — fix that before trusting the jail.

Third, confirm the ban action actually touches the firewall. After an IP is banned, list the active bans:

sudo fail2ban-client get postfix-sasl banip

To see the corresponding firewall entries, inspect whichever backend your system uses — sudo iptables -L -n or sudo nft list ruleset — and look for the Fail2ban chain. You should also receive the alert email at destemail on the first ban.

Undo and roll back

Unban a single address (do this immediately if you've blocked a legitimate sender):

sudo fail2ban-client set postfix-sasl unbanip 203.0.113.50

Unban everything in the jail at once:

sudo fail2ban-client unban --all

Disable the jail without uninstalling — set enabled = false under [postfix-sasl] in jail.local, then sudo fail2ban-client -t && sudo systemctl restart fail2ban.

Remove it entirely:

sudo systemctl disable --now fail2ban
sudo apt remove fail2ban

Bans also clear themselves after bantime elapses, and all of Fail2ban's firewall rules are flushed when the service stops — so nothing you do here is permanent. Start with conservative thresholds, watch fail2ban-client status postfix-sasl for a few days, and tighten maxretry/findtime once you trust what it's catching.

For the authoritative reference on jail options, filter modes, and action syntax, see the official Fail2ban manual (the jail.conf(5) man page shipped with the package, and the Fail2ban project documentation).