A single mail server is a single point of failure. If its host, its VM, or its entire site goes offline, every domain it hosts stops receiving mail until someone intervenes. In this tutorial I'll walk through building a second, independent iRedMail node at a second site, so that either node can serve every mailbox, and DNS automatically points at whichever one is actually healthy.
I'll call the two nodes mail1 (Site 1) and mail2 (Site 2) throughout. Swap in your own hostnames and IP ranges as you go.
A mail server's state splits into three genuinely different problems, and I found it much easier to reason about once I stopped trying to solve them with one tool.
Accounts, domains, passwords, quotas, DKIM tracking, webmail identities: all of this lives in SQL. I put it on a Galera multi-master cluster spanning both sites. This is the single most load-bearing decision in this whole design: with Galera there's no "promotion" step, no replica that can lag behind, and no window where two nodes disagree about who's authoritative. A write commits everywhere or nowhere; a node cut off from the cluster simply can't accept writes at all. That's what actually prevents split-brain, not any cleverness on my part.
Mail content itself, the message files on disk, doesn't live in SQL. If you're on Dovecot 2.3, its own built-in replication (dsync) is the right tool for this and I'd use it without hesitation. If you're on Dovecot 2.4, that option is gone; the project removed replication entirely, and its own upgrade docs simply say "use NFS or another shared filesystem instead." I didn't want that either: shared storage across two sites makes the link between them a hard dependency for the passive side to have mail access at all, which defeats the point of having a second site. So I used Syncthing to sync the Maildir tree directly between the two nodes. This is safe specifically because the two nodes are active/passive, not active/active: only one is ever genuinely "live" at a time, so there's no real concurrent-write race, and Maildir's file-per-message layout with effectively unique filenames means two independently-delivered mailboxes merge safely by simple union.
Getting traffic to the healthy node is a third problem neither of the above solves. That's DNS-level failover, covered in Step 10.
garbd) on a third, independent site to break ties. Percona XtraDB Cluster and MySQL InnoDB Cluster both work the same way if you'd rather use those.iRedMail expects six databases: one for mailboxes and domains, and one each for Amavis, iRedAdmin, RoundCube, iRedAPD, and Fail2ban. Rather than let each node create its own local copies, I create these once, directly on the Galera cluster, and give each service its own scoped user reachable from both sites' subnets.
CREATE DATABASE IF NOT EXISTS mail_vmail CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
CREATE DATABASE IF NOT EXISTS mail_amavisd CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
CREATE DATABASE IF NOT EXISTS mail_iredadmin CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
CREATE DATABASE IF NOT EXISTS mail_roundcubemail CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
CREATE DATABASE IF NOT EXISTS mail_iredapd CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
CREATE DATABASE IF NOT EXISTS mail_fail2ban CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
-- one pair of users per service: a read-only "bind" user for Postfix/Dovecot
-- lookups, and a full-privilege admin user for iRedAdmin/iRedAPD/etc.
-- Scope each to the two site subnets that actually need access.
CREATE USER 'mail_vmail'@'<site1-subnet>' IDENTIFIED BY '<password>';
CREATE USER 'mail_vmail'@'<site2-subnet>' IDENTIFIED BY '<password>';
GRANT SELECT ON mail_vmail.* TO 'mail_vmail'@'<site1-subnet>';
GRANT SELECT ON mail_vmail.* TO 'mail_vmail'@'<site2-subnet>';
CREATE USER 'mail_vmailadmin'@'<site1-subnet>' IDENTIFIED BY '<password>';
CREATE USER 'mail_vmailadmin'@'<site2-subnet>' IDENTIFIED BY '<password>';
GRANT ALL PRIVILEGES ON mail_vmail.* TO 'mail_vmailadmin'@'<site1-subnet>';
GRANT ALL PRIVILEGES ON mail_vmail.* TO 'mail_vmailadmin'@'<site2-subnet>';
-- repeat the same CREATE USER / GRANT pair for mail_amavisd, mail_iredadmin,
-- mail_roundcubemail, mail_iredapd and mail_fail2ban, each with their own
-- dedicated userRun this against any node in the cluster (or a VIP in front of it); Galera's synchronous replication means it's immediately live everywhere.
iRedMail's own installer doesn't support pointing directly at a remote database; it always sets up a local MariaDB during install. So the pattern for each node is: install normally against a disposable local database, then immediately repoint every component at the shared Galera cluster and turn the local one off for good. Do this identically on both mail1 and mail2.
Install the base packages and download iRedMail:
sudo apt-get update
sudo apt-get install -y wget tar
mkdir -p /tmp/iredmail-build && cd /tmp/iredmail-build
wget -q https://github.com/iredmail/iRedMail/archive/refs/tags/1.8.4.tar.gz -O iRedMail-1.8.4.tar.gz
tar zxf iRedMail-1.8.4.tar.gz
cd iRedMail-1.8.4Set the node's own hostname (distinct per node; the shared mail identity gets configured separately in Step 3):
sudo hostnamectl set-hostname mail1.yourdomain.comBuild an unattended install config. iRedMail supports a documented unattended path: a plain config file of environment variables, with AUTO_USE_EXISTING_CONFIG_FILE=y telling the installer to skip its interactive wizard entirely.
genpw() { tr -dc "A-Za-z0-9" < /dev/urandom | head -c "${1:-24}"; }
cat > config <<EOF
export WEB_SERVER=NGINX
export BACKEND_ORIG=MARIADB
export BACKEND=MYSQL
export VMAIL_DB_BIND_PASSWD=$(genpw)
export VMAIL_DB_ADMIN_PASSWD=$(genpw)
export MLMMJADMIN_API_AUTH_TOKEN=$(genpw)
export MYSQL_ROOT_PASSWD=$(genpw)
export USE_IREDADMIN=YES
export USE_ROUNDCUBE=YES
export USE_FAIL2BAN=YES
export AMAVISD_DB_PASSWD=$(genpw)
export IREDADMIN_DB_PASSWD=$(genpw)
export RCM_DB_PASSWD=$(genpw)
export IREDAPD_DB_PASSWD=$(genpw)
export FAIL2BAN_DB_PASSWD=$(genpw)
export FIRST_DOMAIN=yourdomain.com
export DOMAIN_ADMIN_PASSWD_PLAIN=$(genpw 20)
EOF
chmod 600 configRun the installer. We answer "no" to iRedMail managing its own firewall rules, since our reverse proxy is already the perimeter, and run it detached so a dropped SSH session doesn't kill a fifteen-minute install:
sudo bash -c "
cd /tmp/iredmail-build/iRedMail-1.8.4
nohup env \
AUTO_USE_EXISTING_CONFIG_FILE=y \
AUTO_INSTALL_WITHOUT_CONFIRM=y \
AUTO_CLEANUP_REMOVE_SENDMAIL=y \
AUTO_CLEANUP_REPLACE_FIREWALL_RULES=n \
AUTO_CLEANUP_RESTART_FIREWALL=n \
AUTO_CLEANUP_REPLACE_MYSQL_CONFIG=y \
DEBIAN_FRONTEND=noninteractive \
bash iRedMail.sh > /tmp/iredmail-install.log 2>&1 < /dev/null &
disown
"Watch for the real completion marker, not just "the process is still running":
tail -f /tmp/iredmail-install.log | grep -m1 "Congratulations\|fatal\|ERROR"Then verify the evidence, not the banner:
sudo postconf -n | grep -i "PH_\|fatal" || echo clean
sudo systemctl is-active postfix dovecot mariadb nginx amavis clamav-daemon fail2ban php8.4-fpm iredapd
sudo systemctl --failed
sudo rebootA reboot is worth doing once, to confirm everything comes up cleanly on its own rather than relying on install-time startup order.
Now repoint each iRedMail component from its disposable local database to the shared one from Step 1. This table is worth keeping open while you edit:
| Component | What it does | Config file |
|---|---|---|
| Postfix | The mail transfer agent: accepts, routes and delivers SMTP mail | /etc/postfix/mysql/*.cf (all files) |
| Dovecot | The IMAP/POP3 server: stores and serves mailbox content to mail clients | /etc/dovecot/dovecot.conf |
| Amavis | Sits between Postfix and the content filters (ClamAV, SpamAssassin), scanning and DKIM-signing mail | /etc/amavis/conf.d/50-user |
| RoundCube | The webmail interface end users log into | config/config.inc.php |
| iRedAdmin | The admin panel for managing domains, mailboxes and aliases | settings.py |
| iRedAPD | Postfix policy daemon: greylisting, throttling, sender/recipient checks | settings.py |
| Fail2ban | Watches auth logs and temporarily bans IPs showing brute-force patterns | /root/.my.cnf-fail2ban |
Each of these needs its host changed from 127.0.0.1 to the Galera cluster's address, and its database name and credentials changed from the local vmail/amavisd/etc. to the shared mail_vmail/mail_amavisd/etc. from Step 1.
One Dovecot-specific detail worth knowing before you start: on Dovecot 2.4, the entire effective configuration lives in one flat
/etc/dovecot/dovecot.conf; the familiarconf.d/*.conflayout iRedMail's installer still generates is not actually read at all. The live database connection is an inline block directly inside that file:mysql 127.0.0.1 { port = 3306 dbname = vmail user = vmailadmin password = ... }Edit this block in place (changing the host,
dbname,userandpassword), not any file underconf.d/. The same applies to Dovecot'shostnamesetting if you set one, and to anything else Dovecot-related: if it isn't indovecot.confitself, Dovecot 2.4 isn't reading it.
After editing every file in the table, restart the affected services and confirm each is reading from Galera, not local, with a real end-to-end test:
sudo systemctl restart postfix dovecot amavis php8.4-fpm nginx iredapd fail2ban
printf "Subject: Galera backend test\n\nTest.\n" | sudo sendmail -f [email protected] [email protected]
sleep 5
sudo doveadm mailbox status -u [email protected] messages INBOXOnce every component is confirmed reading from Galera, on both nodes, stop and disable the local MariaDB entirely rather than leaving it running unused:
sudo systemctl stop mariadb
sudo systemctl disable mariadbRe-run the mail delivery test one more time with local MariaDB fully gone, to prove nothing was still quietly depending on it.
DKIM keys don't need to be shared between nodes. Each node gets its own keypair per domain, published under its own selector, so there's never anything to keep in sync:
sudo amavisd genrsa /var/lib/dkim/yourdomain.com.pem 2048
sudo chown amavis:amavis /var/lib/dkim/yourdomain.com.pem
sudo chmod 600 /var/lib/dkim/yourdomain.com.pemRegister it in Amavis, using a selector name that identifies the node:
dkim_key('yourdomain.com', 'mail1-dkim', '/var/lib/dkim/yourdomain.com.pem'); # on mail1
dkim_key('yourdomain.com', 'mail2-dkim', '/var/lib/dkim/yourdomain.com.pem'); # on mail2Get the DNS value and publish both selectors, for every domain:
sudo amavisd showkeysmail1-dkim._domainkey.yourdomain.com. TXT "v=DKIM1; p=..."
mail2-dkim._domainkey.yourdomain.com. TXT "v=DKIM1; p=..."Whichever node actually signs an outgoing message, its own selector is already live in DNS. If you're renaming a selector on an already-live node, publish the new DNS record first and flip the signing config only afterward, so there's never a window where a node signs with a selector that doesn't resolve yet.
If you already have one mail server running its own local database (the common iRedMail default) and want to bring it into this HA design, the work is almost identical to Steps 2 and 3, just with one extra step first: dump your existing data and import it into the new Galera-hosted databases before repointing, so you don't lose your existing domains, mailboxes and DKIM history.
mysqldump --single-transaction --routines --skip-triggers --no-create-db vmail \
| mysql -h <galera-address> -u mail_vmailadmin -p mail_vmail
# repeat for amavisd, iredadmin, roundcubemail, iredapd, fail2ban--skip-triggers is there because the scoped users deliberately don't have SUPER, which one schema trigger (auto-populating a domain column on insert into used_quota) needs to be created. Recreate that one trigger by hand, once, with full admin credentials:
DELIMITER $$
CREATE TRIGGER used_quota_before_insert
BEFORE INSERT ON used_quota FOR EACH ROW
BEGIN
SET NEW.domain = SUBSTRING_INDEX(NEW.username, '@', -1);
END$$
DELIMITER ;From there, follow the same table in Step 3 to repoint every component, give the node its own DKIM selector as in Step 4, and stop the local database once everything checks out. Your second node then follows the plain fresh-install path in Steps 2 to 4, since it starts empty and just needs to see the same shared data your first node already migrated in.
Install Syncthing on both nodes from the official repository, running as the same system user that already owns the Maildir tree (vmail on a standard iRedMail install):
sudo mkdir -p /etc/apt/keyrings
sudo curl -sL -o /etc/apt/keyrings/syncthing-archive-keyring.gpg https://syncthing.net/release-key.gpg
echo "deb [signed-by=/etc/apt/keyrings/syncthing-archive-keyring.gpg] https://apt.syncthing.net/ syncthing stable-v2" \
| sudo tee /etc/apt/sources.list.d/syncthing.list
sudo apt-get update && sudo apt-get install -y syncthing jqPoint it at a dedicated config directory with a systemd override, and start it as vmail:
# /etc/systemd/system/[email protected]/override.conf
[Service]
ExecStart=
ExecStart=/usr/bin/syncthing serve --home="/opt/syncthing-config" --no-browser --no-restartsudo mkdir -p /opt/syncthing-config && sudo chown vmail:vmail /opt/syncthing-config
sudo systemctl daemon-reload
sudo systemctl enable --now [email protected]Pair the two nodes and share one folder, using Syncthing's own cli rather than hand-editing its config:
# on mail1, using mail2's device ID (find it via `syncthing cli --home /opt/syncthing-config show system`)
syncthing cli --home /opt/syncthing-config config devices add \
--device-id <mail2-device-id> --name mail2 --addresses tcp://<mail2-address>:22000
syncthing cli --home /opt/syncthing-config config folders add \
--id vmail-ha --label "vmail HA sync" --path /var/vmail --type sendreceive
syncthing cli --home /opt/syncthing-config config folders vmail-ha devices add \
--device-id <mail2-device-id>
# repeat symmetrically on mail2, using mail1's device IDScope the sync to actual mail content, not the whole directory. /var/vmail also holds each node's own local SQL backup dump and a couple of unused mailing-list directories, none of which should be merged between nodes, plus Dovecot's own local performance caches, which are unsafe to sync mid-write and rebuild automatically if missing. Exclude all of it with a .stignore file at the folder root:
# host-local, not mail content -- never sync
/backup
/mlmmj
/mlmmj-archive
/pgp-keys
# Dovecot index/cache files: local performance cache only, rebuilt
# automatically if missing. dovecot-uidlist is NOT excluded; it encodes
# IMAP UID assignments and should stay consistent across replicas.
(?d)dovecot.index
(?d)dovecot.index.log*
(?d)dovecot.index.cache*
(?d)dovecot.list.index*sudo touch /var/vmail/.stignore && sudo chown vmail:vmail /var/vmail/.stignoreTurn on trash-can versioning, so an accidental deletion propagating between nodes isn't unrecoverable:
syncthing cli --home /opt/syncthing-config config folders vmail-ha versioning type set trashcan
syncthing cli --home /opt/syncthing-config config folders vmail-ha versioning params set cleanoutDays 14Syncthing needs to create a couple of marker files at the folder root itself (.stfolder, confirming the folder genuinely exists; .stversions, for the trash can). Since /var/vmail's own top-level directory typically isn't writable by the vmail user (only its subdirectories are), pre-create both as root and hand ownership over:
sudo touch /var/vmail/.stfolder && sudo chown vmail:vmail /var/vmail/.stfolder
sudo mkdir -p /var/vmail/.stversions && sudo chown vmail:vmail /var/vmail/.stversions
sudo systemctl restart [email protected]Give it a few minutes, then confirm both sides report the same file count and an idle, error-free state:
API_KEY=$(sudo grep -oP '(?<=<apikey>)[^<]+' /opt/syncthing-config/config.xml)
curl -s "http://127.0.0.1:8384/rest/db/status?folder=vmail-ha" -H "X-API-Key: $API_KEY" \
| grep -E '"state"|"error"|globalFiles|localFiles|needFiles'If both nodes have ever independently run the iRedMail installer, the very first sync can hit a filename collision: the installer sends a fixed-name postmaster welcome email (mua.eml, links.eml, and so on) on every install, and those aren't organically-delivered mail with unique names the way real messages are. Syncthing handles this correctly on its own, keeping both copies and renaming the loser to *.sync-conflict-<timestamp>-<device> rather than silently discarding anything; it's a one-time thing to tidy up, not an ongoing concern.
If your two sites already have a site-to-site VPN between them (WireGuard, IPsec, OpenVPN, whatever), route Galera replication and Syncthing sync over it rather than the public internet. This keeps your two nodes' own housekeeping traffic off the public internet entirely, avoids exposing MySQL and Syncthing ports on either public IP, and, importantly, gives Fail2ban something stable to trust.
That last point matters more than it looks. Testing or monitoring that reaches a node via its public IP (which is exactly what a real inbound sender does) can trip protections like Fail2ban's pregreet filter, which watches for clients that violate SMTP protocol before the server's own greeting. Since Fail2ban's default ignoreip already exempts private RFC1918 ranges, this only bites cross-site traffic that happens to be flowing over the public internet.
The fix isn't to whitelist each site's public IP; that would blind Fail2ban to exactly the kind of traffic it exists to police, since real inbound mail legitimately arrives from the public internet on both sides too. Instead, whitelist the VPN tunnel's own private address range, which only ever carries your own infrastructure traffic:
# on mail1's fail2ban, add mail2's tunnel address (or subnet)
ignoreip = 127.0.0.1 127.0.0.0/8 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 <mail2-tunnel-address>
# on mail2's fail2ban, add mail1's tunnel address (or subnet)
ignoreip = 127.0.0.1 127.0.0.0/8 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 <mail1-tunnel-address>If either site sits behind a dynamic secondary WAN, point the VPN's peer/endpoint setting at that site's dynamic DNS hostname rather than a hard-coded IP. Most VPN implementations, including WireGuard, re-resolve the endpoint hostname automatically on a failed handshake, so the tunnel reconnects on its own if a site fails over to its backup connection, without any manual intervention.
Worth knowing for the failure case: if a site goes down entirely, its side of the tunnel goes down too, and that's fine, since there's no node left there to sync with anyway. The actual mail failover in Step 10 never depends on this tunnel at all; real senders reach the surviving site over the public internet directly. The tunnel exists purely for the two nodes' own private traffic while both sites are up. One edge case worth being aware of: if the tunnel itself drops while both sites are individually still healthy, that looks like a network partition to Galera too, which is exactly why a quorum arbitrator on a third, independent site (mentioned in the prerequisites) matters, so the cluster can still resolve correctly even when it's specifically the link between your two mail sites that's broken.
Two things worth checking before building this:
If you already have Load Balancer pools set up for your website, reuse them for mail rather than creating new ones; most providers, Cloudflare included, bill per origin server, not per Load Balancer, and attaching a second Load Balancer object to already-existing pools costs nothing extra. The trade-off is that mail's failover then rides on the website's own health check (typically HTTP) rather than a mail-specific one (say, a TCP check on port 993). Given that a real site outage takes the website and the mail server down together, that trade-off is often perfectly reasonable; build a dedicated mail-specific health check and pools if you want tighter detection and don't mind the extra cost.
With Cloudflare, creating the mail-specific Load Balancer looks like this:
curl -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE/load_balancers" \
-H "X-Auth-Email: $CF_EMAIL" -H "X-Auth-Key: $CF_KEY" -H "Content-Type: application/json" \
--data '{
"name": "mail.yourdomain.com",
"proxied": false,
"enabled": true,
"steering_policy": "off",
"default_pools": ["<site1-pool-id>"],
"fallback_pool": "<site2-pool-id>",
"ttl": 60
}'"steering_policy": "off" means strict priority failover: use the default pool until it's genuinely unhealthy, then fall back, rather than distributing traffic across both. For mail, a clear "who's in charge right now" answer is what you want, not load balancing.
Each layer built above deserves its own check, since a failure in any one of them can look identical to "everything's fine" from the others' point of view.
Galera cluster health. If you already monitor the cluster for other services, this is likely already covered; otherwise, a monitor that checks wsrep_cluster_status and wsrep_local_state_comment on each node catches a partition or a node falling out of sync.
Syncthing folder state. Deploy a small monitor script on each node that polls Syncthing's local REST API for every folder's status and pushes a consolidated result to your monitoring tool, checking hourly when healthy and retrying every minute when something's wrong:
SYNCTHING_API_KEY="..." # from each node's own Syncthing config
UPTIME_KUMA_PUSH_URL="http://<your-monitoring-host>/api/push/<token>"Run one instance per node, each pushing to its own push token; sharing one push URL between both nodes conflates their statuses into a single indistinguishable signal, exactly the ambiguity this exists to eliminate.
The actual mail-facing surface, which is what your users and remote senders experience directly:
200 response.(screenshot: monitor list showing all checks green for both nodes)
(screenshot: one monitor's detail view, showing its response-time history)
Test by forcing the primary pool unhealthy in your Load Balancer (the same state a real health-check failure would trigger), and confirm, in order:
Revert once satisfied, and confirm DNS flips back within the TTL too.
If mail genuinely isn't getting through during testing (or for real), work through the layers in order rather than guessing:
Is the message reaching Postfix at all, and what happened to it?
sudo tail -f /var/log/mail.logLook for connect from, reject, status=bounced, or status=deferred lines around the time in question. A missing connect from entry for an external sender's IP, with no corresponding bounce visible anywhere, is the specific signature of a NAT or firewall rule not actually forwarding the connection; check both halves of your port-forward configuration (the NAT rule and its paired firewall pass rule), since editing one without the other is an easy, very quiet mistake.
Did the content filter or DKIM signing choke on it?
sudo grep amavis /var/log/mail.log | tail -30Did authentication or final delivery fail?
sudo journalctl -u dovecot -fdoveadm auth test <user> exercises the exact same SQL path a real login uses, without needing a mail client:
echo 'the-password' | sudo doveadm auth test [email protected]Is the sender's IP (or your own testing IP) actually banned?
sudo fail2ban-client status
sudo fail2ban-client status pregreetIs the node actually talking to Galera, and does the account exist there?
mysql -h <galera-address> -u mail_vmailadmin -p -e "SELECT username, active FROM mailbox WHERE username='[email protected]';" mail_vmailIs the mail content actually synced to the node currently receiving traffic? Check Syncthing's folder status via its API or GUI (Step 6); a folder stuck in an error state, or simply paused, means the currently-active node may be missing recent content the other node has.
Is DNS actually resolving to the node you expect right now?
dig +short mail.yourdomain.com @1.1.1.1Working through these in order, from "did the packet even arrive" outward to "did the application logic accept it", finds the actual break far faster than jumping straight to the layer that seems most likely.