In this article, I did not cover how to best harden your web and DB servers. It is assumed that it is already done.
Backup & Recovery
While this set up is fully HA in terms of the web and DB servers being available on two sites (with a Galera witness on a VPS outside), HA does not equal backups. In the event of a breach (like ransomware) or a cascading service failure, you will need a form of archived backups, ideally going back a few months, so that you can determine the nearest and still safest version of your data.
- The recommended approach is to follow the 3-2-1 principle, i.e. by having 3 total copies of your data (the original plus two backups), storing them on 2 different media types (e.g. SSDs and cloud-based), and keeping at least 1 copy in an off-site location to protect against data loss from hardware failure, cyberattacks, or natural disasters.
- In addition, it is highly advisable to have a copy stored not just off-site but also off-line from secure threats. Because consider what happens if you lose all your passwords and stored SSH keys! Proper care needs to be given, even if it is not done as often. In a homelab environment, an example could be copying an encrypted version of the exported Vaultwarden data onto a memory stick that is stored in your parents’ house twice a year.
- Components to back up:
| Component | Location | Priority | Notes |
|---|---|---|---|
| Vaultwarden data directory | /opt/vaultwarden/vw-data/ | Critical | Contains RSA keys, attachments, icons, config |
| MariaDB/Galera database | Vaultwarden database | Critical | Contains all vault entries, users, organizations |
| Docker Compose file | /opt/vaultwarden/docker-compose.yml | High | Contains your environment configuration |
| Nginx virtual host | /etc/nginx/conf.d/vaultwarden.conf | Medium | Can be recreated but saves time |
| SSL certificates | Managed by ACME/OPNsense | Low | Can be regenerated |
Check out my previous article on how to back up VMs and LXCs from Proxmox onto a GCP’s archive storage!
Recovery Situations
Below are three common scenarios that may occur:
- Scenario A: Single Web Node Failure If one web server fails but others are operational:
- Restore the VM from Proxmox backup (or rebuild from template)
- Syncthing will automatically sync
/opt/vaultwarden/vw-data/from healthy nodes - Verify RSA keys match:
md5sum /opt/vaultwarden/vw-data/rsa_key.*(compare across nodes) - Start the container:
cd /opt/vaultwarden && sudo docker compose up -d - Verify in HAProxy that the node rejoins the backend pool
- Scenario B: Database Corruption or Loss
- Stop Vaultwarden containers on ALL web nodes to prevent writes:
cd /opt/vaultwarden && sudo docker compose down - On your preferred (primary) Galera node, restore the database:
gunzip < /var/backups/vaultwarden/vaultwarden_YYYY-MM-DD_HHMMSS.sql.gz | mysql -u root -p vaultwarden - Verify Galera sync status:
mysql -u root -p -e "SHOW STATUS LIKE 'wsrep_cluster_size';" - Restart Vaultwarden containers on all nodes
- Stop Vaultwarden containers on ALL web nodes to prevent writes:
- Scenario C: Complete Site Loss
- If both sites are compromised (e.g., ransomware), do NOT connect backup media to infected systems.
- Rebuild infrastructure from clean Proxmox templates
- Restore database from off-site/offline backup
- Restore
/opt/vaultwarden/vw-data/from backup (this includes the RSA keys). - Update DNS if IP addresses changed – do not open a public DNS record but rather a local record (such as on OPNSense by utilizing the Unbound DNS service).
- Have all users verify their vaults and re-authenticate devices
Testing Your Backups
Backups are worthless if you cannot restore from them. Schedule quarterly restore tests:
# On a test VM (not production!), verify database backup integrity:* gunzip -t /var/backups/vaultwarden/vaultwarden_latest.sql.gz && echo "Archive OK" # Test actual restoration to a temporary database:* mysql -u root -p -e "CREATE DATABASE vaultwarden_test;" gunzip < /var/backups/vaultwarden/vaultwarden_latest.sql.gz | mysql -u root -p vaultwarden_test mysql -u root -p -e "SELECT COUNT(*) FROM vaultwarden_test.users;" mysql -u root -p -e "DROP DATABASE vaultwarden_test;"
Updating Vaultwarden
This part is simple, yet for completion, let’s cover it.
- It is always good to look into the release notes before updating in case some larger changes have been made that could break a dependency.
- Take a snapshot of your web node before you carry out any changes.
- Then proceed on each web node:
# Pull the latest image: sudo docker pull vaultwarden/server # Restart the docker container cd /opt/vaultwarden sudo docker compose down && sudo docker compose up --force-recreate # Once it loads and you can see the new version has loaded, press 'd' to detach it
Securing other components of your stack
Yet if you are looking for some tips, look at my previous article on how to deploy a DB Galera cluster that touch on how you can use tools like ufw firewall, fail2ban for securing your SSH access.
Putting Vaultwarden behind a VPN
So what else could we consider? You could consider NOT having a resolvable public DNS name for your Vaultwarden instance and instead, have it available only internally on your LAN. Then, if you have a VPN client deployed (such as a WireGuard plugin in your OPNSense instance), you could make it available only via that VPN connection. Security by obscurity is still a thing!
On your phone and other mobile devices, you can then install the VPN client to reach your Vaultwarden server. Alternatively, if you do not often update your passwords, simply let it sync whenever you are on your LAN – when outside, your mobile device will work off its local cache, preserving access to all the passwords up to the time of your last sync.
What else could go wrong?
While this guide is already rather comprehensive, there are quite a few other things that could go wrong (mostly related to the infrastructure) that could affect your Vaultwarden’s uptime. Examples include:
- Galera cluster split-brain scenarios – What happens if sites lose connectivity while both are accepting writes?
- Certificate renewal failures – ACME/Let’s Encrypt certificates expire; what if renewal fails?
- Syncthing out-of-sync states – How to identify and resolve when folders get stuck in ‘Syncing’ state
- HAProxy backend health check failures – How to diagnose when health checks fail but the service appears to be running
- WebSocket connection issues – Users report ‘offline’ status in apps despite the site being reachable
- Database migration version conflicts – What if nodes are running different Vaultwarden versions with different schema expectations?
Let me know in the comments below in case you have come across any or if you would like me to expand on how to recover from these before you encounter them yourself 😇
This concludes our guide. I hope you enjoyed it.