# Backup your Proxmox hosts with ProxSave

[TOC]

## Step 1 - Configure NFS/SMB share to save your backups

## Step 2 - Configure a GCP bucket for remote Proxmox backups

## Step 3 - Configure emails in Proxmox (pmf)

- ProxSave can send emails via the author's relay SMTP server. You can use it for staging. For production, it is best to set up your own. However, the Go binary does not support MUA (like `mutt`), so it is best to switch it to Proxmox's own `pmf` method.
- Ensure you have an email address recipient in Proxmox:
    - Go to Datacenter → Permissions → Users
    - Find your user and ensure that there is an email address defined.

<p align="center"><a href="assets/1-step-3-configure-emails-in.png" target="_blank"><img src="assets/1-step-3-configure-emails-in.png" alt="1 step 3 configure emails in" width="300" /></a></p>

- Here is how to set up SMTP in Proxmox UI:
    - Go to Datacenter → Notifications → Notifications
    - Add → SMTP
    - Endpoint name: Gmail (use short names, no spaces and special chars)
    - Enable: Tick
    - Server: smtp.gmail.com
    - Authenticate: Tick
    - Encryption: TLS
    - Recipient: root@pam (or whatever user in Proxmox you use)

<p align="center"><a href="assets/2-step-3-configure-emails-in.png" target="_blank"><img src="assets/2-step-3-configure-emails-in.png" alt="2 step 3 configure emails in" width="300" /></a></p>

- While in the same section under Notification Matchers, make sure that the default matcher is on.
    - Open it, select the 'Targets to notify' option and tick the new notification created above.

<p align="center"><a href="assets/3-step-3-configure-emails-in.png" target="_blank"><img src="assets/3-step-3-configure-emails-in.png" alt="3 step 3 configure emails in" width="300" /></a></p>

> 💡 **Note**
>
> Add other Proxmox hosts to your SSH config to account for custom ports. The proxsave tool will want to connect to at least the other hosts in the cluster.
> ```sql
> sudo nano /root/.ssh/config
> Host proxmox1 192.168.8.3
> 		Port 2222
> Host proxmox2 192.168.8.4
>     Port 2222
> Host proxmox3 192.168.6.3
>     Port 2222
> chmod 600 /root/.ssh/config
> ```

## Install ProxSave

> 📌 **Note**
>
> The backup config for ProxSave is saved on Gitea.
> - For a new install, follow the above instructions and then restore the config below.
> - For updates (including variable migrations during major changes):
> ```sql
> bash -c "$(curl -fsSL https://raw.githubusercontent.com/tis24dev/proxsave/main/install.sh)"
> ```
> - To restore the config, run the following:
> ```bash
> # Upgrade/update
> # Download it from Gitea, it will auto-populate hostname:
> git clone https://gitea.bachelor-tech.com/jan/proxsave-config.git /opt/proxsave/proxsave-config-backup
> cd /opt/proxsave/proxsave-config-backup
> bash /opt/proxsave/proxsave-config-backup/deploy.sh
> ```
> - Check crontab, send output to null in order to not receive two emails per job.
> ```bash
> crontab -e
> 00 02 * * * /usr/local/bin/proxsave >/dev/null 2>&1
> ```

- SSH into your Proxmox host and read the recommendations on the project's GitHub page before proceeding:

🔗 [Link](https://github.com/tis24dev/proxsave)

```yaml
bash -c "$(curl -fsSL https://raw.githubusercontent.com/tis24dev/proxsave/main/install.sh)"
```

- Tweaking the config post-installation:

```sql
nano /opt/proxsave/configs/backup.env


# Disk space - set min. on secondary storage to be lower than the default 90:
MIN_DISK_SPACE_SECONDARY_GB=5

# Find suspicious ports and if you use port 2222 for SSH, then remove it:
SUSPICIOUS_PORTS="6666 6665 1337 31337 4444 5555 4242 6324 8888 3389 5900"

# Replace the original with these to lower CPU usage:
COMPRESSION_TYPE=zstd
COMPRESSION_LEVEL=12
COMPRESSION_THREADS=0
COMPRESSION_MODE=standard

# For your secondary storage, if you have mounted an NFS share (such as a NAS):
SECONDARY_ENABLED=true
SECONDARY_PATH=/mnt/nas-backup/proxmox1
SECONDARY_LOG_PATH=/mnt/nas-backup/proxmox1/log

# Email Notifications
EMAIL_ENABLED=true
EMAIL_DELIVERY_METHOD=pmf
EMAIL_FALLBACK_SENDMAIL=true
EMAIL_RECIPIENT=your-email@example.com
EMAIL_FROM=your-email@example.com

# To disable ZFS warnings if you do not use ZFS:
BACKUP_ZFS_CONFIG=false

```

- If you are using the new installer, you will see the following:

<p align="center"><a href="assets/4-install-proxsave.png" target="_blank"><img src="assets/4-install-proxsave.png" alt="4 install proxsave" width="300" /></a></p>

- The issue is that many options are missing there and until it is improved, it is best to (a) proceed with the installation and then (b) migrate it to the legacy .env file that can be edited manually.
- To do so, proceed with the installer with or without filling in details. To edit the config later on, remember that the path is `/opt/proxmox-backup/configs/backup.env`.

## Set up Backup Location 1 - local NAS file storage

- For NFS storage to work on a proxmox host, we need to ensure that it has the required packages:

```yaml
apt install nfs-common cifs-utils -y
```

- Mount your storage:

```yaml
mkdir -p /mnt/nas-backup

# Option 1: As NFS - Linux to Linux (recommended)
mount -t nfs 192.168.8.14:/home/jan/Backups/Proxmox /mnt/nas-backup

# Option 2: As SMB
mount -t cifs -o username=jan,password=YOUR_PASSWORD //192.168.8.14/home/jan/Backups/Proxmox /mnt/nas-backup

# Unmount it
umount /mnt/nas-backup
```

- Make it permanent:

```yaml
nano /etc/fstab

# NFS option (do not hang if it not reachable on boot):
192.168.8.14:/home/jan/Backups/Proxmox /mnt/nas-backup nfs hard,intr,timeo=50,retrans=3,_netdev,nofail,x-systemd.automount,x-systemd.mount-timeout=15 0 0

# NFS option (auto-mount of first touch, not on first boot - useful for VPN tunnel 
# connections that get established later during uptime 
192.168.8.14:/home/jan/Backups/Proxmox /mnt/nas-backup nfs noauto,hard,intr,timeo=50,retrans=3,_netdev,nofail,x-systemd.automount,x-systemd.mount-timeout=15 0 0

# SMB option
//192.168.8.14/home/jan/Backups/Proxmox /mnt/nas-backup cifs username=jan,password=YOUR_PASS,_netdev,nofail,iocharset=utf8 0 0
```

- In case you used the soft auto-load on access option:

```yaml
# Confirm that nothing is mounting on access at this point:
ls /mnt/nas-backup

# Reload systemd
systemctl daemon-reload

# Restart the Remote-FS target service (to activate the automount listener):
systemctl restart remote-fs.target

# Verify that the mounting works
ls /mnt/nas-backup
```

- For statically mapped shares:

```yaml
# Verify that the mounting works
mount -a
ls /mnt/nas-backup
```

So now we have our local NFS/SMB location ready and can move on to the remote one.

## Set up Backup Location 2 - A GCP bucket

- To mount a remote bucket storage on GCP, please [follow this guide](https://bachelor-tech.com/tutorial/bucket-storage-for-proxmox-vm-backups/) first to get the rclone config set up.
- Note: Modify your rclone.config for bucket level policies:

```yaml
nano /root/.config/rclone/rclone.conf

# Add this line:
bucket_policy_only = true
```

- Once the config file is set up (legacy or new), proceed with a dry run:

```yaml
# Trigger a dry-run:
proxmox-backup --dry-run

(Full path: /opt/proxmox-backup/script/proxmox-backup.sh --dry-run )

proxmox-backup

# Ensure that the backups are on both storage locations and that there are no errors!

# If you get an error related to the lack of existence of corosync.config while 
# your host is not running in a cluster, then just create an empty file to prevent
# the script from errorring out:
touch /etc/pve/corosync.conf

# Create a cron job
crontab -e

# Append this line at the end:
0 3 * * 0 /usr/local/bin/proxsave >/dev/null 2>&1

```

## Note - how to install NFS

```yaml
# Install it on the NAS or backup server
sudo apt install nfs-kernel-server -y

# Take into account which user you want the ownership of the files to go to.
# You can use your current non-root user, just check out the ID - let's say it is a 1000
id jan

sudo nano /etc/exports

# Provide access to all the 192.168.x.x subnets and force user 1000 (jan)
/home/jan/Backups/Proxmox 192.168.0.0/16(rw,sync,no_subtree_check,all_squash,anonuid=1000,anongid=1000)

# Apply the changes
sudo exportfs -ar

# Restart the NFS service:
sudo systemctl restart nfs-kernel-server

```

- If you use this NFS share also as a Samba (SMB) share, it may be worth installing an ACL tool that will override the settings of either service:

```yaml
# Install the tool while still on the NFS server (NAS, backup server..)
sudo apt update && sudo apt install acl -y

# Change the user as you need
sudo setfacl -R -m "default:user:jan:rwx" /home/jan/Backups/Proxmox
sudo setfacl -R -m "default:group:jan:rwx" /home/jan/Backups/Proxmox

# You can always verify the permissions for the given folder
getfacl /home/jan/Backups/Proxmox
```

- From another server on the LAN, check that the mount is available:

```yaml
showmount -e 192.168.8.14

# Mount it (NFS)
mount -t nfs 192.168.8.14:/home/jan/Backups/Proxmox /mnt/nas-backup

# Try a write operation and remove the file:
touch /mnt/nas-backup/testfile
rm /mnt/nas-backup/testfile
```

> 💡 **Note**
>
> If you try to move a file to the NAS, you will receive an error after copying, as the destination server will not allow permissions changing from the source due to the 'setfacl' that we did earlier.

- Save it in fstab without hanging on startup if not available:

```sql
sudo nano /etc/fstab
192.168.8.14:/home/jan/Backups/Proxmox /mnt/nas-backup nfs hard,intr,timeo=50,retrans=3,_netdev,nofail,x-systemd.automount,x-systemd.mount-timeout=15 0 0
```

- The mounting command in `fstab` explained:

| Option | Why |
| --- | --- |
| `nofail` | Systemd won't block boot if this mount fails |
| `x-systemd.automount` | Mount is lazy, nothing happens at boot at all, it only mounts on first access. This is your real boot-hang fix |
| `x-systemd.mount-timeout=15` | If it *is* accessed and the NAS is unreachable, give up after 15s instead of hanging |
| `_netdev` | Tells systemd this needs network, orders it correctly |
| `hard,intr,timeo=50` | Deliberate choice - keep trying to connect during outage (when already connected) |

- Remember to restart the daemon after `fstab` changes:

```bash
systemctl daemon-reload
```

## Update proxmox-backup (major version changes)

Simply run the installer script, as per `cat /opt/proxmox-backup/README.md | grep -i -A5 "update\|upgrade"`

```sql
cd /opt/proxmox-backup
bash -c "$(curl -fsSL https://raw.githubusercontent.com/tis24dev/proxmox-backup/main/install.sh)"
```

## Alternative approach: creating an rclone alias

- Once done, map out the folder where you would like to add the config to and let's map it as an alias, since proxmox-backup, for some hilarious reason, does not support specifying the destination folder.
- Note: rclone does not support creating empty folders, so if you want to create a folder for your proxmox-backup, then create it in a different way, such as via the GCP console.

```yaml
rclone config

- Type n for New remote.
- Name: gcp-alias
- Storage: Type alias
- Target: gcp:/proxmox-backup-bachelor/proxmox-backups/proxmox2
- Save and exit
```

- Then test it again:

```yaml
rclone ls gcp-alias
```