What does each service do?
- pfsync: This is used for synchronizing the state tables between two OPNsense units. It ensures that active network connections and their states (e.g., a web session, an ongoing download, or a VPN tunnel) are replicated from the active to the standby firewall. This means that during a failover event, established connections don’t drop, providing a seamless transition for users.
- It is for this service (not for CARP) that we use the dedicated SYNC line, if possible. While it is not mandatory, in case there is a broadcasting storm or other undesired traffic on the LAN, we may run into a situation where the main unit cannot be determined due to packet loss and face a split brain scenario.
- XML-RPC (Configuration Sync): uses HTTPS (TCP/443, not flexible to change). It syncs settings like firewall rules, aliases, NAT, users, etc. If you keep each unit synced manually then you do not need to use this service, but it is recommended, as otherwise you might run into a situation where you configure new firewall rules or settings on the ‘main’ unit and forget to do the same on the ‘backup unit(s).
Set up pfSync and XML-RCP services
- Enable pfsync for firewall state syncing. On the main unit (not backup!), go to ‘System’ → ‘High Availability’ → ‘Settings’ and choose the following:
- Leave the first two items unticked
- Synchronize all states via: SYNC
- Sync compatibility: 24.7 or above
- Synchronize Peer IP: the backup unit’s IP address
- Synchronize Config: the backup unit’s IP address (again)
- Remote System username: root (you can set up a specific user for this purpose if you prefer)
- Remote System password: <whatever password you set up>
- Services: choose all BUT ‘System tunables’
- Then click on the ‘Apply’ button, followed by the ‘Perform synchronization’ button.
- As soon as you click on the sync button, you should see something similar as on the screenshot below:
- Click on the ‘Synchronize config to backup’ button to push the settings from the main to the backup unit to keep them consistent.
- Perform the High Availability (pfsync only) set up on your backup unit.
- Just make sure to provide the correct IP address of the main unit (since now you are on the backup one), login details and do not fill in the IP address for the ‘Synchronize Config’ section, as we assume the main unit will always be the most up to date with config changes.
- It is recommended to tick the ‘Disable preempt’ box in case you would like this unit to go back to backup in the event it becomes main for some time during the main’s unit outage.
Troubleshooting
- Double/triple check your firewall rules for pfSync and XML-RPC, esp. the source and destination. No ports are to be specified for those services.
- Make sure that if you configured ICMP firewall rules on each OPNSense unit, that they can ping each other. If they cannot ping each other despite fw rules, then some other firewall is in place. For example, in a virtualized environment (such as Proxmox), ensure that firewall is not applied on that interface and that there is no IP set up on the SYNC or WAN interfaces, since OPNSense is handling that.
- If you run the WebGUI in a different port than 443, then the question is what else is using that port – for example, the HAProxy plugin for OPNSense.
- If unsure about which service is using the 443 interface on your OPSense unit (if not the WebGUI), you can run ‘
sockstat -l | grep :443’.
- If unsure about which service is using the 443 interface on your OPSense unit (if not the WebGUI), you can run ‘
- If you do run HAProxy on port 443 and the WebGUI on another, you will need to change it – rather configure HAProxy to run on another port (such as 8443 or 4443).
- You will then need to change your firewall rules that forward HTTPS traffic from each WAN interface to ‘This Firewall’ to use that port (instead of 443 to use the custom one, such as 4443 or 8443 or whatever you use).
- On your WAN router, change the port forwarding from external 443 to internal 8443 (or 4443, whatever you decide to use).
- Lastly, you can change the WebGUI port back to 443 – go to ‘Settings’ → ‘System’ → ‘Administration’ and change it there.
- Issues with XML-RPC? On either unit, run ‘cat /var/log/system/latest.log | grep xmlrpc’ to see what the logs indicate.
- If XML-RPC is still not working, try connecting to the Web interface of each OPNsense by using the SYNC IP of each. Just make sure that the SYNC interface on each OPNsense unit is also selected via ‘Settings’ – ‘System’ – ‘Administration’. If still not accessible, check that you have no NAT rule that could be interfering.
- Unsure if pfsync is running and in which direction? Run ‘pfctl -s state | grep 20.0.0’ from either OPSense unit:
