As explained in the previous step, the SYNC interface is optional but highly recommended to not lose connectivity during broadcast storms and other traffic that may prevent communication between the two or more nodes. Once this direct connection is established, we can run pfSync (and nothing else) on it. With pfSync, the states will be synced in real time, so that in case of a fail-over (using CARP), traffic will remain preserved. This is why we are setting up the SYNC interface and then pfSync first before CARP.
Configure the SYNC interface
- Start by connecting the OPNSense units together via a dedicated ethernet cable.
- Create a new interface on each OPNSense under ‘Interfaces’ -> ‘Assignments’. Choose the correct MAC address of the interface.
- Under ‘Interfaces’ → ‘SYNC’ (or whatever name you chose for your interface, assign a static IP on each OPNSense unit:
- You might ask why /29 ? This is because /30 is very restrictive, it allows for only 2 usable hosts. What if you want to add another device in the future as another backup? So the next best option is /29, which allows for 6 usable hosts.
- Set up the following required firewall rules – while they are connected to each, the default firewall rule is to deny all. So even for them to be able to ping each other, we will need to set up the required firewall rules.
Set Up Firewall Rules
- On each OPNSense unit, go to ‘Firewall’ → ‘Rules’ → ‘SYNC’ and create a new rule.
- Rule no.1: Allow ICMP on the firewall to test for pings:
- Action: Pass
- Interface: SYNC
- Direction: in
- TCP/IP: IPv4
- Protocol: ICMP
- Source: Any
- Destination: Any (or just SYNC net if you do not need to ping them from your LAN).
- Destination ports: Any
- Rule no.2: pfSync – to sync firewall states with each other:
- Action: Pass
- Interface: SYNC
- Direction: in
- TCP/IP: IPv4
- Protocol: PFSYNC
- Source: SYNC net
- Destination: This Firewall
- Destination ports: any
- Rule no.3: XMLRPC (WebGUI access to copy configs from main to backup)
- Action: Pass
- Interface: SYNC
- Direction: in
- TCP/IP: IPv4
- Protocol: TCP
- Source: SYNC net
- Destination: This Firewall
- Destination port range: from 443 to 443
- Important note: If your OPNsense web admin portal is using an alternative port because you use port 443 for other services such as HAProxy, then check the steps below under ‘Troubleshooting’ to remedy that first before proceeding.
- To verify that the instances can reach one another, if you can, connect to each OPNSense unit via SSH / console, choose option 7 and run the following ICPM (ping) tests:
- E.g. ‘ping 20.0.0.1’ from OPN2
- E.g. ‘ping 20.0.0.2’ from OPN1
Troubleshooting
- On virtualized OPNSense units, certain hypervisors like Proxmox apply firewall by default. Select the interface and ensure that the firewall is removed.
- On virtualized OPNSense units, in case pinging does not work and you get an error called ‘ping: sendto: Host is down’ , you will need to temporarily remove the network card on the hypervisor under the VM’s hardware settings (save its MAC address) and re-attach a new one (with the same MAC address). Then remember to re-assign it under ‘Interfaces’ → ‘Assignments’.
- No pings getting through? Make sure that the instances are interconnected via an ethernet (or another type of) cable. Double/triple check the firewall rules, inc. direction (in), protocol (TCP), source (SYNC net unless using LAN without a dedicated interface) and destination (This Firewall, not SYNC net or SYNC address!).
Let’s configure pfSync and XML-RPC to sync config and states before diving into CARP.