I’m thinking of expanding my homelab to support running some paid SaaS projects out of my house, and so I need to start thinking about uptime guarantees.
I want to set up a cluster where every service lives on at least two machines, so that no single machine dying can take a service down. The problem is the reverse proxy: the router still has to point port 443 at a single fixed IP address running Caddy, and that machine will always be a single point of failure. How would I go about running two or more reverse proxy servers with failover?
I’m guessing the answer has something to do with the router, and possibly getting a more advanced router or running an actual OS on the router that can handle failover. But at that point the router is a single point of failure! And yes, that’s unavoidable… but I’m reasonably confident that the unmodified commodity router I’ve used for years is unlikely to spontaneously die, whereas I’ve had very bad luck with cheap fanless and single-board computers, so anything I buy to use as an advanced router is just a new SPOF and I might as well have used it for the reverse proxy.
OPNsense and HAproxy might be a place to start, they work well together. You can define a backend pool of servers for roundrobinning, and if you buy a block of IPs you can roundrobin the incoming requests as well. I run OPNsense as a VM so that I can use Proxmox’s high availability service for the router and it’ll failover or manually livemigrate if I’m doing maintenance. You can VLAN the servers off from the rest of the network as well with OPNsense, and set up VPNs there for clients if needed, or use the SDN functions in the hypervisor to segregate servers if you’re running them on the hypervisor.