BGP for Route Distribution

[lachlan2k]

Problem

Currently, when running Wag in a cluster, NAT is really the only viable option to ensure traffic is returned to the right node.

NAT isn't an ideal solution a lot of the time, especially if you want visibility of traffic through a central firewall, or auditing access based on source IPs.

Additionally, BGP could be useful in the (much less common) scenario of 2 upstream routers/L3 switches to provide upstream routing redundancy or load balancing.

Proposed Solution

When running in a cluster, Wag nodes should optionally run BGP to allow operation without NAT.

• Each Wag node speaks BGP and advertises the addresses of its connected clients
• Each Wag node peers with the user's router. This allows the user's router to direct traffic to the correct node.
• Each Wag node peers with every other Wag node. This ensures if a packet arrives at Node B, but the client is connected to Node A, it gets directed between nodes. Just in case the user's router is slow at respecting BGP update messages, or its directed to the wrong node.

Additional

• The web UI could show the status of BGP (i.e. what routes each node is advertising, and any connected peers).
• This is neat https://github.com/osrg/gobgp

[NHAS]

Agreed that this would be a very cool thing to implement.

I'm currently working on stabilising the clustering version of wag, then giving some TLC to some issues that have cropped up while I've been doing this work.

So I will think of this in a year or two.

[davidcoles]

Hi. You should be able to use BIRD to achieve this. Have BIRD monitor your local interfaces and advertise the client's IP addresses into your network.

[NHAS]

Yep definitely, advertising routes is definitely not the main difficulty in doing this, it's more around how each cluster member might advertise what clients are currently attached to it, what to do with traffic that is being sent to nodes that dont have the client the traffic is destined for due to old bgp routes and Im sure like 100 different things.

I'd also probably use something written in native go, just so it'll integration nicer

[davidcoles]

I have a similar project - I get the daemon to add/remove the client IP address to the wireguard interface on a node when it sees the packet counters for associated key incrementing/stopping.

Then I just (with Ansible) form a fully meshed BGP overlay network (wireguard tunnels between each node) which routes traffic to/from the right node. Has been working well for over a year for me.

Incidentally, I run the cluster in AWS with a load balancer routing traffic to the nodes which form their own little autonomous system, then form some eBGP links with BIRD over wireguard from some "point of presence" nodes inside our network.