Skip to content

Commit

Permalink
ffmuc-mesh-vpn-wireguard-vxlan: Support wgkex loadbalancing mechanism
Browse files Browse the repository at this point in the history
  • Loading branch information
@DasSkelett
DasSkelett committed Mar 10, 2024
1 parent 5ac2ac9 commit fd30674
Show file tree
Hide file tree
Showing 4 changed files with 134 additions and 63 deletions.
4 changes: 2 additions & 2 deletions ffmuc-mesh-vpn-wireguard-vxlan/Makefile
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
include $(TOPDIR)/rules.mk

PKG_NAME:=ffmuc-mesh-vpn-wireguard-vxlan
PKG_VERSION:=1
PKG_VERSION:=2
PKG_RELEASE:=1

PKG_MAINTAINER:=Annika Wickert <aw@awlnx.space>
Expand All @@ -11,7 +11,7 @@ include $(TOPDIR)/../package/gluon.mk

define Package/ffmuc-mesh-vpn-wireguard-vxlan
TITLE:=Support for connecting meshes via wireguard
DEPENDS:=+gluon-mesh-vpn-core +micrond +kmod-wireguard +wireguard-tools +ip-full
DEPENDS:=+gluon-mesh-vpn-core +micrond +kmod-wireguard +wireguard-tools +ip-full +lua-jsonc
endef

$(eval $(call BuildPackageGluon,ffmuc-mesh-vpn-wireguard-vxlan))
73 changes: 39 additions & 34 deletions ffmuc-mesh-vpn-wireguard-vxlan/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,53 +1,58 @@
# ffmuc-mesh-vpn-wireguard-vxlan

You can use this package for connecting with wireguard to the Freifunk Munich network.
This package adds support for WireGuard+VXLAN as Mesh VPN protocol stack as it is used in the Freifunk Munich network.

### Dependencies

This relies on [wgkex](https://github.com/freifunkMUC/wgkex), the FFMUC WireGuard key exchange broker running on the configured broker address. The broker programms the gateway to accept the WireGuard key which is transmitted during connection.
Starting with the key exchange API v2, the wgkex broker also returns WireGuard peer data for a gateway selected by the broker, which this package then configures as mesh VPN peer/endpoint. This can be enabled by setting the `loadbalancing` option to `1`.

For the health-checks a webserver of some kind needs to listen to `HTTP GET` requests on the gateways.

### How it works

When `checkuplink` gets called (which happens every minute via cronjob), it checks if the gateway connection is still alive by calling `wget` and connecting to the WireGuard peer link address. If this address replies, we also start a `batctl ping` to the same address. If both checks succeed the connection just stays alive.

If one of the checks above bails out with an error the reconnect cycle is started. This means `checkuplink` registers itself with `wireguard.broker` by sending the WireGuard public key over either HTTP or HTTPS (depending on the device support).
The broker responds with JSON data containing the gateway peer data (pubkey, address, port, allowed IPs aka link address). `checkuplink` adds the peer to the wg interface using this data, and sets up the VXLAN interface with the peer link address as remote endpoint.

This script prefers to establish connections over IPv6 and falls back to IPv4 **only if there is no IPv6 default route**.

### Configuration

You should use something like the following in the site.conf:


```
mesh_vpn = {
mtu = 1400,
wireguard = {
enabled = '1',
iface = 'mesh-vpn',
iface = 'wg_mesh_vpn', -- not 'mesh-vpn', which is used for the VXLAN interface
limit = '1', -- actually unused
broker = 'broker.ffmuc.net/api/v1/wg/key/exchange',
peers = {
{
publickey ='N9uF5Gg1B5AqWrE9IuvDgzmQePhqhb8Em/HrRpAdnlY=',
endpoint ='ffkwsn01.freifunk-koenigswinter.de:30020',
link_address = 'fe80::f000:22ff:fe12:01',
},
{
publickey ='liatbdT62FbPiDPHKBqXVzrEo6hc5oO5tmEKDMhMTlU=',
endpoint ='ffkwsn02.freifunk-koenigswinter.de:30020',
link_address = 'fe80::f000:22ff:fe12:02',
},
{
publickey ='xakSGG39D1v90j3Z9eVWzojh6nDbnsVUc/RByVdcKB0=',
endpoint ='ffkwsn03.freifunk-koenigswinter.de:30020',
link_address = 'fe80::f000:22ff:fe12:07',
},
broker = 'broker.ffmuc.net', -- base path of broker, will be combined with API path
loadbalancing = '1' -- controls whether to use the loadbalancing/gateway assignment feature of the broker
peers = { -- only needed if 'loadbalancing = '0''
{
publickey ='N9uF5Gg1B5AqWrE9IuvDgzmQePhqhb8Em/HrRpAdnlY=',
endpoint ='ffkwsn01.freifunk-koenigswinter.de:30020',
link_address = 'fe80::f000:22ff:fe12:01',
},
{
publickey ='liatbdT62FbPiDPHKBqXVzrEo6hc5oO5tmEKDMhMTlU=',
endpoint ='ffkwsn02.freifunk-koenigswinter.de:30020',
link_address = 'fe80::f000:22ff:fe12:02',
},
{
publickey ='xakSGG39D1v90j3Z9eVWzojh6nDbnsVUc/RByVdcKB0=',
endpoint ='ffkwsn03.freifunk-koenigswinter.de:30020',
link_address = 'fe80::f000:22ff:fe12:07',
},
},
},
```
And you should include the package in the site.mk of course!

### Dependencies

This relies on [wgkex](https://github.com/freifunkMUC/wgkex) the FFMUC wireguard broker running on the configured broker address. The broker programms the gateway to accept the WireGuard key which is transmitted during connection.

For the health-checks a webserver of some kind needs to listen to `HTTP GET` requests on the gateways.

### How it works

When `checkuplink` gets called (which happens every minute via cronjob), it checks if the gateway connection is still alive by calling `wget` and connecting to `wireguard.peer.peer_[number].link_address`. If this address replies we also start a `batctl ping` to the same address. If both checks succeed the connection just stays alive.

If one of the checks above bails out with an error the reconnect cycle is started. Which means `checkuplink` registers itself with `wireguard.broker` by sending the WireGuard public_key over either http or https (depending on the device support). After the key was sent the script tries to randomely connect to one of the `wireguard.peer`. This script prefers to establish connections over IPv6 and falls back to IPv4 only if there is no IPv6 default route.

### Interesting Links

- [FFMUC: Half a year with WireGuard](https://www.slideshare.net/AnnikaWickert/ffmuc-half-a-year-with-wireguard)
Expand Down
108 changes: 81 additions & 27 deletions ffmuc-mesh-vpn-wireguard-vxlan/files/lib/gluon/gluon-mesh-wireguard-vxlan/checkuplink
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,22 +27,33 @@ clean_port() {
echo "$1" | sed -r 's/:[0-9]+$|\[|\]//g'
}

check_address_family() {
local peer_endpoint="$2"
local gateway
gateway="$(clean_port "$peer_endpoint")"
extract_port() {
echo "$1" | awk -F: '{print $NF}'
}

combine_ip_port() {
local ip="$1"
local port="$2"

if [[ ${ip} == *":"* ]]; then
ip="[${ip}]"
fi
echo "$ip:$port"
}

resolve_host() {
local gateway="$1"
# Check if we have a default route for v6 if not fallback to v4
if ip -6 route show table 1 | grep -q 'default via'
then
local ipv6
ipv6="$(gluon-wan nslookup "$gateway" | grep 'Address:\? [0-9]' | grep -E -o '([a-f0-9:]+:+)+[a-f0-9]+')"
echo "[$ipv6]$(echo "$peer_endpoint" | grep -E -oe ":[0-9]+$")"
ipv6="$(gluon-wan nslookup "$gateway" | grep 'Address:\? [0-9]' | grep -oE '([a-f0-9:]+:+)+[a-f0-9]+')"
echo "$ipv6"
else
local ipv4
ipv4="$(gluon-wan nslookup "$gateway" | grep 'Address:\? [0-9]' | grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b")"
echo "$ipv4$(echo "$peer_endpoint" | grep -E -oe ":[0-9]+$")"
ipv4="$(gluon-wan nslookup "$gateway" | grep 'Address:\? [0-9]' | grep -oE '\b([0-9]{1,3}\.){3}[0-9]{1,3}\b')"
echo "$ipv4"
fi

}

# Do we already have a private-key? If not generate one
Expand Down Expand Up @@ -93,17 +104,6 @@ if [ "$(uci get wireguard.mesh_vpn.enabled)" = "true" ] || [ "$(uci get wireguar
exit 3
fi

# Get the number of configured peers and randomly select one
NUMBER_OF_PEERS=$(uci -q show wireguard | grep -E -ce "peer_[0-9]+.endpoint")
PEER="$(awk -v min=1 -v max="$NUMBER_OF_PEERS" 'BEGIN{srand(); print int(min+rand()*(max-min+1))}')"
PEER_PUBLICKEY="$(uci get wireguard.peer_"$PEER".publickey)"

logger -t checkuplink "Selected peer $PEER"

endpoint="$(check_address_family "$PEER_PUBLICKEY" "$(uci get wireguard.peer_"$PEER".endpoint)")"

logger -t checkuplink "Connecting to $endpoint"

# Delete Interfaces
{
ip link set nomaster dev mesh-vpn >/dev/null 2>&1
Expand All @@ -114,7 +114,7 @@ if [ "$(uci get wireguard.mesh_vpn.enabled)" = "true" ] || [ "$(uci get wireguar
PUBLICKEY=$(uci get wireguard.mesh_vpn.privatekey | wg pubkey)
SEGMENT=$(uci get gluon.core.domain)

# Push public key to broker, test for https and use if supported
# Push public key to broker and receive gateway data, test for https and use if supported
ret=0
wget -q "https://[::1]" || ret=$?
# returns Network Failure =4 if https exists
Expand All @@ -124,7 +124,64 @@ if [ "$(uci get wireguard.mesh_vpn.enabled)" = "true" ] || [ "$(uci get wireguar
else
PROTO=https
fi
LD_PRELOAD=libpacketmark.so LIBPACKETMARK_MARK=1 gluon-wan wget -q -O- --post-data='{"domain": "'"$SEGMENT"'","public_key": "'"$PUBLICKEY"'"}' "$PROTO://$(uci get wireguard.mesh_vpn.broker)"

# Remove API path suffix if still present in config
WGKEX_BROKER_BASE_PATH="$(uci get wireguard.mesh_vpn.broker | sed 's|/api/v1/wg/key/exchange||')"

if [ "$(uci get wireguard.mesh_vpn.loadbalancing)" = "1" ]; then
# Use /api/v2, get gateway peer details from broker response
WGKEX_BROKER="$PROTO://$WGKEX_BROKER_BASE_PATH/api/v2/wg/key/exchange"
logger -t checkuplink "Contacting wgkex broker $WGKEX_BROKER"
if ! WGKEX_DATA=$(LD_PRELOAD=libpacketmark.so LIBPACKETMARK_MARK=1 gluon-wan wget -q -O- --post-data='{"domain": "'"$SEGMENT"'","public_key": "'"$PUBLICKEY"'"}' "$WGKEX_BROKER"); then
logger -p err -t checkuplink "Contacting wgkex broker failed, response: $WGKEX_DATA"
exit 1
fi

logger -p info -t checkuplink "Got data from wgkex broker: $WGKEX_DATA"

# Parse the returned JSON in a Lua script, returning the endpoint address, port, pubkey and first allowed IP, separated by newlines
if ! data=$(lua /lib/gluon/gluon-mesh-wireguard-vxlan/parse-wgkex-response.lua "$WGKEX_DATA"); then
logger -p err -t checkuplink "Parsing wgkex broker data failed"
exit 1
fi

logger -p debug -t checkuplink "Successfully parsed wgkex broker data"

PEER_ADDRESS="$(echo "$data" | sed -n 1p)"
PEER_PORT="$(echo "$data" | sed -n 2p)"
PEER_PUBLICKEY="$(echo "$data" | sed -n 3p)"
PEER_LINKADDRESS=$(echo "$data" | sed -n 4p)

PEER_ADDRESS="$(resolve_host "$PEER_ADDRESS")"
PEER_ENDPOINT="$(combine_ip_port "$PEER_ADDRESS" "$PEER_PORT")"

else
# Use /api/v1, get gateway peer details from config
WGKEX_BROKER="$PROTO://$WGKEX_BROKER_BASE_PATH/api/v1/wg/key/exchange"
logger -p info -t checkuplink "Contacting wgkex broker $WGKEX_BROKER"
if ! LD_PRELOAD=libpacketmark.so LIBPACKETMARK_MARK=1 gluon-wan wget -q -O- --post-data='{"domain": "'"$SEGMENT"'","public_key": "'"$PUBLICKEY"'"}' "$WGKEX_BROKER"; then
logger -p err -t checkuplink "Contacting wgkex broker failed"
exit 1
fi

# Get the number of configured peers and randomly select one
NUMBER_OF_PEERS=$(uci -q show wireguard | grep -E -ce "peer_[0-9]+.endpoint")
PEER="$(awk -v min=1 -v max="$NUMBER_OF_PEERS" 'BEGIN{srand(); print int(min+rand()*(max-min+1))}')"

logger -p info -t checkuplink "Selected peer $PEER"

PEER_HOSTPORT="$(uci get wireguard.peer_"$PEER".endpoint)"
PEER_HOST="$(clean_port "$PEER_HOSTPORT")"
PEER_ADDRESS="$(resolve_host "$PEER_HOST")"
PEER_PORT="$(extract_port "$PEER_HOSTPORT")"
PEER_ENDPOINT="$(combine_ip_port "$PEER_ADDRESS" "$PEER_PORT")"

PEER_PUBLICKEY="$(uci get wireguard.peer_"$PEER".publickey)"
PEER_LINKADDRESS="$(uci get wireguard.peer_"$PEER".link_address)"

fi

logger -p info -t checkuplink "Connecting to $PEER_ENDPOINT"

# Bring up the wireguard interface
ip link add dev "$MESH_VPN_IFACE" type wireguard
Expand All @@ -136,10 +193,7 @@ if [ "$(uci get wireguard.mesh_vpn.enabled)" = "true" ] || [ "$(uci get wireguar

# Add link-address and Peer
ip address add "${LINKLOCAL}"/64 dev "$MESH_VPN_IFACE"
if [ "$endpoint" = "" ]; then
endpoint=$(uci get wireguard.peer_"$PEER".endpoint)
fi
gluon-wan wg set "$MESH_VPN_IFACE" peer "$(uci get wireguard.peer_"$PEER".publickey)" persistent-keepalive 25 allowed-ips "$(uci get wireguard.peer_"$PEER".link_address)/128" endpoint "$endpoint"
gluon-wan wg set "$MESH_VPN_IFACE" peer "$PEER_PUBLICKEY" persistent-keepalive 25 allowed-ips "$PEER_LINKADDRESS/128" endpoint "$PEER_ENDPOINT"

# We need to allow incoming vxlan traffic on mesh iface
sleep 10
Expand All @@ -152,7 +206,7 @@ if [ "$(uci get wireguard.mesh_vpn.enabled)" = "true" ] || [ "$(uci get wireguar
fi

# Bring up VXLAN
if ! ip link add mesh-vpn type vxlan id "$(lua -e 'print(tonumber(require("gluon.util").domain_seed_bytes("gluon-mesh-vpn-vxlan", 3), 16))')" local "${LINKLOCAL}" remote "$(uci get wireguard.peer_"$PEER".link_address)" dstport 8472 dev "$MESH_VPN_IFACE"
if ! ip link add mesh-vpn type vxlan id "$(lua -e 'print(tonumber(require("gluon.util").domain_seed_bytes("gluon-mesh-vpn-vxlan", 3), 16))')" local "${LINKLOCAL}" remote "$PEER_LINKADDRESS" dstport 8472 dev "$MESH_VPN_IFACE"
then
logger -p err -t checkuplink "Unable to create mesh-vpn interface"
exit 2
Expand Down
12 changes: 12 additions & 0 deletions ...-vpn-wireguard-vxlan/luasrc/lib/gluon/gluon-mesh-wireguard-vxlan/parse-wgkex-response.lua
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
local json = require 'jsonc'

local input = assert(arg[1])
local data = assert(json.parse(input))
if not data.Endpoint or not data.Endpoint.Address or not data.Endpoint.Port
or not data.Endpoint.PublicKey or not data.Endpoint.AllowedIPs or not data.Endpoint.AllowedIPs[1] then
error("Malformed JSON response, missing required value")
end
print(data.Endpoint.Address)
print(data.Endpoint.Port)
print(data.Endpoint.PublicKey)
print(data.Endpoint.AllowedIPs[1])

0 comments on commit fd30674

Please sign in to comment.