[RFC] Replace WireGuard AllowedIPs with IP route attribute
Hi wireguard, birds, and babelers, tl;dr I want to add a new Linux route attribute (think "via $wgpeer") to supplement wireguard's internal AllowedIPs logic for both routing and source address filtering. I've been pondering how to better integrate wireguard into dynamic routing daemons, particularly BIRD and babeld. Essentially we want to be able to dynamically add/remove AllowedIPs depending on current reachability and/or link quality stats. Looking at the wg netlink API I see two major efficiency/scalability problems: 1) there is no way to be notified of changes in AllowedIPs made by other processes meaning we have to do periodic scans and 2) a peer's AllowedIPs set can only be replaced wholesale, not modified incrementally. This is problematic as "someone" might, in the worst case, want to install an entire internet routing table's worth of AllowedIPs and the set will likely change frequently. FYI: The IPv4 table has ~1M entries at present, yikes. Assuming external AllowedIPs changes are infrequent occationally dumping them all to keep a consistent view of the state shouldn't be too much of an issue as long as the netlink interface is performant enoug, so I'm going to concentrate on the add/remove API for now. Instead of doing the obvious thing and adding a more efficient incremental AllowedIPs netlink interface I figure why not just add a route attribute to select a target wg peer on a device. That way we could not only save memory (no separate AllowedIPs trie) but also simplify routing daemon implementation considerably. This would mirror how on ethernet you can have `dev eth0 via $router_ip`. I'm still reviewing the net/ code to find the best way to do this, but I'm thinking either a new RTA_WGPEER, like: `default dev wg0 via-wgpeer $peer_pubkey` or perhaps re-using RTA_VIA and keying off a statically configured AllowedIP addresses. To start I'd make this an opt-in replacement for our usual AllowedIPs logic, making sure to only activate it if any via* RTAs are active on a particular device, but if it proves to work well I don't see why we couldn't adapt the netlink code to maintain AllowedIPs using this RTA (but invisible to userspace) to re-use the same code and get rid of allowedips.c altogether. That's assuming this ends up being less code overall or perhaps more performant. Happy to hear your thoughts, --Daniel
Hi Daniel, Chances are high I do miss something, but I've just set AllowedIPs to 0.0.0.0/0 and ::/0 and just used the routing protocol of my choice and filters to select which routes got exported and imported... :shrug: Best, Bernd On 19.08.23 16:02, Daniel Gröber wrote:
Hi wireguard, birds, and babelers,
tl;dr I want to add a new Linux route attribute (think "via $wgpeer") to supplement wireguard's internal AllowedIPs logic for both routing and source address filtering.
I've been pondering how to better integrate wireguard into dynamic routing daemons, particularly BIRD and babeld. Essentially we want to be able to dynamically add/remove AllowedIPs depending on current reachability and/or link quality stats.
Looking at the wg netlink API I see two major efficiency/scalability problems: 1) there is no way to be notified of changes in AllowedIPs made by other processes meaning we have to do periodic scans and 2) a peer's AllowedIPs set can only be replaced wholesale, not modified incrementally. This is problematic as "someone" might, in the worst case, want to install an entire internet routing table's worth of AllowedIPs and the set will likely change frequently. FYI: The IPv4 table has ~1M entries at present, yikes.
Assuming external AllowedIPs changes are infrequent occationally dumping them all to keep a consistent view of the state shouldn't be too much of an issue as long as the netlink interface is performant enoug, so I'm going to concentrate on the add/remove API for now.
Instead of doing the obvious thing and adding a more efficient incremental AllowedIPs netlink interface I figure why not just add a route attribute to select a target wg peer on a device. That way we could not only save memory (no separate AllowedIPs trie) but also simplify routing daemon implementation considerably.
This would mirror how on ethernet you can have `dev eth0 via $router_ip`. I'm still reviewing the net/ code to find the best way to do this, but I'm thinking either a new RTA_WGPEER, like: `default dev wg0 via-wgpeer $peer_pubkey` or perhaps re-using RTA_VIA and keying off a statically configured AllowedIP addresses.
To start I'd make this an opt-in replacement for our usual AllowedIPs logic, making sure to only activate it if any via* RTAs are active on a particular device, but if it proves to work well I don't see why we couldn't adapt the netlink code to maintain AllowedIPs using this RTA (but invisible to userspace) to re-use the same code and get rid of allowedips.c altogether. That's assuming this ends up being less code overall or perhaps more performant.
Happy to hear your thoughts, --Daniel
Hi Bernd, On Sat, Aug 19, 2023 at 07:50:38PM +0200, Bernd Naumann wrote:
Chances are high I do miss something, but I've just set AllowedIPs to 0.0.0.0/0 and ::/0 and just used the routing protocol of my choice and filters to select which routes got exported and imported... :shrug:
Right, let me expand a bit. You are absolutely right, right now if you want to use wg with dynamic routing daemons you essentially have to have one wg tunnel per remote node with AllowedIPs=::/0 and that works just fine at small scales. The idea here is that we would like to go back to having just one tunnel for all nodes involved in this particular network instead, due to general operations scalability, mine is a mesh network so the number of tunnels gets rather large quickly :) Lots of tunnels suck for various reasons, monitoring if they're all up and configured properly is one example but my understanding from previous discussions is the performance is probably not ideal either. --Daniel
participants (2)
-
Bernd Naumann -
Daniel Gröber