overriding static routes in FreeBSD
Hello group, I'm running BIRD on FreeBSD and I came across a problem with overriding static routes created outside of BIRD. The situation is that for installation of the operating system (contacting repositories, Puppet and some other services) system needs to have a static default gateway but once the system is fully installed it runs BIRD and talks BGP to core routers. Default routes are created with the usual FreeBSD method: [9:08:14] hwlb-awtest-01 ~/ # cat /etc/rc.conf.d/routing defaultrouter="1.2.3.4" ipv6_defaultrouter="a::f" Routes are created with UGS flags: [9:11:17] hwlb-awtest-01 ~/ # netstat -rn | grep default default 1.2.3.4 UGS public default a::f UGS public Once BIRD is started the log gets full of those messages: Jun 15 08:57:32 hwlb-awtest-01.ndco.ig.local bird[2216] KRT: Error sending route 0.0.0.0/0 to kernel: File exists Jun 15 08:57:50 hwlb-awtest-01.ndco.ig.local bird6[2213] KRT: Error sending route ::/0 to kernel: File exists I've looked at BIRD's source and I understand that BIRD replaces routes using krt_replace_rte and krt_send_route, and that all routes managed by BIRD have RTF_UP | RTF_PROTO1 flags. Those are not the flags created by `route add` on FreeBSD by default. However if I make default routes with those flags via `route add default 1.2.3.4 -proto1 -nostatic`, BIRD will happily replace them. Kernel protocol is configured with "learn" option. Is this a bug? Or should I just adjust the static routes to be BIRD-compatible? -- | pozdrawiam / greetings | Powered by macOS, Debian and FreeBSD | | Kajetan Staszkiewicz | www: http://vegeta.tuxpowered.net | `------------------------^--------------------------------------'
Hi, I think it is intentional so that bird should not meddle with "foreign" routes. In Linux, for example, it uses it's own protocol label and does not touch other routes. It can import them, but does not change them. In your situation, I think the optimal solution is to have your original default routes with some high metric, so that bird's default should override it later. In that case it will not need to replace routes it does not own, because usually you can have routes with the same prefix but different metrics simultaneousely in your routing table. On Tue, Jun 15, 2021 at 11:41 AM Kajetan Staszkiewicz <vegeta@tuxpowered.net> wrote:
Hello group,
I'm running BIRD on FreeBSD and I came across a problem with overriding static routes created outside of BIRD. The situation is that for installation of the operating system (contacting repositories, Puppet and some other services) system needs to have a static default gateway but once the system is fully installed it runs BIRD and talks BGP to core routers.
Default routes are created with the usual FreeBSD method: [9:08:14] hwlb-awtest-01 ~/ # cat /etc/rc.conf.d/routing defaultrouter="1.2.3.4" ipv6_defaultrouter="a::f"
Routes are created with UGS flags: [9:11:17] hwlb-awtest-01 ~/ # netstat -rn | grep default default 1.2.3.4 UGS public default a::f UGS public
Once BIRD is started the log gets full of those messages: Jun 15 08:57:32 hwlb-awtest-01.ndco.ig.local bird[2216] KRT: Error sending route 0.0.0.0/0 to kernel: File exists Jun 15 08:57:50 hwlb-awtest-01.ndco.ig.local bird6[2213] KRT: Error sending route ::/0 to kernel: File exists
I've looked at BIRD's source and I understand that BIRD replaces routes using krt_replace_rte and krt_send_route, and that all routes managed by BIRD have RTF_UP | RTF_PROTO1 flags. Those are not the flags created by `route add` on FreeBSD by default. However if I make default routes with those flags via `route add default 1.2.3.4 -proto1 -nostatic`, BIRD will happily replace them.
Kernel protocol is configured with "learn" option.
Is this a bug? Or should I just adjust the static routes to be BIRD-compatible?
-- | pozdrawiam / greetings | Powered by macOS, Debian and FreeBSD | | Kajetan Staszkiewicz | www: http://vegeta.tuxpowered.net | `------------------------^--------------------------------------'
On Tue, Jun 15, 2021 at 02:50:09PM +0200, Alexander Zubkov wrote:
Hi,
I think it is intentional so that bird should not meddle with "foreign" routes. In Linux, for example, it uses it's own protocol label and does not touch other routes. It can import them, but does not change them.
Hi Yes, that is true.
In your situation, I think the optimal solution is to have your original default routes with some high metric, so that bird's default should override it later. In that case it will not need to replace routes it does not own, because usually you can have routes with the same prefix but different metrics simultaneousely in your routing table.
That works in Linux, but i do not think it works in FreeBSD. AFAIK there is only one route per network there. Using -proto1 -nostatic for route is an easy way how to workaround this issue on BSD. -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
Oh, looks like FreeBSD does not have metric for the routes. So yes, it is not the possibility then. On Tue, Jun 15, 2021 at 3:24 PM Ondrej Zajicek <santiago@crfreenet.org> wrote:
On Tue, Jun 15, 2021 at 02:50:09PM +0200, Alexander Zubkov wrote:
Hi,
I think it is intentional so that bird should not meddle with "foreign" routes. In Linux, for example, it uses it's own protocol label and does not touch other routes. It can import them, but does not change them.
Hi
Yes, that is true.
In your situation, I think the optimal solution is to have your original default routes with some high metric, so that bird's default should override it later. In that case it will not need to replace routes it does not own, because usually you can have routes with the same prefix but different metrics simultaneousely in your routing table.
That works in Linux, but i do not think it works in FreeBSD. AFAIK there is only one route per network there.
Using -proto1 -nostatic for route is an easy way how to workaround this issue on BSD.
-- Elen sila lumenn' omentielvo
Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
W dniu 15.06.2021 o 16:04, Alexander Zubkov pisze:
Oh, looks like FreeBSD does not have metric for the routes. So yes, it is not the possibility then.
Probably it can be done. FreeBSD routing stack was significantly changed[1] during last year. It gained support for ECMP[2] via nexthop groups[3] and all routes have weigths[2] now. Likely only FRR received some updates to support this new routing stack.
On Tue, Jun 15, 2021 at 3:24 PM Ondrej Zajicek <santiago@crfreenet.org> wrote:
On Tue, Jun 15, 2021 at 02:50:09PM +0200, Alexander Zubkov wrote:
Hi,
I think it is intentional so that bird should not meddle with "foreign" routes. In Linux, for example, it uses it's own protocol label and does not touch other routes. It can import them, but does not change them.
Hi
Yes, that is true.
In your situation, I think the optimal solution is to have your original default routes with some high metric, so that bird's default should override it later. In that case it will not need to replace routes it does not own, because usually you can have routes with the same prefix but different metrics simultaneousely in your routing table.
That works in Linux, but i do not think it works in FreeBSD. AFAIK there is only one route per network there.
Using -proto1 -nostatic for route is an easy way how to workaround this issue on BSD.
-- Elen sila lumenn' omentielvo
Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
[1] https://reviews.freebsd.org/D24141 [2] https://reviews.freebsd.org/D26449 [3] https://reviews.freebsd.org/D24232 -- Marek Zarychta
participants (4)
-
Alexander Zubkov -
Kajetan Staszkiewicz -
Marek Zarychta -
Ondrej Zajicek