<div dir="ltr">Hi all!<div><br></div><div>I've got a small mesh deployment using BIRD 2.0.7 and OSPFv2 which has been working well for a while now distributing a mix of device and static routes going into the kernel table export-only.</div><div><br></div><div>I've now run into a need to have a number of "dynamic" static routes managed by an external system and was looking into using a 2nd import-only kernel routing table for that
managed via netlink
rather than constantly reloading BIRD - but I've run into an issue where things aren't behaving as I'd expect when multiple routers advertise the same route sourced from the kernel protocol. What I'm seeing is that when the route gets added on the first router, it gets exported via BIRD to the kernel table as directly connected (as specified) and on the other router as a next-hop of the first router (as you'd expect). But when the same route is added on the other router (while still on the first one), the kernel routing table of the 2nd router does not change to be direct, it keeps the next-hop of the 1st router. However, if I specify the exact same routes in the static protocol and reload BIRD, everything works correctly and both routers end up with a direct route in the kernel table! (and other routers in the area get multi-path routes via the ones that have it added.)</div><div><br></div><div>I think I've tracked the behaviour difference down to how they're being added / appearing in OSPF, but I'm not sure why they might be behaving differently (it's as if the static routes get merged with what is already in the table, but the pipe isn't doing that) - I've included the config and some command output:</div><div><br></div><div>BIRD config (the same apart from id/prefsrc on both I'm testing with, trimmed out of the "production" config which also includes PtMP neighbours on remote routers):</div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><font face="monospace">router id 172.19.4.2;</font></div><div><font face="monospace"><br></font></div><div><font face="monospace">debug protocols { states, routes, filters, interfaces, events };</font></div><div><font face="monospace"><br></font></div><div><font face="monospace">ipv4 table clustr;</font></div><div><font face="monospace"><br></font></div><div><font face="monospace">protocol device {</font></div><div><font face="monospace">}</font></div><div><font face="monospace"><br></font></div><div><font face="monospace">protocol direct {</font></div><div><font face="monospace"> ipv4;</font></div><div><font face="monospace"> interface "ens224";</font></div><div><font face="monospace">}</font></div><div><font face="monospace"><br></font></div><div><font face="monospace">protocol kernel {</font></div><div><font face="monospace"> ipv4 {</font></div><div><font face="monospace"> import none;</font></div><div><font face="monospace"> export filter {</font></div><div><font face="monospace"> krt_prefsrc = 172.19.4.2;</font></div><div><font face="monospace"> accept;</font></div><div><font face="monospace"> };</font></div><div><font face="monospace"> };</font></div><div><font face="monospace"> merge paths yes;</font></div><div><font face="monospace">}</font></div><div><font face="monospace"><br></font></div><div><font face="monospace">protocol kernel {</font></div><div><font face="monospace"> ipv4 {</font></div><div><font face="monospace"> table clustr;</font></div><div><font face="monospace"> import all;</font></div><div><font face="monospace"> export none;</font></div><div><font face="monospace"> };</font></div><div><font face="monospace"> scan time 2;</font></div><div><font face="monospace"> learn yes;</font></div><div><font face="monospace"> kernel table 100;</font></div><div><font face="monospace">}</font></div><div><font face="monospace"><br></font></div><div><font face="monospace">protocol pipe {</font></div><div><font face="monospace"> table master4;</font></div><div><font face="monospace"> peer table clustr;</font></div><div><font face="monospace"> import all;</font></div><div><font face="monospace"> export none;</font></div><div><font face="monospace">}</font></div><div><font face="monospace"><br></font></div><div><font face="monospace">protocol ospf v2 {</font></div><div><font face="monospace"> ipv4 {</font></div><div><font face="monospace"> import all;</font></div><div><font face="monospace"> export where dest != RTD_UNREACHABLE;</font></div><div><font face="monospace"> };</font></div><div><font face="monospace"> area 0 {</font></div><div><font face="monospace"> interface "ens224" {</font></div><div><font face="monospace"> bfd yes;</font></div><div><font face="monospace"> };</font></div><div><font face="monospace"> };</font></div><div><font face="monospace"> ecmp yes;</font></div><div><font face="monospace"> merge external yes;</font></div><div><font face="monospace">}</font></div><div><font face="monospace"><br></font></div><div><font face="monospace">protocol bfd {</font></div><div><font face="monospace"> interface "ens*" {</font></div><div><font face="monospace"> min rx interval 100 ms;</font></div><div><font face="monospace"> min tx interval 200 ms;</font></div><div><font face="monospace"> idle tx interval 1000 ms;</font></div><div><font face="monospace"> };</font></div><div><font face="monospace">}</font></div><div><font face="monospace"><br></font></div><div><font face="monospace">protocol static {</font></div><div><font face="monospace"> ipv4;</font></div><div><font face="monospace"> route <a href="http://172.27.0.0/19">172.27.0.0/19</a> via "ens224";</font></div><div><font face="monospace"> route <a href="http://10.0.0.0/8">10.0.0.0/8</a> unreachable;</font></div><div><font face="monospace"> route <a href="http://100.64.0.0/10">100.64.0.0/10</a> unreachable;</font></div><div><font face="monospace"> route <a href="http://169.254.0.0/16">169.254.0.0/16</a> unreachable;</font></div><div><font face="monospace"> route <a href="http://172.16.0.0/12">172.16.0.0/12</a> unreachable;</font></div><div><font face="monospace"> route <a href="http://192.168.0.0/16">192.168.0.0/16</a> unreachable;</font></div><div><font face="monospace">}</font></div></blockquote><div><div><div class="gmail_signature" data-smartmail="gmail_signature"><br>Initial OSPF state:</div></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div><div class="gmail_signature" data-smartmail="gmail_signature"><font face="monospace">[root@ukdev-bastion-003 ~]# birdc show ospf state all<br>BIRD 2.0.7 ready.<br><br>area 0.0.0.0<br><br> router 172.19.4.2<br> distance 0<br> network <a href="http://172.19.0.0/19">172.19.0.0/19</a> metric 10<br> stubnet <a href="http://172.19.2.2/32">172.19.2.2/32</a> metric 0<br> external <a href="http://172.19.2.2/32">172.19.2.2/32</a> metric2 10000<br> external <a href="http://172.19.0.0/19">172.19.0.0/19</a> metric2 10000<br> external <a href="http://172.27.0.0/19">172.27.0.0/19</a> metric2 10000<br><br> router 172.19.4.3<br> distance 10<br> network <a href="http://172.19.0.0/19">172.19.0.0/19</a> metric 10<br> stubnet <a href="http://172.19.2.3/32">172.19.2.3/32</a> metric 0<br> external <a href="http://172.19.2.3/32">172.19.2.3/32</a> metric2 10000<br> external <a href="http://172.19.0.0/19">172.19.0.0/19</a> metric2 10000<br> external <a href="http://172.27.0.0/19">172.27.0.0/19</a> metric2 10000<br><br> network <a href="http://172.19.0.0/19">172.19.0.0/19</a><br> dr 172.19.4.2<br> distance 10<br> router 172.19.4.2<br> router 172.19.4.3</font><br></div></div></div></blockquote><div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><br></div><div class="gmail_signature" data-smartmail="gmail_signature">Adding a test route on the first router:</div></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div><div class="gmail_signature" data-smartmail="gmail_signature"><font face="monospace">[root@ukdev-bastion-003 ~]# ip route add <a href="http://192.168.0.100/32">192.168.0.100/32</a> table 100 dev ens224 proto static metric 32</font><br></div></div></div></blockquote><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><br></div><div class="gmail_signature" data-smartmail="gmail_signature">To avoid overwhelming everyone with lots of repeated <font face="monospace">birdc show ospf state all</font> output (but do ask if anything would be useful!)
I'll describe what happens then:</div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><ol><li>The route shows up in the router section it was added for as an external (as I'd expect).</li><li>When the same route is added on the 2nd router, there is no change to the OSPF state.</li><li>When the route is removed from the 1st router (while still on the 2nd) the external is removed from the first router and appears on the 2nd one!</li><li>Finally, removing it from the 2nd router causes the external to disappear from the OSPF state.</li></ol><div>The kernel routing table behaviour (described previously) behaves exactly as I'd expect given what OSPF is doing here. But that isn't what I'm trying to achieve, at step 2 I would expect the external network to be registered by both routers simultaneously - and this is how it behaves if the route is configured using the static protocol (as you can see with the
<a href="http://172.27.0.0/19">172.27.0.0/19</a> route).</div><div><br></div><div>If anyone has any insight as to what I might be doing wrong here (or other ways of accomplishing the same goal) it would be greatly appreciated! I'm no expert and mostly just muddling my way through :)<br></div><div><br></div><div>Many thanks,<br>Asher</div><div><br></div><div>P.S. I think a API-based routing protocol as discussed on the list a few years back would be really cool to see for this sort of stuff.</div></div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><br></div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">~~~~~<br>"Their heads are green, and their hands are blue,<br> And they went to sea in a Sieve." - Edward Lear</div></div></div>