kernel: does not learn routes with RTPROT_KERNEL (proto kernel in ip-route(8) output)
Hello. Recently I spot following issue with BIRD: kernel syncer does not learn routes with RTPROT_KERNEL even with "learn" and "device routes" turned on. Config is wery simple: # Main routing table (master), mapped to ipt_main kernel table. # ipt_main == 254 (see /etc/iproute2/rt_tables) protocol kernel kernel254 { persist no; scan time 120; learn yes; device routes yes; kernel table ipt_main; import all; export all; } $ birdc 'show route' BIRD 1.3.10 ready. $ ip -4 route show table main 172.16.1.0/24 dev lo255 proto kernel scope link src 172.16.1.254 Can some one explain why this was done, and is there any way other than patching BIRD (at least sysdep/linux/netlink.c does checking and ignoring and sysdep/unix/krt.c makes its final verdict to ignore such route). Background of problem or why this might be important (at least for me). There is lot of text, and no need to read it, however it describes why learning routes with RTPROT_KENREL might be important. ----------------------------------------------------------------------------------------------- There is requirement to create separate Linux kernel RIB and install rules to direct traffic to it (ip-rules(8)). This is an PBR (Policy-Based Routing) BIRD has all necessary means to do that: protocols direct - to learn directly connected routes and pipe to import from main (master in BIRD) routing table some routes. And everything works as expected, except that protocol direct "imports" (better to say create may be?) routes based on interface ip address configuration. Contrary to this, kernel installs such directly connected routes with some attributes: like proto kernel, and very important as I discover src (krt_prefsrc attribute in BIRD). Actually attribute perfsrc used in kernel to determine outgoning source for packets that has no src addresses in it (application does not call bind(2), which is common for UDP). Without prefsrc on route, kernel uses fallback mechanism to find source: this is inet_select_addr() function, which is not ideal in functionality (in some cases strange source address might be choosen), and all it does it looks outgoing interface itself for primary address and other interfaces if interface has no such one). Recent kernels (linux-2.6.39+) has implemented genid table to cache information from inet_select_addr() (commits 436c3b66ec98 "ipv4: Invalidate nexthop cache nh_saddr more correctly." and 1fc050a13473 "ipv4: Cache source address in nexthop entries.") lookups, but implementation has also wholes (for example if interface has zero ip address on it and host on lan referenced by a route with interface as nexthop (so called ip unnumbered in proven vendor terms) as only event to regenerate (and thus recall inet_select_addr()) genid is changing state of the interface (i.e. changing route pointing to device has no chance to regenerate genid and packets might still go with source from previous call to inet_select_addr()). Some distributions with longterm support still provides 2.6.32 kernels. Having system with thousands interfaces and sending UDP for example over all of it is wery expensive in CPU terms. Thus I decided to learn routes from kernel protocol and then pass them trought pipe and install it in new kernel table for PBR. -- SP5474-RIPE Sergey Popovich
On Wed, Aug 07, 2013 at 02:11:46PM +0400, ???????????? ?????????????? wrote:
Hello.
Recently I spot following issue with BIRD: kernel syncer does not learn routes with RTPROT_KERNEL even with "learn" and "device routes" turned on.
Can some one explain why this was done, and is there any way other than patching BIRD (at least sysdep/linux/netlink.c does checking and ignoring and sysdep/unix/krt.c makes its final verdict to ignore such route).
Well, i don't know why kernel device routes are not imported in 'learn' mode and instead direct protocol is used. Some of these reasons are probably lost in the past (like in ancient Linux where device routes have to be generated by user), another possible reason is that you still need some external protocol for these routes - if you can learn kernel device routes from kernel protocol but if you overwrite them by a route from another protocol, then kernel protocol stops announcing such device routes and the device route is not reintroduced when the other protocol withdraws its route.
Background of problem or why this might be important (at least for me). There is lot of text, and no need to read it, however it describes why learning routes with RTPROT_KENREL might be important.
Thanks for the explanation, problem with missing krt_prefsrc is interesting. Perhaps the simple fix would be to add krt_prefsrc to routes generated by direct protocol? -- Elen sila lumenn' omentielvo Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
В письме от 13 августа 2013 10:33:45 пользователь Ondrej Zajicek написал:
On Wed, Aug 07, 2013 at 02:11:46PM +0400, ???????????? ?????????????? wrote:
Well, i don't know why kernel device routes are not imported in 'learn' mode and instead direct protocol is used. Some of these reasons are probably lost in the past (like in ancient Linux where device routes have to be generated by user), another possible reason is that you still need some external protocol for these routes - if you can learn kernel device routes from kernel protocol but if you overwrite them by a route from another protocol, then kernel protocol stops announcing such device routes and the device route is not reintroduced when the other protocol withdraws its route.
Yes, I'm aware of this problem, and this could happen only if I learn router from KRT (Kernel Routing Table) and then (possibly by changing some attrs) export it to the same KRT (correct me if other cases exists). But for case when these kernel routers installed in different KRT this would not happen.
Background of problem or why this might be important (at least for me). There is lot of text, and no need to read it, however it describes why learning routes with RTPROT_KENREL might be important.
Thanks for the explanation, problem with missing krt_prefsrc is interesting. Perhaps the simple fix would be to add krt_prefsrc to routes generated by direct protocol?
For now I have patch to address this issue. It works wery well for our setup on Linux with PBR and tousands of interfaces, inet_select_addr() is not case now as krt_prefsrc is preserved when reexporting route from one KRT to another. Also patch address some on *BSD systems (where also not all routes learned), however I do not have such system to test this. This patch also incorporates patch from Wiki (FAQ, BIRD does not import some routers from kernel): kern_dev.patch which was lost (not seen currently on new Wiki) possibly due to migration to the new Wiki page. Thank for your reply. -- SP5474-RIPE Sergey Popovich
Hello!
Well, i don't know why kernel device routes are not imported in 'learn' mode and instead direct protocol is used. Some of these reasons are probably lost in the past (like in ancient Linux where device routes have to be generated by user), another possible reason is that you still need some external protocol for these routes - if you can learn kernel device routes from kernel protocol but if you overwrite them by a route from another protocol, then kernel protocol stops announcing such device routes and the device route is not reintroduced when the other protocol withdraws its route.
If I remember correctly, the original intention was to keep kernel device routes completely separated, primarily to keep off various possible (though unlikely) problems, like the one mentioned by SanTiago. However, learning of device routes should work and it could be an interesting option to add. Have a nice fortnight -- Martin `MJ' Mares <mj@ucw.cz> http://mj.ucw.cz/ Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth Current root password is "p3s5vwF50". Keep secret.
participants (4)
-
Martin Mares -
Ondrej Zajicek -
Sergey Popovich -
Сергей Попович