On Thu, Dec 16, 2021 at 04:41:09PM +0100, Nico Schottelius wrote:
Ondrej Zajicek <santiago@crfreenet.org> writes:
I expect it is in multihop / gateway-recursive mode, as it is default for IBGP?
It's in direct mode for all links, as the APU-ROUTERS have physical links inside the kubernetes cluster as well as physical connection to the ROUTER.
This is likely the issue. In BIRD, relations between default values are:
Removing the direct statement from the iBGP sessions indeed fixes the problem. So technically speaking, what is the disadvantage/drawback of avoiding the "direct" statement everywhere? We have so far added it whenever the connection was in the same network segment (hence: direct), but if it is enabled for eBGP by default (makes sense) and it does not "help" in the iBGP case, should we ever use it?
It has some minor advantages (like BGP is enabled immediately after iface with neighbor address appears, BGP state can depend on layer-2 link state, you can use TTL security more reliably, you can run BGP over link-local addresses).
EBGP/IBGP -> direct / multihop -> gateway direct/recursive.
So, by changing link mode to direct, you implicitly changed gateway mode from recursive to direct.
In gateway direct mode, the next hop is not resolved through routing table, just through interface ranges. And if not found, it is rejected with 'Invalid next hop', so having OSPF route is irrelevant. But because link is IBGP, the original (indirect) next hop is there.
I understand the code path logic, but I don't understand the "configuration logic". Or as said above: does it ever make sense to apply direct explicitly?
Yes, if you have flat topology (all EBGP routers connected to shared layer-2 segment), do not run IGP/OSPF, and use 'next hop self' on all IBGP links, then enabling 'direct / gateway direct' can make sense compared to regular multihop IBGP with recursive next hop resolution. For example, in your case, you can decide to drop OSPF, propagate 2a0a:e5c0::/64 also through BGP and enable 'next hop self' on IBGP sessions. Then enabling 'direct' would make sense. But in your current setup i would probably recommend to use regular multihop IBGP. Also note the reverse case for this option - switching EBGP to multihop (which is used mostly in diagnostic setups / route collectors). One could argue that perhaps configuration logic: EBGP/IBGP -> direct / multihop EBGP/IBGP -> gateway direct/recursive would make more sense (i.e. switching IBGP to direct would not implicitly enable gatewa direct mode), but it makes less sense for the reverese case - if you switch EBGP to multihop then you do not want gateway direct. -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."