On Thu, Dec 16, 2021 at 02:37:00PM +0100, Nico Schottelius wrote:
Ondrej Zajicek <santiago@crfreenet.org> writes:
Yes, this is kind of confusing error message (as i noted in response to Simon Ruderich).
If there is one thing I'd suggest to improve first: print out the specific route and the next_hop - this would already help so much to debug the issue.
Agreed.
So the BGP link between kubernetes and APU-ROUTER is EBGP, while between APU-ROUTER and ROUTER is IBGP?
Correct.
I expect it is in multihop / gateway-recursive mode, as it is default for IBGP?
It's in direct mode for all links, as the APU-ROUTERS have physical links inside the kubernetes cluster as well as physical connection to the ROUTER.
This is likely the issue. In BIRD, relations between default values are: EBGP/IBGP -> direct / multihop -> gateway direct/recursive. So, by changing link mode to direct, you implicitly changed gateway mode from recursive to direct. In gateway direct mode, the next hop is not resolved through routing table, just through interface ranges. And if not found, it is rejected with 'Invalid next hop', so having OSPF route is irrelevant. But because link is IBGP, the original (indirect) next hop is there. Note that whole direct / gateway direct mode is designed primarily for EBGP. It can be used with IBGP on flat topology and 'next hop self' on IBGP links (so IGP/OSPF routes are not needed). Technically you can configure direct mode, and also 'gateway recursive' in channels. That should work, although it is probably less tested combination.
I actually just realised that the "k8s_p5_1_v6" protocol did not have the direct statement, however adding it does not change the situation (probably because bird detects it as a direct link anyway).
That is EBGP, which is direct by default (and that is OK). The issue is direct on IBGP. -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."