On Tue, Dec 14, 2021 at 08:35:48PM +0100, Nico Schottelius wrote:
Hello,
every year or then bird is putting me into the Invalid NEXT_HOP message.
Hello Yes, this is kind of confusing error message (as i noted in response to Simon Ruderich).
TL;DR: Why does bird on router1+router2 refuse the route 2a0a:e5c0:0:12:b01a:5ae3:1bd4:1e00/122 via 2a0a:e5c0::225:90ff:fe1e:3e62
even though router1+router2 know how to reach 2a0a:e5c0::/64 via fe80::20d:b9ff:fe57:2f91 by means of ospf?
Note that in gateway-recursive mode, the fact of whether reachability is known is not relevant for 'Invalid NEXT_HOP' message. If it is unreachable, then the route would be accepted as 'unreachable' and changes to reachable if reachability is changed. It is usually some other issue like zero or local IP address.
In detail:
router1, router2 are peered to apu-router1,apu-router2 via OSPF + BGP.
apu-router1,apu-router2 are peered to a set of kubernetes hosts.
The goal is to have router1 + router2 import the routes sent by the kubernetes hosts:
router1 router2---------| | \ | | | \ | | | \ | | apu-router1 apu-router2 | . | . | |-------------------------- . . [ kubernetes cluster via apu-routers ]
The problem: router1+router reject the routes with:
Dec 14 20:33:51 router1 daemon.err bird: apu_router1_place5_ungleich_ch_v6: Invalid NEXT_HOP attribute
The setup:
router1, router2, apu-router1, apu-router2 = ASN209898 kubernetes hosts = ASN65533 kubernetes peers with apu-routers only.
So the BGP link between kubernetes and APU-ROUTER is EBGP, while between APU-ROUTER and ROUTER is IBGP? I expect it is in multihop / gateway-recursive mode, as it is default for IBGP?
The routes: Kubernetes announces parts of 2a0a:e5c0:0:12::/64 and 2a0a:e5c0:0:13::/64, for instance the route 2a0a:e5c0:0:12:b01a:5ae3:1bd4:1e00/122.
Kubernetes nodes live in 2a0a:e5c0::/64.
apu-routers have a leg in 2a0a:e5c0::/64, via eth1.2. They reach the cluster directly. They have the routes.
routers1+2 receive the route for 2a0a:e5c0::/64 via ospf:
bird> show route 2a0a:e5c0::/64 Table master6: 2a0a:e5c0::/64 unicast [ospf6 17:08:18.515] * I (150/20) [0.0.0.47] via fe80::20d:b9ff:fe57:2f91 on bond0.8 The apu-routers: - They import the route [0]
Could you show 'show route all' on apu-router for failed routes to see their BGP_NEXT_HOP attribute? And also ideally tcpdump output to see BGP_NEXT_HOP as sent from apu-router to router?
- They export the route to the routers [1]
Could you also show 'show protocol all' for the session from apu-router to the kubernetes hosts?
The routers: - print 4x the Invalid NEXT_HOP attribute, once per exported kubernetes network - They ignore the 4 routes [2]
Question: why does bird on the routers not accept the routes? Or is there a different problem I am not seeing? Aside from that, shouldn't bird on the apu-routers set itself as nexthop for the kubernetes routes?
Not, because sending it to routerx over IBGP link, where BGP_NEXT_HOP is kept unmodified by default (unless 'next hop self' option is used). It might be an issue related to IPv6 dual next-hops (global and link-local), where global is empty. -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."