Force gateway recursive lookup in iBGP routes

28 Feb 2020

      Hello there,

I am currently trying to use BIRD for route propagation from our
baremetal Kubernetes clusters (Calico CNI, iBGP sessions within AS65100)
into infrastructure via eBGP (private AS) and it works well.

The issue I have is when I want also to create BGP peering between BIRD
and (MetalLB) service inside Kubernetes cluster (multihop, no NAT
involved, session established OK) and I receive routes /32 with
BGP.next_hop to IP within Kubernetes cluster (=not directly connected).
These routes are marked as "unreachable" even if I explicitly set
"gateway recursive".

I know this recursive gateway lookup works well for routes learned from
eBGP, but I can't make Calico peers external because it won't work in my
setup. Calico nodes has iBGP full mesh and I would receive all routes
from every single node and I wouldn't be able to distinguish which lives
where.

Unfortunately BGP peer inside cluster has no support to modify next_hop
and always sends self, so I am looking for workaround. Also I cannot set
specific "gw" in import filter, because I have these multihop peers
configured with bgp neighbor range subnet (there will be multiple of
them, I don't know exact IP addresses in advance).

Configuration snippet like this:

# calico cluster peers
filter bgp_in_okubedev1_calico {
        if net ~ [ 10.96.16.0/20+ ] then accept;
        reject;
}

protocol bgp okubedev1m1 {
        local 10.30.20.180 as 65100;
        neighbor 10.30.21.19 as 65100;

        passive yes;

        ipv4 {
                import filter bgp_in_okubedev1_calico;
                export none;
        };
}

# metallb multihop peers
filter bgp_in_okubedev1_metallb {
        # gw is recursively looked up localy and passed into BGP.next_hop
        #bgp_next_hop = gw;

        if net ~ [ 10.96.255.32/28+ ] then accept;
        reject;
}

protocol bgp okubedev1_lb_tpl {
        local 10.30.20.180 as 65100;
        neighbor range 10.96.16.0/20 as 65100;

        passive yes;
        multihop;

        ipv4 {
                gateway recursive;

                import filter bgp_in_okubedev1_metallb;
                export none;
        };
}

Produces following routing table:

bird> show route all
Table master4:
10.96.255.33/32      unreachable [okubedev1_lb1 10:32:14.973 from 10.96.20.25] * (100) [?]
        Type: BGP univ
        BGP.origin: Incomplete
        BGP.as_path: 
        BGP.next_hop: 10.96.20.25
        BGP.local_pref: 0
10.96.20.0/26        unicast [okubedev1m1 11:15:45.704] * (100) [i]
        via 10.30.21.19 on enp0s4
        Type: BGP univ
        BGP.origin: IGP
        BGP.as_path: 
        BGP.next_hop: 10.30.21.19
        BGP.local_pref: 100
10.30.20.0/22        unicast [direct1 13:52:46.301] * (240)
        dev enp0s4
        Type: device univ

I am almost sure I am missing some key BIRD or BGP feature, which I need
to know to understand this behavior properly.

Any comment or suggestion would be appreciated.

Best regards

-- 
Miroslav Kalina
Systems developement specialist

miroslav.kalina@livesport.eu
+420 773 071 848

Livesport s.r.o.
Aspira Business Centre
Bucharova 2928/14a, 158 00 Praha 5
www.livesport.eu