Understanding IPv6 next hops

[Neil Jerram]

Good morning all!

I'm debugging a situation where I'm seeing different IPv6 next hop
behaviour in two setups with different versions of my team's software.

In both setups:
There are 3 routers A, B and C, all peered with another router X.
They are all on the same L2 bridge, and have global IPv6 addresses in the
2001:20::/64 subnet.
A, B and C all export a route for fd00:10:96::/112

In setup #1 on X I see ECMP routes for fd00:10:96::/112 via link-local IPv6
addresses:

# ip -6 r
...
fd00:10:96::/112 proto bird metric 1024
        nexthop via fe80::42:acff:fe11:2 dev eth0 weight 1
        nexthop via fe80::42:acff:fe11:3 dev eth0 weight 1
        nexthop via fe80::42:acff:fe11:4 dev eth0 weight 1
        nexthop via fe80::42:acff:fe11:5 dev eth0 weight 1 pref medium

But in setup #2 on X I see ECMP routes for fd00:10:96::/112 via the global
IPv6 addresses:

# ip -6 r
...
fd00:10:96::/112 proto bird metric 1024
        nexthop via 2001:20::1 dev eth0 weight 1
        nexthop via 2001:20::2 dev eth0 weight 1
        nexthop via 2001:20::3 dev eth0 weight 1
        nexthop via 2001:20::8 dev eth0 weight 1 pref medium

That is the difference that I am trying to understand.  Digging further, I
used tcpdump to capture the BGP protocol on X, and in setup #1 I see

     nexthop: 2001:20::3, fe80::42:acff:fe11:4, nh-length: 32, no SNPA

in the BGP Update Reach NLRI, whereas in setup #2 I see just one next hop
address:

     nexthop: 2001:20::3, nh-length: 16, no SNPA

All this is with BIRD code, the same for both setups, that is 1.6.8 plus
some patches that I would not expect to be relevant.  (Specifically, this
code: https://github.com/projectcalico/bird/commits/feature-ipinip)

There are slight config differences between the two setups, but nothing
that is obviously relevant to this.  (I can provide those if need be.)

Any ideas?  Can you advise where I should look or check next, to try to
understand why the UPDATE message has two next hop addresses in one setup,
but only one in the other?

Also, does the passing of two next hop addresses in setup #1 fully explain
why the ECMP routes programmed into the kernel use link-local gateway
addresses?

Also, are the routes with global next hops more correct in some sense than
those with link-local next hops; or vice versa?  Would you expect them both
to forward data correctly?

Many thanks,
   Neil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://trubka.network.cz/pipermail/bird-users/attachments/20200206/6ec516ed/attachment.htm>