IPv6 routes not imported into Kernel

Robert Finze robert at finze.it
Tue Feb 27 22:58:47 CET 2024


Hi Gerdriaan,

thanks a lot for your input!
I haven't had much time to continue on this until now.

Please see my replies inline:


On 01.01.24 19:15, Gerdriaan Mulder wrote:
> Hi Robert,
> 
> On 15/11/2023 22:58, Robert Finze wrote:
>> The Bird config on both systems is nearly identical (only IPs differ) 
>> and also the systems are setup in a similar manner.
> 
> It would be good to have a dump of the configuration of the non-working 
> system (redact sensitive information such as passwords etc, but leave 
> other information intact).

I've attached the config.

>> The routes are correctly learned from upstream and exported to the 
>> kernel, but the kernel is not "learning" them.
> 
> Interesting. The following dumps you sent might further help debugging 
> the problem.
> 
>> Netlink route
>> 0000   00 04 03 38 00 00 00 00 00 00 00 00 00 00 00 00   ...8............
>> 0010   68 00 00 00 18 00 05 05 13 0d 66 02 00 00 00 00   h.........f.....
>> 0020   0a 28 00 00 fe 0c 00 01 00 00 00 00 14 00 01 00   .(..............
>> 0030   26 07 ff 00 0b 00 00 00 00 00 00 00 00 00 00 00   &...............
>> 0040   08 00 06 00 20 00 00 00 14 00 07 00 2a 0e 39 40   .... .......*.9@
>> 0050   10 00 00 00 00 00 00 00 00 00 00 02 08 00 04 00   ................
>> 0060   02 00 00 00 14 00 05 00 2a 0e 39 40 de ad 00 00   ........*.9 at ....
>> 0070   00 00 00 00 00 00 00 01                           ........
> 
> This decodes to (Wireshark supports "Import from hexdump", as I found out):
> 
> Linux rtnetlink (route netlink) protocol
>      Netlink message header (type: Add network route)
>          Length: 104
>          Message type: Add network route (24)
>          Flags: 0x0505
>          Flags: 0x0505
>          Sequence: 40242451
>          Port ID: 0
>      Address family: AF_INET6 (10)
>      Length of destination: 40
>      Length of source: 0
>      TOS filter: 0x00
>      Routing table ID: 254
>      Routing protocol: BIRD (0x0c)
>      Route origin: global route (0x00)
>      Route type: Gateway or direct route (0x01)
>      Route flags: 0x00000000
>      Attribute: Route destination address
>          Len: 20
>          Type: 0x0001, Route destination address (1)
>          Data: 2607ff000b0000000000000000000000
>      Attribute: RTA_PRIORITY
>          Len: 8
>          Type: 0x0006, RTA_PRIORITY (6)
>          Data: 20000000
>      Attribute: RTA_PREFSRC
>          Len: 20
>          Type: 0x0007, RTA_PREFSRC (7)
>          Data: 2a0e3940100000000000000000000002
>      Attribute: Output interface index: 2
>          Len: 8
>          Type: 0x0004, Output interface index (4)
>          Output interface index: 2
>      Attribute: Gateway of the route
>          Len: 20
>          Type: 0x0005, Gateway of the route (5)
>          Data: 2a0e3940dead00000000000000000001
> 
>> 0000   00 04 03 38 00 00 00 00 00 00 00 00 00 00 00 00   ...8............
>> 0010   7c 00 00 00 02 00 00 00 13 0d 66 02 7a 31 09 81   |.........f.z1..
>> 0020   ea ff ff ff 68 00 00 00 18 00 05 05 13 0d 66 02   ....h.........f.
>> 0030   00 00 00 00 0a 28 00 00 fe 0c 00 01 00 00 00 00   .....(..........
>> 0040   14 00 01 00 26 07 ff 00 0b 00 00 00 00 00 00 00   ....&...........
>> 0050   00 00 00 00 08 00 06 00 20 00 00 00 14 00 07 00   ........ .......
>> 0060   2a 0e 39 40 10 00 00 00 00 00 00 00 00 00 00 02   *.9 at ............
>> 0070   08 00 04 00 02 00 00 00 14 00 05 00 2a 0e 39 40   ............*.9@
>> 0080   de ad 00 00 00 00 00 00 00 00 00 01               ............
> 
> decodes as:
> 
> Netlink message
>      Netlink message header (type: Error)
>          Length: 124
>          Message type: Error (0x0002)
>          Flags: 0x0000
>          Sequence: 40242451
>          Port ID: 2164863354
>      Error code: Invalid argument (-EINVAL) (-22)
>      Netlink message header (type: 0x0018)
>          Length: 104
>          Message type: Protocol-specific (0x0018)
>          Flags: 0x0505
>          Flags: 0x0505
>          Sequence: 40242451
>          Port ID: 0
> 
> The first message could probably be replicated by running:
> 
> ip -6 route add 2607:ff00:b::/40 via 2a0e:3940:dead::1 table 254 
> protocol bird scope global src 2a0e:3940:1000::2 dev 2

this returns:
RTNETLINK answers: No route to host

> - where dev 2 indicates the network interface with index 2, this is 
> probably ens20 in your setup?

It should be ens19. I'm currently not sure how to verify that.
"ip a" shows:

1: lo
2: ens18
3: ens19
4: ens20
5: dummy0

> - table 254 is most likely the main table (see /etc/iproute2/rt_tables)

Correct, this is 'main'.

> I'm unsure how to decode RTA_PRIORITY correctly here. Regardless, you 
> could run this command on the non-working host. Perhaps `ip route` can 
> tell you a bit more information. In a slightly modified case (I've 
> replaced the `via ...` with a known gateway), I get: "Error: Invalid 
> source address." (with: iproute2-6.5.0)
> 
> My current hunch is that `src 2a0e:3940:1000::2` is not a valid address 
> on your system. A closer read on your earlier comment:

This ip is bound on the dummy0 interface:

5: dummy0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue 
state DOWN group default qlen 1000
     link/ether ba:bc:b1:56:92:71 brd ff:ff:ff:ff:ff:ff
     inet 45.95.204.2/32 scope global dummy0
        valid_lft forever preferred_lft forever
     inet6 2a0e:3940:1000::2/128 scope global tentative


One difference here to the system running 20.04 is the state of the 
dummy interface, which is shown there as:

8: dummy0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue 
state UNKNOWN group default qlen 1000

Both interfaces are configured using Netplan and the config there is the 
same (apart the IP address).

> 
>> The Bird config on both systems is nearly identical (only IPs differ) 
> 
> suggests to look in this direction.
> 
> Best regards,
> Gerdriaan Mulder

Not sure if this helps, but this is the current ip6 routing table:

# ip -6 r
::1 dev lo proto kernel metric 256 pref medium
2a0e:3940:1000::2 dev dummy0 proto kernel metric 256 linkdown pref medium
2a0e:3940:1000::/36 dev ens19 proto bird metric 32 pref medium
2a0e:3940:2000::/36 dev ens19 proto bird metric 32 pref medium
2a0e:3940:dead::/64 dev ens18 proto kernel metric 256 pref medium
fe80::/64 dev ens20 proto kernel metric 256 pref medium
fe80::/64 dev ens19 proto kernel metric 256 pref medium
fe80::/64 dev ens18 proto kernel metric 256 pref medium


In the meantime I've setup a clean new VM with Ubuntu 22.04 and the same 
issues occurred.
I've upgraded that new VM to 24.04 and still the same.

Next I want to try a fresh 20.04 install and see what happens.
Maybe I'll try the 3.0alpha and give that a shot.

To be honest, I'm not even sure if this is a bird issue or a "linux" 
issue. But starting debugging this from the bird side seems sensible to me.

Thanks a lot for the support!

Best,
Robert
-------------- next part --------------
# ---- Parameters ----

timeformat base iso long;
timeformat log iso long;
timeformat protocol iso long;
timeformat route iso long;

log syslog all;

define ASN = 60767;
router id 45.95.204.2;

include "variables.conf";
include "functions.conf";

# ---- Protocols ---- #
protocol device {};

protocol direct {
  ipv4 {
    import all;
  };
  ipv6 {
    import all;
  };
}

protocol kernel {
  scan time 10;
  ipv4 {
    export filter {
      if (proto = "explstatic4") then accept;
      if source = RTS_STATIC && proto != "static4" then {
        accept;
      } else if source = RTS_BGP then {
        krt_prefsrc = 45.95.204.2;
        accept;
      }
      reject;
    };
  };
}

protocol kernel {
  scan time 10;
  ipv6 {
    export filter {
      if (proto = "explstatic6") then accept;
      if source = RTS_STATIC && proto != "static6" then {
        accept;
      } else if source = RTS_BGP then {
        krt_prefsrc = 2a0e:3940:1000::2;
        accept;
      }
      reject;
    };
  };
}

protocol static static4 {
  ipv4;
  route 45.95.204.0/24 reject {
    preference = 10;
  };
}

protocol static static6 {
  ipv6;
  route 2a0e:3940:1000::/36 reject {
    preference = 10;
  };
  route 2a0e:3940:2000::/36 reject {
    preference = 10;
  };
}

protocol static localstatic4 {
  ipv4;
  route 45.95.204.0/24 via "ens19";
  route 10.10.20.0/24 via 10.10.10.254;
}

protocol static localstatic6 {
  ipv6;
  route 2a0e:3940:1000::/36 via "ens19";
  route 2a0e:3940:2000::/36 via "ens19";
}

# ---- RPKI ----
roa4 table rpki4;
roa6 table rpki6;

protocol rpki {
  roa4 { table rpki4; };
  roa6 { table rpki6; };

  transport tcp;
  remote "127.0.0.1" port 8282;
  #remote "rtr.rpki.cloudflare.com" port 8282;

  retry keep 90;
  refresh keep 900;
  expire keep 172800;
}
function reject_rpki_invalid() {
  if (net.type = NET_IP4) then {
    if (roa_check(rpki4, net, bgp_path.last_nonaggregated) = ROA_INVALID) then _reject("RPKI invalid");
  }

  if (net.type = NET_IP6) then {
    if (roa_check(rpki6, net, bgp_path.last_nonaggregated) = ROA_INVALID) then _reject("RPKI invalid");
  }
}

function force_rpki_strict() {
  if (net.type = NET_IP4) then {
    if (roa_check(rpki4, net, bgp_path.last_nonaggregated) != ROA_VALID) then _reject("RPKI != ROA_VALID");
  }

  if (net.type = NET_IP6) then {
    if (roa_check(rpki6, net, bgp_path.last_nonaggregated) != ROA_VALID) then _reject("RPKI != ROA_VALID");
  }
}

# ---- Peers ---- #

protocol bgp ibgp_4 {
    local as ASN;
    neighbor 10.10.10.1 as 60767;
    description "internal v4";
    allow local as ASN;
    direct;

    ipv4 {
        next hop self;
        import table on;
        import all;
        export all;
    };
}

protocol bgp ibgp_6 {
    local as ASN;
    neighbor 2a0e:3940:dead::1 as 60767;
    description "internal v6";
    allow local as ASN;
    direct;

    ipv6 {
        next hop self;
        import table on;
        import all;
        export all;
    };
}


More information about the Bird-users mailing list