We are setting up a container based proof of concept with IPv6 only using ECMP, ANYCAST, BGP with BFP and Bird2 and I've hit a problem where BFD doesn't recover after a local link goes down and comes back and are seeking some advice as whether this is expected behaviour or a bug. We have installed Bird2 into 2 LXC containers both with BFD and BGP enabled talking via IPv6 using ULA. Each container is on a different subnet (fd00::1/112 & fd01::1/112). Each container is connected to a different bridge (lxdbr0 & lxdbr1) and the host has routes to link them together. This works fine! We are now testing what happens when the link is broken in various ways. 1. Deleting a host route - works fine BFP sessions recovers when route is re-added. 2. Link down on the host side veth that is attached to the bridge - BFD fails to recover when link is brought back up. The problem seems to be "bfd1: Socket error: bind: Cannot assign requested address" If on the container we then do "birdc resart bfd1" the link recovers. "CONFIG" log syslog all; define BGP_ID = 10.9.8.7; define ROUTE_IP = fd00::1; define HOST_ASN = 64512; define STATIC_IP = fd00:b00b:1e00::/64; define NEIGH_IP = fd01::1; define NEIGH_ASN = 64513; define NEIGH_PASS = "test"; router id BGP_ID; protocol device {     scan time 10;    # Set scanning time. Default is disabled. } protocol direct {     disabled;    # Enabled by Default. } filter noroutes {   if net = STATIC_IP then accept;   else reject;        # Disabled by Default. } filter allroutes {     accept;        # Disabled by Default. } protocol kernel {   ipv6 {        # Connect protocol to IPv6 table by channel.     import none;    # Import to table. Default is import all.     export none;    # Export to protocol. Default is export none.     }; } protocol static {   ipv6;     route STATIC_IP via ROUTE_IP; } protocol bgp infra_v6 {            # Name of the BGP interface.     description "BGP uplink 0";    # Description of the BGP interface.         local ROUTE_IP as HOST_ASN;     neighbor NEIGH_IP as NEIGH_ASN;     hold time 600;            # Hold time, time in seconds to wait for the keepalive to initiate.     password NEIGH_PASS;        # Password, type whatever password you wish.     multihop;            # Multi-hop enabled for more servers in a cluster.     bfd on;                # Switch to turn on BFD   ipv6 {     import filter noroutes;        # Importing previous IPv6 filters.     export filter allroutes;    # Exporting all routed filters shown above (Line 30-33).     }; } protocol bfd {     multihop {         interval 2s; };     neighbor NEIGH_IP; } "END OF CONFIG" Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Lost carrier Jan 27 15:01:17 c2 systemd-networkd[7304]: NDISC: Stopping IPv6 Router Solicitation client Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Removing address fd01::1 Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Removing address fd42:1af0:83cf:4510:216:3eff:fe98:2650 Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Removing route: dst: fd00::1/112, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: main, proto: static, type: unicast Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Removing route: dst: n/a, src: n/a, gw: fe80::216:3eff:fe41:d39f, prefsrc: n/a, scope: global, table: main, proto: ra, type: unicast Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Removing route: dst: fd42:1af0:83cf:4510::/64, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: main, proto: ra, type: unicast Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: State is configured, dropping config Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Removing route: dst: fd00::/112, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: main, proto: static, type: unicast Jan 27 15:01:17 c2 systemd-networkd[7304]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/network1 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=26 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Forgetting address: fd01::1/112 (valid forever) Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Forgetting route: dst: fd01::1/128, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: local, proto: kernel, type: local Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Forgetting route: dst: fd01::/112, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: main, proto: kernel, type: unicast Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Forgetting address: fd42:1af0:83cf:4510:216:3eff:fe98:2650/64 (valid for 59min 34s) Jan 27 15:01:17 c2 systemd-networkd[7304]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/network1/link/_310 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=27 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a Jan 27 15:01:17 c2 systemd-networkd[7304]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/network1 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=28 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Forgetting route: dst: fd42:1af0:83cf:4510:216:3eff:fe98:2650/128, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: local, proto: kernel, type: local Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Forgetting route: dst: fd00::/112, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: main, proto: static, type: unicast Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Forgetting route: dst: n/a, src: n/a, gw: fe80::216:3eff:fe41:d39f, prefsrc: n/a, scope: global, table: main, proto: ra, type: unicast Jan 27 15:01:17 c2 systemd-networkd[7304]: eth0: Forgetting route: dst: fd42:1af0:83cf:4510::/64, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: main, proto: ra, type: unicast Jan 27 15:01:18 c2 bird[6234]: bfd1: Socket error: Network is unreachable Jan 27 15:01:20 c2 bird[6234]: bfd1: Socket error: Network is unreachable Jan 27 15:01:20 c2 systemd-networkd[7304]: rtnl: received non-static neighbor, ignoring. Jan 27 15:01:21 c2 bird[6234]: bfd1: Socket error: Network is unreachable Jan 27 15:01:23 c2 bird[6234]: bfd1: Socket error: Network is unreachable Jan 27 15:01:25 c2 bird[6234]: bfd1: Socket error: Network is unreachable Jan 27 15:01:26 c2 bird[6234]: bfd1: Socket error: Network is unreachable Jan 27 15:01:26 c2 bird[6234]: bfd1: Socket error: Network is unreachable Jan 27 15:01:26 c2 bird[6234]: bfd1: Socket error: bind: Cannot assign requested address Jan 27 15:01:32 c2 systemd-networkd[7304]: eth0: Flags change: +LOWER_UP +RUNNING Jan 27 15:01:32 c2 systemd-networkd[7304]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/network1/link/_310 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=29 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a Jan 27 15:01:32 c2 systemd-networkd[7304]: eth0: Gained carrier Jan 27 15:01:32 c2 systemd-networkd[7304]: eth0: Discovering IPv6 routers Jan 27 15:01:32 c2 systemd-networkd[7304]: NDISC: Started IPv6 Router Solicitation client Jan 27 15:01:32 c2 systemd-networkd[7304]: eth0: State changed: configured -> configuring Jan 27 15:01:32 c2 systemd-networkd[7304]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/network1/link/_310 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=30 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a Jan 27 15:01:32 c2 systemd-networkd[7304]: eth0: Setting addresses Jan 27 15:01:32 c2 systemd-networkd[7304]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/network1 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=31 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a Jan 27 15:01:32 c2 systemd-networkd[7304]: eth0: Remembering route: dst: fd01::/112, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: main, proto: kernel, type: unicast Jan 27 15:01:32 c2 systemd-networkd[7304]: eth0: Remembering updated address: fd01::1/112 (valid forever) Jan 27 15:01:32 c2 systemd-networkd[7304]: eth0: Addresses set Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Remembering updated address: fd01::1/112 (valid forever) Jan 27 15:01:33 c2 systemd-networkd[7304]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/network1/link/_310 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=32 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Configuring route: dst: fd00::1/112, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: main, proto: static, type: unicast Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Setting routes Jan 27 15:01:33 c2 systemd-networkd[7304]: NDISC: Sent Router Solicitation, next solicitation in 3s Jan 27 15:01:33 c2 systemd-networkd[7304]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/network1 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=33 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Remembering route: dst: fd01::1/128, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: local, proto: kernel, type: local Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Remembering route: dst: fd00::/112, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: main, proto: static, type: unicast Jan 27 15:01:33 c2 systemd-networkd[7304]: NDISC: Received Router Advertisement: flags OTHER preference medium lifetime 1800 sec Jan 27 15:01:33 c2 systemd-networkd[7304]: NDISC: Invoking callback for 'router' event. Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Acquiring DHCPv6 lease on NDisc request Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Configuring route: dst: n/a, src: n/a, gw: fe80::216:3eff:fe41:d39f, prefsrc: n/a, scope: global, table: main, proto: ra, type: unicast Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Configuring route: dst: fd42:1af0:83cf:4510::/64, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: main, proto: ra, type: unicast Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Routes set Jan 27 15:01:33 c2 systemd-networkd[7304]: rtnl: received non-static neighbor, ignoring. Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Received remembered route: dst: n/a, src: n/a, gw: fe80::216:3eff:fe41:d39f, prefsrc: n/a, scope: global, table: main, proto: ra, type: unicast Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Received remembered route: dst: fd42:1af0:83cf:4510::/64, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: main, proto: ra, type: unicast Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Remembering updated address: fd42:1af0:83cf:4510:216:3eff:fe98:2650/64 (valid for 1h) Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Configuring route: dst: fd00::1/112, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: main, proto: static, type: unicast Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Setting routes Jan 27 15:01:33 c2 systemd-networkd[7304]: eth0: Routes set Jan 27 15:01:33 c2 systemd-networkd[7304]: rtnl: received non-static neighbor, ignoring. Jan 27 15:01:34 c2 systemd-networkd[7304]: eth0: Remembering updated address: fd42:1af0:83cf:4510:216:3eff:fe98:2650/64 (valid for 59min 59s) Jan 27 15:01:34 c2 systemd-networkd[7304]: eth0: State changed: configuring -> configured Jan 27 15:01:34 c2 systemd-networkd[7304]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/network1/link/_310 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=34 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a Jan 27 15:01:34 c2 systemd-networkd[7304]: eth0: Remembering route: dst: fd42:1af0:83cf:4510:216:3eff:fe98:2650/128, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: local, proto: kernel, type: local Jan 27 15:01:39 c2 systemd-networkd[7304]: rtnl: received non-static neighbor, ignoring. Jan 27 15:01:39 c2 systemd-networkd[7304]: rtnl: received non-static neighbor, ignoring. Jan 27 15:01:44 c2 systemd-networkd[7304]: NDISC: No RA received before link confirmation timeout Jan 27 15:01:44 c2 systemd-networkd[7304]: NDISC: Invoking callback for 'timeout' event.