bird2 doesn't negotiate with gobgp over ipv6
Hi! I'm running bird 2.15.1 and I'm trying to make it talk to cilium that uses gobgp under the hood. I can negotiate the session over ipv4 with no issues, but due to a bug in the cilium implementation, the ipv6 routes aren't advertised properly. The suggestion from cilium was to negotiate the session over ipv6, but this creates issues for bird. I have a config that looks somewhat like this: protocol bgp { local port 179 as 65100; neighbor range 0::/0 port 179 as 65100 internal; dynamic name "GW6_"; direct; passive; ipv4 { import all; }; ipv6 { import all; }; } protocol bgp { local port 179 as 65100; neighbor range 10.0.0.0/8 port 179 as 65100 internal; dynamic name "GW4_"; direct; passive; ipv4 { import all; }; ipv6 { import all; }; } When I connect to bird over ipv4, gobgp sends a packet like this: Frame 4: 154 bytes on wire (1232 bits), 154 bytes captured (1232 bits) on interface vmbrdev, id 0 Ethernet II, Src: 22:fc:7f:47:6b:f4 (22:fc:7f:47:6b:f4), Dst: 6e:82:6c:1b:e9:95 (6e:82:6c:1b:e9:95) Internet Protocol Version 4, Src: 10.224.129.15, Dst: 10.224.129.1 Transmission Control Protocol, Src Port: 39201, Dst Port: 179, Seq: 1, Ack: 1, Len: 88 Border Gateway Protocol - OPEN Message Marker: ffffffffffffffffffffffffffffffff Length: 88 Type: OPEN Message (1) Version: 4 My AS: 65100 Hold Time: 90 BGP Identifier: 10.224.129.15 Optional Parameters Length: 59 Optional Parameters Optional Parameter: Capability Parameter Type: Capability (2) Parameter Length: 57 Capability: Route refresh capability Type: Route refresh capability (2) Length: 0 Capability: FQDN Capability Type: FQDN Capability (73) Length: 12 Hostname Length: 10 Hostname: dev-master Domain Name Length: 0 Domain Name: Capability: Unknown capability 75 Type: Unknown (75) Length: 13 Unknown: 0c476f4247502f332e32332e30 Capability: Multiprotocol extensions capability Type: Multiprotocol extensions capability (1) Length: 4 AFI: IPv4 (1) Reserved: 00 SAFI: Unicast (1) Capability: Multiprotocol extensions capability Type: Multiprotocol extensions capability (1) Length: 4 AFI: IPv6 (2) Reserved: 00 SAFI: Unicast (1) Capability: Support for 4-octet AS number capability Type: Support for 4-octet AS number capability (65) Length: 4 AS Number: 65100 Capability: Extended Next Hop Encoding Type: Extended Next Hop Encoding (5) Length: 6 AFI: IPv4 (1) SAFI: Unicast (1) Next hop AFI: IPv6 (2) Which is followed up by OPEN from bird2 and they establish the session. The logs say: May 04 14:47:02 ammy bird[7755]: bgp2: Incoming connection from 10.224.129.15 (port 39201) accepted May 04 14:47:02 ammy bird[7755]: GW4_1: Initializing May 04 14:47:02 ammy bird[7755]: GW4_1: Starting May 04 14:47:02 ammy bird[7755]: GW4_1: State changed to start May 04 14:47:02 ammy bird[7755]: GW4_1: Started May 04 14:47:02 ammy bird[7755]: GW4_1: Got OPEN(as=65100,hold=90,id=10.224.129.15) May 04 14:47:02 ammy bird[7755]: GW4_1: Sending OPEN(ver=4,as=65100,hold=240,id=0ae08101) May 04 14:47:02 ammy bird[7755]: GW4_1: Sending KEEPALIVE May 04 14:47:02 ammy bird[7755]: GW4_1: Got KEEPALIVE May 04 14:47:02 ammy bird[7755]: GW4_1: BGP session established When I try to establish the connection over ipv6, gobgp sends an OPEN that's exactly the same sans being transferred over ipv6: Frame 15: 174 bytes on wire (1392 bits), 174 bytes captured (1392 bits) on interface vmbrdev, id 0 Ethernet II, Src: 22:fc:7f:47:6b:f4 (22:fc:7f:47:6b:f4), Dst: 6e:82:6c:1b:e9:95 (6e:82:6c:1b:e9:95) Internet Protocol Version 6, Src: 2a01:xxx:xxx:xxx:xxx:xxx:xxx:d930, Dst: fd00::53 Transmission Control Protocol, Src Port: 58663, Dst Port: 179, Seq: 1, Ack: 1, Len: 88 and then bird just stalls the connection. The logs say: May 04 14:49:17 ammy bird[7755]: bgp1: Incoming connection from 2a01:4f8:173:fe11:6f81:e7cd:2787:fb71 (port 35581) accepted May 04 14:49:17 ammy bird[7755]: GW6_1: Initializing May 04 14:49:17 ammy bird[7755]: GW6_1: Starting May 04 14:49:17 ammy bird[7755]: GW6_1: State changed to start May 04 14:49:17 ammy bird[7755]: GW6_1: Waiting for 2a01:xxx:xxx:xxx:xxx:xxx:xxx:d930 to become my neighbor I suspect it's either a bug or a misconfiguration on the bird side, because gobgp behaves the same in both scenarios. How do I investigate/debug this further?
Not at my computer, just a wild guess – would you try please, instead of ::/0, narrowing the range, e.g. like 2000::/3? I haven't seen the code recently but this may help. Maria On 4 May 2024 17:16:21 CEST, Vladimir Pouzanov <farcaller@gmail.com> wrote:
Hi!
I'm running bird 2.15.1 and I'm trying to make it talk to cilium that uses gobgp under the hood. I can negotiate the session over ipv4 with no issues, but due to a bug in the cilium implementation, the ipv6 routes aren't advertised properly. The suggestion from cilium was to negotiate the session over ipv6, but this creates issues for bird.
I have a config that looks somewhat like this:
protocol bgp { local port 179 as 65100; neighbor range 0::/0 port 179 as 65100 internal;
dynamic name "GW6_"; direct; passive;
ipv4 { import all; };
ipv6 { import all; }; }
protocol bgp { local port 179 as 65100; neighbor range 10.0.0.0/8 port 179 as 65100 internal;
dynamic name "GW4_"; direct; passive;
ipv4 { import all; };
ipv6 { import all; }; }
When I connect to bird over ipv4, gobgp sends a packet like this:
Frame 4: 154 bytes on wire (1232 bits), 154 bytes captured (1232 bits) on interface vmbrdev, id 0 Ethernet II, Src: 22:fc:7f:47:6b:f4 (22:fc:7f:47:6b:f4), Dst: 6e:82:6c:1b:e9:95 (6e:82:6c:1b:e9:95) Internet Protocol Version 4, Src: 10.224.129.15, Dst: 10.224.129.1 Transmission Control Protocol, Src Port: 39201, Dst Port: 179, Seq: 1, Ack: 1, Len: 88 Border Gateway Protocol - OPEN Message Marker: ffffffffffffffffffffffffffffffff Length: 88 Type: OPEN Message (1) Version: 4 My AS: 65100 Hold Time: 90 BGP Identifier: 10.224.129.15 Optional Parameters Length: 59 Optional Parameters Optional Parameter: Capability Parameter Type: Capability (2) Parameter Length: 57 Capability: Route refresh capability Type: Route refresh capability (2) Length: 0 Capability: FQDN Capability Type: FQDN Capability (73) Length: 12 Hostname Length: 10 Hostname: dev-master Domain Name Length: 0 Domain Name: Capability: Unknown capability 75 Type: Unknown (75) Length: 13 Unknown: 0c476f4247502f332e32332e30 Capability: Multiprotocol extensions capability Type: Multiprotocol extensions capability (1) Length: 4 AFI: IPv4 (1) Reserved: 00 SAFI: Unicast (1) Capability: Multiprotocol extensions capability Type: Multiprotocol extensions capability (1) Length: 4 AFI: IPv6 (2) Reserved: 00 SAFI: Unicast (1) Capability: Support for 4-octet AS number capability Type: Support for 4-octet AS number capability (65) Length: 4 AS Number: 65100 Capability: Extended Next Hop Encoding Type: Extended Next Hop Encoding (5) Length: 6 AFI: IPv4 (1) SAFI: Unicast (1) Next hop AFI: IPv6 (2)
Which is followed up by OPEN from bird2 and they establish the session. The logs say:
May 04 14:47:02 ammy bird[7755]: bgp2: Incoming connection from 10.224.129.15 (port 39201) accepted May 04 14:47:02 ammy bird[7755]: GW4_1: Initializing May 04 14:47:02 ammy bird[7755]: GW4_1: Starting May 04 14:47:02 ammy bird[7755]: GW4_1: State changed to start May 04 14:47:02 ammy bird[7755]: GW4_1: Started May 04 14:47:02 ammy bird[7755]: GW4_1: Got OPEN(as=65100,hold=90,id=10.224.129.15) May 04 14:47:02 ammy bird[7755]: GW4_1: Sending OPEN(ver=4,as=65100,hold=240,id=0ae08101) May 04 14:47:02 ammy bird[7755]: GW4_1: Sending KEEPALIVE May 04 14:47:02 ammy bird[7755]: GW4_1: Got KEEPALIVE May 04 14:47:02 ammy bird[7755]: GW4_1: BGP session established
When I try to establish the connection over ipv6, gobgp sends an OPEN that's exactly the same sans being transferred over ipv6:
Frame 15: 174 bytes on wire (1392 bits), 174 bytes captured (1392 bits) on interface vmbrdev, id 0 Ethernet II, Src: 22:fc:7f:47:6b:f4 (22:fc:7f:47:6b:f4), Dst: 6e:82:6c:1b:e9:95 (6e:82:6c:1b:e9:95) Internet Protocol Version 6, Src: 2a01:xxx:xxx:xxx:xxx:xxx:xxx:d930, Dst: fd00::53 Transmission Control Protocol, Src Port: 58663, Dst Port: 179, Seq: 1, Ack: 1, Len: 88
and then bird just stalls the connection. The logs say:
May 04 14:49:17 ammy bird[7755]: bgp1: Incoming connection from 2a01:4f8:173:fe11:6f81:e7cd:2787:fb71 (port 35581) accepted May 04 14:49:17 ammy bird[7755]: GW6_1: Initializing May 04 14:49:17 ammy bird[7755]: GW6_1: Starting May 04 14:49:17 ammy bird[7755]: GW6_1: State changed to start May 04 14:49:17 ammy bird[7755]: GW6_1: Waiting for 2a01:xxx:xxx:xxx:xxx:xxx:xxx:d930 to become my neighbor
I suspect it's either a bug or a misconfiguration on the bird side, because gobgp behaves the same in both scenarios. How do I investigate/debug this further?
-- Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o.
I actually started with my /64 and then expanded to ::/0 to the same result. On Sat 4 May 2024 at 16:54, Maria Matejka <maria.matejka@nic.cz> wrote:
Not at my computer, just a wild guess – would you try please, instead of ::/0, narrowing the range, e.g. like 2000::/3? I haven't seen the code recently but this may help.
Maria
On 4 May 2024 17:16:21 CEST, Vladimir Pouzanov <farcaller@gmail.com> wrote:
Hi!
I'm running bird 2.15.1 and I'm trying to make it talk to cilium that uses gobgp under the hood. I can negotiate the session over ipv4 with no issues, but due to a bug in the cilium implementation, the ipv6 routes aren't advertised properly. The suggestion from cilium was to negotiate the session over ipv6, but this creates issues for bird.
I have a config that looks somewhat like this:
protocol bgp { local port 179 as 65100; neighbor range 0::/0 port 179 as 65100 internal;
dynamic name "GW6_"; direct; passive;
ipv4 { import all; };
ipv6 { import all; }; }
protocol bgp { local port 179 as 65100; neighbor range 10.0.0.0/8 port 179 as 65100 internal;
dynamic name "GW4_"; direct; passive;
ipv4 { import all; };
ipv6 { import all; }; }
When I connect to bird over ipv4, gobgp sends a packet like this:
Frame 4: 154 bytes on wire (1232 bits), 154 bytes captured (1232 bits) on interface vmbrdev, id 0 Ethernet II, Src: 22:fc:7f:47:6b:f4 (22:fc:7f:47:6b:f4), Dst: 6e:82:6c:1b:e9:95 (6e:82:6c:1b:e9:95) Internet Protocol Version 4, Src: 10.224.129.15, Dst: 10.224.129.1 Transmission Control Protocol, Src Port: 39201, Dst Port: 179, Seq: 1, Ack: 1, Len: 88 Border Gateway Protocol - OPEN Message Marker: ffffffffffffffffffffffffffffffff Length: 88 Type: OPEN Message (1) Version: 4 My AS: 65100 Hold Time: 90 BGP Identifier: 10.224.129.15 Optional Parameters Length: 59 Optional Parameters Optional Parameter: Capability Parameter Type: Capability (2) Parameter Length: 57 Capability: Route refresh capability Type: Route refresh capability (2) Length: 0 Capability: FQDN Capability Type: FQDN Capability (73) Length: 12 Hostname Length: 10 Hostname: dev-master Domain Name Length: 0 Domain Name: Capability: Unknown capability 75 Type: Unknown (75) Length: 13 Unknown: 0c476f4247502f332e32332e30 Capability: Multiprotocol extensions capability Type: Multiprotocol extensions capability (1) Length: 4 AFI: IPv4 (1) Reserved: 00 SAFI: Unicast (1) Capability: Multiprotocol extensions capability Type: Multiprotocol extensions capability (1) Length: 4 AFI: IPv6 (2) Reserved: 00 SAFI: Unicast (1) Capability: Support for 4-octet AS number capability Type: Support for 4-octet AS number capability (65) Length: 4 AS Number: 65100 Capability: Extended Next Hop Encoding Type: Extended Next Hop Encoding (5) Length: 6 AFI: IPv4 (1) SAFI: Unicast (1) Next hop AFI: IPv6 (2)
Which is followed up by OPEN from bird2 and they establish the session. The logs say:
May 04 14:47:02 ammy bird[7755]: bgp2: Incoming connection from 10.224.129.15 (port 39201) accepted May 04 14:47:02 ammy bird[7755]: GW4_1: Initializing May 04 14:47:02 ammy bird[7755]: GW4_1: Starting May 04 14:47:02 ammy bird[7755]: GW4_1: State changed to start May 04 14:47:02 ammy bird[7755]: GW4_1: Started May 04 14:47:02 ammy bird[7755]: GW4_1: Got OPEN(as=65100,hold=90,id=10.224.129.15) May 04 14:47:02 ammy bird[7755]: GW4_1: Sending OPEN(ver=4,as=65100,hold=240,id=0ae08101) May 04 14:47:02 ammy bird[7755]: GW4_1: Sending KEEPALIVE May 04 14:47:02 ammy bird[7755]: GW4_1: Got KEEPALIVE May 04 14:47:02 ammy bird[7755]: GW4_1: BGP session established
When I try to establish the connection over ipv6, gobgp sends an OPEN that's exactly the same sans being transferred over ipv6:
Frame 15: 174 bytes on wire (1392 bits), 174 bytes captured (1392 bits) on interface vmbrdev, id 0 Ethernet II, Src: 22:fc:7f:47:6b:f4 (22:fc:7f:47:6b:f4), Dst: 6e:82:6c:1b:e9:95 (6e:82:6c:1b:e9:95) Internet Protocol Version 6, Src: 2a01:xxx:xxx:xxx:xxx:xxx:xxx:d930, Dst: fd00::53 Transmission Control Protocol, Src Port: 58663, Dst Port: 179, Seq: 1, Ack: 1, Len: 88
and then bird just stalls the connection. The logs say:
May 04 14:49:17 ammy bird[7755]: bgp1: Incoming connection from 2a01:4f8:173:fe11:6f81:e7cd:2787:fb71 (port 35581) accepted May 04 14:49:17 ammy bird[7755]: GW6_1: Initializing May 04 14:49:17 ammy bird[7755]: GW6_1: Starting May 04 14:49:17 ammy bird[7755]: GW6_1: State changed to start May 04 14:49:17 ammy bird[7755]: GW6_1: Waiting for 2a01:xxx:xxx:xxx:xxx:xxx:xxx:d930 to become my neighbor
I suspect it's either a bug or a misconfiguration on the bird side, because gobgp behaves the same in both scenarios. How do I investigate/debug this further?
-- Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o.
Then i was totally wrong with that, sorry. Now i checked once again the log, and there are actually two different neighbor addresses, one ending :fb71, another :d930. This may be some discrepancy in your ipv6 settings. The error message actually means "you requested direct connection but i can't see the right interface to use because there is no interface with this range assigned". Maria On 4 May 2024 17:59:21 CEST, Vladimir Pouzanov <farcaller@gmail.com> wrote:
I actually started with my /64 and then expanded to ::/0 to the same result.
On Sat 4 May 2024 at 16:54, Maria Matejka <maria.matejka@nic.cz> wrote:
Not at my computer, just a wild guess – would you try please, instead of ::/0, narrowing the range, e.g. like 2000::/3? I haven't seen the code recently but this may help.
Maria
On 4 May 2024 17:16:21 CEST, Vladimir Pouzanov <farcaller@gmail.com> wrote:
Hi!
I'm running bird 2.15.1 and I'm trying to make it talk to cilium that uses gobgp under the hood. I can negotiate the session over ipv4 with no issues, but due to a bug in the cilium implementation, the ipv6 routes aren't advertised properly. The suggestion from cilium was to negotiate the session over ipv6, but this creates issues for bird.
I have a config that looks somewhat like this:
protocol bgp { local port 179 as 65100; neighbor range 0::/0 port 179 as 65100 internal;
dynamic name "GW6_"; direct; passive;
ipv4 { import all; };
ipv6 { import all; }; }
protocol bgp { local port 179 as 65100; neighbor range 10.0.0.0/8 port 179 as 65100 internal;
dynamic name "GW4_"; direct; passive;
ipv4 { import all; };
ipv6 { import all; }; }
When I connect to bird over ipv4, gobgp sends a packet like this:
Frame 4: 154 bytes on wire (1232 bits), 154 bytes captured (1232 bits) on interface vmbrdev, id 0 Ethernet II, Src: 22:fc:7f:47:6b:f4 (22:fc:7f:47:6b:f4), Dst: 6e:82:6c:1b:e9:95 (6e:82:6c:1b:e9:95) Internet Protocol Version 4, Src: 10.224.129.15, Dst: 10.224.129.1 Transmission Control Protocol, Src Port: 39201, Dst Port: 179, Seq: 1, Ack: 1, Len: 88 Border Gateway Protocol - OPEN Message Marker: ffffffffffffffffffffffffffffffff Length: 88 Type: OPEN Message (1) Version: 4 My AS: 65100 Hold Time: 90 BGP Identifier: 10.224.129.15 Optional Parameters Length: 59 Optional Parameters Optional Parameter: Capability Parameter Type: Capability (2) Parameter Length: 57 Capability: Route refresh capability Type: Route refresh capability (2) Length: 0 Capability: FQDN Capability Type: FQDN Capability (73) Length: 12 Hostname Length: 10 Hostname: dev-master Domain Name Length: 0 Domain Name: Capability: Unknown capability 75 Type: Unknown (75) Length: 13 Unknown: 0c476f4247502f332e32332e30 Capability: Multiprotocol extensions capability Type: Multiprotocol extensions capability (1) Length: 4 AFI: IPv4 (1) Reserved: 00 SAFI: Unicast (1) Capability: Multiprotocol extensions capability Type: Multiprotocol extensions capability (1) Length: 4 AFI: IPv6 (2) Reserved: 00 SAFI: Unicast (1) Capability: Support for 4-octet AS number capability Type: Support for 4-octet AS number capability (65) Length: 4 AS Number: 65100 Capability: Extended Next Hop Encoding Type: Extended Next Hop Encoding (5) Length: 6 AFI: IPv4 (1) SAFI: Unicast (1) Next hop AFI: IPv6 (2)
Which is followed up by OPEN from bird2 and they establish the session. The logs say:
May 04 14:47:02 ammy bird[7755]: bgp2: Incoming connection from 10.224.129.15 (port 39201) accepted May 04 14:47:02 ammy bird[7755]: GW4_1: Initializing May 04 14:47:02 ammy bird[7755]: GW4_1: Starting May 04 14:47:02 ammy bird[7755]: GW4_1: State changed to start May 04 14:47:02 ammy bird[7755]: GW4_1: Started May 04 14:47:02 ammy bird[7755]: GW4_1: Got OPEN(as=65100,hold=90,id=10.224.129.15) May 04 14:47:02 ammy bird[7755]: GW4_1: Sending OPEN(ver=4,as=65100,hold=240,id=0ae08101) May 04 14:47:02 ammy bird[7755]: GW4_1: Sending KEEPALIVE May 04 14:47:02 ammy bird[7755]: GW4_1: Got KEEPALIVE May 04 14:47:02 ammy bird[7755]: GW4_1: BGP session established
When I try to establish the connection over ipv6, gobgp sends an OPEN that's exactly the same sans being transferred over ipv6:
Frame 15: 174 bytes on wire (1392 bits), 174 bytes captured (1392 bits) on interface vmbrdev, id 0 Ethernet II, Src: 22:fc:7f:47:6b:f4 (22:fc:7f:47:6b:f4), Dst: 6e:82:6c:1b:e9:95 (6e:82:6c:1b:e9:95) Internet Protocol Version 6, Src: 2a01:xxx:xxx:xxx:xxx:xxx:xxx:d930, Dst: fd00::53 Transmission Control Protocol, Src Port: 58663, Dst Port: 179, Seq: 1, Ack: 1, Len: 88
and then bird just stalls the connection. The logs say:
May 04 14:49:17 ammy bird[7755]: bgp1: Incoming connection from 2a01:4f8:173:fe11:6f81:e7cd:2787:fb71 (port 35581) accepted May 04 14:49:17 ammy bird[7755]: GW6_1: Initializing May 04 14:49:17 ammy bird[7755]: GW6_1: Starting May 04 14:49:17 ammy bird[7755]: GW6_1: State changed to start May 04 14:49:17 ammy bird[7755]: GW6_1: Waiting for 2a01:xxx:xxx:xxx:xxx:xxx:xxx:d930 to become my neighbor
I suspect it's either a bug or a misconfiguration on the bird side, because gobgp behaves the same in both scenarios. How do I investigate/debug this further?
-- Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o.
-- Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o.
On Sun, May 5, 2024 at 7:35 AM Maria Matejka <maria.matejka@nic.cz> wrote:
The error message actually means "you requested direct connection but i can't see the right interface to use because there is no interface with this range assigned".
Thanks, that was it! I must have missed the note in the docs on that, because with ipv4 I just had the same subnet, but with ipv6 I don't have a public address assigned on the bird side. `multihop 1` Fixed the negotiation issue. I got a few more, but those seem to be related to either gobgp sending malformed updates (Invalid NEXT_HOP attribute - mismatched address family (2a01:xx:yy:zz:4746:92e9:327c:d930 for ipv4)), or the bird configuration not being able to parse a `-` in the interface name. Both unrelated to this particular issue at hand.
You may wish to have v6 nexthops for v4 prefixes though – to enable these, set "extended next hop" in the BGP channel config. Maria On 5 May 2024 09:45:32 CEST, Vladimir Pouzanov <farcaller@gmail.com> wrote:
On Sun, May 5, 2024 at 7:35 AM Maria Matejka <maria.matejka@nic.cz> wrote:
The error message actually means "you requested direct connection but i can't see the right interface to use because there is no interface with this range assigned".
Thanks, that was it!
I must have missed the note in the docs on that, because with ipv4 I just had the same subnet, but with ipv6 I don't have a public address assigned on the bird side. `multihop 1` Fixed the negotiation issue. I got a few more, but those seem to be related to either gobgp sending malformed updates (Invalid NEXT_HOP attribute - mismatched address family (2a01:xx:yy:zz:4746:92e9:327c:d930 for ipv4)), or the bird configuration not being able to parse a `-` in the interface name. Both unrelated to this particular issue at hand.
-- Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o.
Vladimir, I love the direction you are going with cilium and bird. We actually had cilium on our list for potential CNIs, but hold yourself: had to drop it, because cilium does NOT work on IPv6 only hosts! I.e. if the k8s workers are IPv6 only, cilium crashes [0]. In case you make any progress in that direction, I'd love to hear from you. BR from Switzerland, Nico [0] https://redmine.ungleich.ch/projects/open-infrastructure/wiki/The_ungleich_k... Vladimir Pouzanov <farcaller@gmail.com> writes:
On Sun, May 5, 2024 at 7:35 AM Maria Matejka <maria.matejka@nic.cz> wrote:
The error message actually means "you requested direct connection but i can't see the right interface to use because there is no interface with this range assigned".
Thanks, that was it!
I must have missed the note in the docs on that, because with ipv4 I just had the same subnet, but with ipv6 I don't have a public address assigned on the bird side. `multihop 1` Fixed the negotiation issue. I got a few more, but those seem to be related to either gobgp sending malformed updates (Invalid NEXT_HOP attribute - mismatched address family (2a01:xx:yy:zz:4746:92e9:327c:d930 for ipv4)), or the bird configuration not being able to parse a `-` in the interface name. Both unrelated to this particular issue at hand.
-- Sustainable and modern Infrastructures by ungleich.ch
As far as I know, the recent cilium releases are capable of running ipv6 only (although I didn't try it personally, yet). BGP-wise, the only change you should need to make is to specify the router id: https://docs.cilium.io/en/latest/network/bgp-control-plane/#specifying-route... . As for my case, I figured out the basic configuration that makes cilium push both ipv4 and ipv6 routes and it's actually working as intended. Here's a post with my notes https://farcaller.net/2024/making-cilium-bgp-work-with-ipv6/, but really, all the struggles I had were to do with my misunderstanding how exactly bird operates. On Mon, May 6, 2024 at 8:43 AM Nico Schottelius < nico.schottelius@ungleich.ch> wrote:
Vladimir,
I love the direction you are going with cilium and bird. We actually had cilium on our list for potential CNIs, but hold yourself: had to drop it, because cilium does NOT work on IPv6 only hosts!
I.e. if the k8s workers are IPv6 only, cilium crashes [0].
In case you make any progress in that direction, I'd love to hear from you.
BR from Switzerland,
Nico
[0] https://redmine.ungleich.ch/projects/open-infrastructure/wiki/The_ungleich_k...
Vladimir Pouzanov <farcaller@gmail.com> writes:
On Sun, May 5, 2024 at 7:35 AM Maria Matejka <maria.matejka@nic.cz> wrote:
The error message actually means "you requested direct connection but i can't see the right interface to use because there is no interface with this range assigned".
Thanks, that was it!
I must have missed the note in the docs on that, because with ipv4 I just had the same subnet, but with ipv6 I don't have a public address assigned on the bird side. `multihop 1` Fixed the negotiation issue. I got a few more, but those seem to be related to either gobgp sending malformed updates (Invalid NEXT_HOP attribute - mismatched address family (2a01:xx:yy:zz:4746:92e9:327c:d930 for ipv4)), or the bird configuration not being able to parse a `-` in the interface name. Both unrelated to this particular issue at hand.
-- Sustainable and modern Infrastructures by ungleich.ch
-- Sincerely, Vladimir "Farcaller" Pouzanov http://farcaller.net/
participants (3)
-
Maria Matejka -
Nico Schottelius -
Vladimir Pouzanov