I?m trying to use a bonded interface on linux to connect to two routers, one router on each physical link, each with a /31 subnet. Only one of the routers (Force10 S4810) forms adjacency with the linux host (whichever comes first), the other gets stuck in EXSTART until I shut/no shut the link, then Bird creates adjacency with both routers.

What are you trying to do with this design ? It's rather strange. -- Raphael Mazelier

Hello, you cannot use LACP between 3 devices. That is only possible if two of those devices (Force10 routers/switches) are forming one logical device (Cisco VSS, MEC, virtual PortChannel, HP IRF), I don't know if Force10 has something like that. If you do this however, those 2 routers will appear as one logical device (one OSPF neighbor) to the server, then you don't have a problem. This is preferred solution, because it takes the problem from OSPF to much faster technologies. On the other hand you can do VRRP between the routers and do OSPF on the hypervisor with both of the routers, in this case just beware the asymetric routing (which may/may not be a problem, depending on the setup). Regarding your setup, I assume you're using the same IP on both of the routers, this won't work because from the router perspective the links are UP and they're advertising the same /31 to the rest of the network, this will cause half of the packets/flows to be lost. So, you can either use some virtualization switching technology (if Force10 provides that), or you can use VRRP with 2 OSPF neighborships (but in that case you need /29 subnet), or you can do some sort of script on the server and use master-slave bonding mode, but be sure to always shutdown the inactive interface (be sure to always have enabled only one of them physically), that way only one of the Force10 routers would advertise the subnet... Martin 2013-12-16 10:24 odosielateľ napísal:

Yes - the reason is that this router is a VM with two passthrough NICs. The hypervisor is connected to both Force10 routers/switches with LACP, so the VM needs to run linux bonding mode 2 to provide a bond0 interface to the VM. Neighbourship then needs to be established to both routers on this bonded interface.

I tried to create neighbourship directly on the interfaces, but this does not work, I assume because the switches loadbalance traffic on the LACP portchannel.

I could create a neighbourship with a VRRP interface, but as I understand it this will not work due to different router-ids in case of failover.

So basically as I see it, this is the only way to make this work - unless you have another idea?

Thanks

Regards Kristoffer

On 13/12/2013, at 17.37.29, Raphael Mazelier <raph@futomaki.net> wrote:

...

...

I?m trying to use a bonded interface on linux to connect to two routers, one router on each physical link, each with a /31 subnet. Only one of the routers (Force10 S4810) forms adjacency with the linux host (whichever comes first), the other gets stuck in EXSTART until I shut/no shut the link, then Bird creates adjacency with both routers.

What are you trying to do with this design ? It's rather strange.

-- Raphael Mazelier

Hi, Thanks for your input - the Force10 switches have a VLT function (almost the same as cisco vPC) which makes it possible to run LACP against two non-stacked switches. But this VLT stuff is only for L2, not L3, which means that basically servers see the switches as one device through L2, but for routing protocols and other L3 stuff they are two seperate devices... I got it working by disabling peer-routing as written earlier, I’ll test some more. Regards Kristoffer On 16/12/2013, at 10.49.13, Kveri <kveri@kveri.com> wrote:

Hello,

you cannot use LACP between 3 devices. That is only possible if two of those devices (Force10 routers/switches) are forming one logical device (Cisco VSS, MEC, virtual PortChannel, HP IRF), I don't know if Force10 has something like that. If you do this however, those 2 routers will appear as one logical device (one OSPF neighbor) to the server, then you don't have a problem. This is preferred solution, because it takes the problem from OSPF to much faster technologies.

On the other hand you can do VRRP between the routers and do OSPF on the hypervisor with both of the routers, in this case just beware the asymetric routing (which may/may not be a problem, depending on the setup).

Regarding your setup, I assume you're using the same IP on both of the routers, this won't work because from the router perspective the links are UP and they're advertising the same /31 to the rest of the network, this will cause half of the packets/flows to be lost.

So, you can either use some virtualization switching technology (if Force10 provides that), or you can use VRRP with 2 OSPF neighborships (but in that case you need /29 subnet), or you can do some sort of script on the server and use master-slave bonding mode, but be sure to always shutdown the inactive interface (be sure to always have enabled only one of them physically), that way only one of the Force10 routers would advertise the subnet...

Martin

2013-12-16 10:24 odosielateľ napísal:

...

Yes - the reason is that this router is a VM with two passthrough NICs. The hypervisor is connected to both Force10 routers/switches with LACP, so the VM needs to run linux bonding mode 2 to provide a bond0 interface to the VM. Neighbourship then needs to be established to both routers on this bonded interface. I tried to create neighbourship directly on the interfaces, but this does not work, I assume because the switches loadbalance traffic on the LACP portchannel. I could create a neighbourship with a VRRP interface, but as I understand it this will not work due to different router-ids in case of failover. So basically as I see it, this is the only way to make this work - unless you have another idea? Thanks Regards Kristoffer On 13/12/2013, at 17.37.29, Raphael Mazelier <raph@futomaki.net> wrote:

...

...

I?m trying to use a bonded interface on linux to connect to two routers, one router on each physical link, each with a /31 subnet. Only one of the routers (Force10 S4810) forms adjacency with the linux host (whichever comes first), the other gets stuck in EXSTART until I shut/no shut the link, then Bird creates adjacency with both routers. What are you trying to do with this design ? It's rather strange. -- Raphael Mazelier

The Force10 manual states about peer-routing: VLT unicast routing locally routes packets destined for the L3 endpoint of the VLT peer. So if this means that if LSAs for router1 are sent down the link in the port channel connecting to router2, then router2 will respond instead of sending the traffic to router1 - I’m not sure, but this seems plausible as to why it does not work with the peer routing feature. Funny actually that it works with quagga then… Anyway, it seems to be stable now - let me know if you would like the tcpdump anyway. Thanks for your help! Regards Kristoffer On 16/12/2013, at 11.07.50, Kristoffer Egefelt <kristoffer@itoc.dk> wrote:

It seems this is working if I disable the peer-routing feature on the Force10 routers - I’ll test a little more and get back with a tcpdump

Thanks.

On 13/12/2013, at 18.01.10, Ondrej Zajicek <santiago@crfreenet.org> wrote:

...

On Fri, Dec 13, 2013 at 04:47:17PM +0100, Kristoffer Egefelt wrote:

...

Hi,

Is this not supported?

This is supported on Linux.

...

I?m trying to use a bonded interface on linux to connect to two routers, one router on each physical link, each with a /31 subnet. Only one of the routers (Force10 S4810) forms adjacency with the linux host (whichever comes first), the other gets stuck in EXSTART until I shut/no shut the link, then Bird creates adjacency with both routers.

That is even more strange. It would be useful if you could make verbose tcpdump log (tcpdump -i ethX -vv -s 0) together with BIRD OSPF log ('debug all' for OSPF) and send it to me.

-- Elen sila lumenn' omentielvo

Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."