Several ptmp OSPF sessions over a single interface
Hello, I'm currently performing some experiments to find the "best" way to inter-connect a bunch of routers using a shared medium. A common scenario would be a set of 4 routers, for instance a pair of border routers and a pair of core routers. I'd like either a full-mesh or circle topology. Putting all nodes in a /29 isn't ideal because of the DR/BDR/etc stuff. I want all nodes and links to be treated equally. Using point-to-multipoint sounds like a decent solution. I can configure several sessions on a single link without interference, up to the point where the circle is created. Like this: A - B | | D - C However, if I try to add a link between A and C, things start to go not as expected. A already has 192.168.255.2/31 (towards B) and 192.168.255.4/31 (towards D) configured. If I add 192.168.255.10/31 (towards C) and 192.168.255.11/31 on C (towards A), A and C can't ping each other using those addresses. This is because A will announce 192.168.255.10/32 throughout the area and thus "overriding" the 192.168.255.10/31 route on C (traffic will would be sent to either B or D, and not directly to A). What would be the cleanest solution for this? Regards, Ruben Laban
On 8/27/2012 1:58 PM, Ruben Laban wrote:
Hello,
I'm currently performing some experiments to find the "best" way to inter-connect a bunch of routers using a shared medium. A common scenario would be a set of 4 routers, for instance a pair of border routers and a pair of core routers. I'd like either a full-mesh or circle topology. Putting all nodes in a /29 isn't ideal because of the DR/BDR/etc stuff. I want all nodes and links to be treated equally. Using point-to-multipoint sounds like a decent solution. I can configure several sessions on a single link without interference, up to the point where the circle is created. Like this:
A - B | | D - C
However, if I try to add a link between A and C, things start to go not as expected. A already has 192.168.255.2/31 (towards B) and 192.168.255.4/31 (towards D) configured. If I add 192.168.255.10/31 (towards C) and 192.168.255.11/31 on C (towards A), A and C can't ping each other using those addresses. This is because A will announce 192.168.255.10/32 throughout the area and thus "overriding" the 192.168.255.10/31 route on C (traffic will would be sent to either B or D, and not directly to A).
What would be the cleanest solution for this?
FTR, the solution I'm currently testing with is using: ip address add 192.168.255.10 peer 192.168.255.11 dev eth1 instead of: ip address add 192.168.255.10/31 dev eth1 Seems to work fine so far, but suggestions/comments/etc are still more than welcome. Regards, Ruben
On 8/27/2012 2:10 PM, Ruben Laban wrote:
On 8/27/2012 1:58 PM, Ruben Laban wrote:
Hello,
I'm currently performing some experiments to find the "best" way to inter-connect a bunch of routers using a shared medium. A common scenario would be a set of 4 routers, for instance a pair of border routers and a pair of core routers. I'd like either a full-mesh or circle topology. Putting all nodes in a /29 isn't ideal because of the DR/BDR/etc stuff. I want all nodes and links to be treated equally. Using point-to-multipoint sounds like a decent solution. I can configure several sessions on a single link without interference, up to the point where the circle is created. Like this:
A - B | | D - C
However, if I try to add a link between A and C, things start to go not as expected. A already has 192.168.255.2/31 (towards B) and 192.168.255.4/31 (towards D) configured. If I add 192.168.255.10/31 (towards C) and 192.168.255.11/31 on C (towards A), A and C can't ping each other using those addresses. This is because A will announce 192.168.255.10/32 throughout the area and thus "overriding" the 192.168.255.10/31 route on C (traffic will would be sent to either B or D, and not directly to A).
What would be the cleanest solution for this?
FTR, the solution I'm currently testing with is using:
ip address add 192.168.255.10 peer 192.168.255.11 dev eth1
instead of:
ip address add 192.168.255.10/31 dev eth1
Seems to work fine so far, but suggestions/comments/etc are still more than welcome.
Forgot to mention something in my previous follow-up: the downside of the alternative mentioned above is that now 192.168.255.10 and 192.168.255.11 don't show up in the routing tables of the other nodes. Which, without applying other "fixes" would break traffic originating from those nodes using those IP addresses, including for instance traceroutes. Regards, Ruben Laban
On Mon, Aug 27, 2012 at 01:58:27PM +0200, Ruben Laban wrote:
Hello,
I'm currently performing some experiments to find the "best" way to inter-connect a bunch of routers using a shared medium. A common scenario would be a set of 4 routers, for instance a pair of border routers and a pair of core routers. I'd like either a full-mesh or circle topology. Putting all nodes in a /29 isn't ideal because of the DR/BDR/etc stuff. I want all nodes and links to be treated equally. Using point-to-multipoint sounds like a decent solution. I can configure several sessions on a single link without interference, up to the point where the circle is created. Like this:
First note that (our implementation of) OSPFv2 (on IPv4) is not based on physical interfaces, but on 'logical ifaces' based on network prefixes/addresses. So if your iface has two independent address prefixes or ptp address pairs, it is handled as two ifaces. Conclusion is that if you use /31 or ptp addresses there is no reason to use ptmp mode, you can use ptp mode. Another note is that whether DR/BDR is used does not depend on whether /29 is used, but on 'iface type', which is by default 'bcast' that chooses DR, but you can switch it to 'ptmp' which will give you probably what you want (but as ptmp mode does not have peer autodetection, you have to configure).
A - B | | D - C
I see three possibilities: (1) ptp links using /31 addresses for each link (2) ptp links using ptp addresses for each link (3) one /29 address and ptmp mode (4) one /29 address and ptp (!!) mode (1) Generally should work, you should probably propagate another /32 as loopback address. (2) generally should work and you can use half the addresses than in the first case as you can share local address for each ptp pair, so you can use just four addresses. This setting has one problem - if for example link between A and B (with addresses assigned like 'ip addr add 192.168.1.1 peer 192.168.1.2' on A) fails, then communication from A to IP 192.168.1.2 would always go through that link because there is 192.168.1.2/32 kernel 'neighbor' route in routing table which currently cannot be overridden by BIRD OSPF routes. Workaround for this is to propagate another /32 as loopback address on each router (which will be routed as usuall). (3) this should work OK, but you have to explicitly configure your neighbors using 'neighbors' on each router, because PTMP is generally assumed to be run on non-broadcast media, so there is no autodetection of peers. Your IP is automatically propagated as /32. (4) this is a bit nonstandard hack based on fact that PTP and PTMP implementation is almost identical so we allow more neighbors even on PTP ifaces. There are two small differences compared to (3) - PTP uses multicast autodetection so no need to configure peers, but you have to explicitly announce that IP as /32 using 'stubnet' (PTP do not automatically announce its IP in BIRD). So the conclusion is to use either (1) or (2), but consider these IP addresses used for links as 'link-local' and propagate different /32 loopback address, or use (3) if you want precise control which neighbor adjacencies are established, or use (4) if you want automatic establishment of all adjacencies and are not afraid of non-standard hacks. I should probably write some thorough explanation of all OSPF interface modes in BIRD. -- Elen sila lumenn' omentielvo Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
On 8/27/2012 4:49 PM, Ondrej Zajicek wrote:
On Mon, Aug 27, 2012 at 01:58:27PM +0200, Ruben Laban wrote:
Hello,
I'm currently performing some experiments to find the "best" way to inter-connect a bunch of routers using a shared medium. A common scenario would be a set of 4 routers, for instance a pair of border routers and a pair of core routers. I'd like either a full-mesh or circle topology. Putting all nodes in a /29 isn't ideal because of the DR/BDR/etc stuff. I want all nodes and links to be treated equally. Using point-to-multipoint sounds like a decent solution. I can configure several sessions on a single link without interference, up to the point where the circle is created. Like this:
First note that (our implementation of) OSPFv2 (on IPv4) is not based on physical interfaces, but on 'logical ifaces' based on network prefixes/addresses. So if your iface has two independent address prefixes or ptp address pairs, it is handled as two ifaces.
Conclusion is that if you use /31 or ptp addresses there is no reason to use ptmp mode, you can use ptp mode.
Another note is that whether DR/BDR is used does not depend on whether /29 is used, but on 'iface type', which is by default 'bcast' that chooses DR, but you can switch it to 'ptmp' which will give you probably what you want (but as ptmp mode does not have peer autodetection, you have to configure).
A - B | | D - C
I see three possibilities:
(1) ptp links using /31 addresses for each link
(2) ptp links using ptp addresses for each link
(3) one /29 address and ptmp mode
(4) one /29 address and ptp (!!) mode
(1) Generally should work, you should probably propagate another /32 as loopback address.
(2) generally should work and you can use half the addresses than in the first case as you can share local address for each ptp pair, so you can use just four addresses. This setting has one problem - if for example link between A and B (with addresses assigned like 'ip addr add 192.168.1.1 peer 192.168.1.2' on A) fails, then communication from A to IP 192.168.1.2 would always go through that link because there is 192.168.1.2/32 kernel 'neighbor' route in routing table which currently cannot be overridden by BIRD OSPF routes. Workaround for this is to propagate another /32 as loopback address on each router (which will be routed as usuall).
(3) this should work OK, but you have to explicitly configure your neighbors using 'neighbors' on each router, because PTMP is generally assumed to be run on non-broadcast media, so there is no autodetection of peers. Your IP is automatically propagated as /32.
(4) this is a bit nonstandard hack based on fact that PTP and PTMP implementation is almost identical so we allow more neighbors even on PTP ifaces. There are two small differences compared to (3) - PTP uses multicast autodetection so no need to configure peers, but you have to explicitly announce that IP as /32 using 'stubnet' (PTP do not automatically announce its IP in BIRD).
So the conclusion is to use either (1) or (2), but consider these IP addresses used for links as 'link-local' and propagate different /32 loopback address, or use (3) if you want precise control which neighbor adjacencies are established, or use (4) if you want automatic establishment of all adjacencies and are not afraid of non-standard hacks.
I should probably write some thorough explanation of all OSPF interface modes in BIRD.
Thank you Ondrej, for these pointers. Without having done any additional testing (yet), (3) seems to be pretty close to what I'd want (having to specify all neighbors is both a pro and a con I guess). So I'll be sure to do some experimenting with that tomorrow. Some detailed documentation regarding the various modes would surely be appreciated indeed ;-) Regards, Ruben Laban
Hi Ondrej, list, On 8/27/2012 4:38 PM, Ruben Laban wrote:
On 8/27/2012 4:49 PM, Ondrej Zajicek wrote:
On Mon, Aug 27, 2012 at 01:58:27PM +0200, Ruben Laban wrote: (3) this should work OK, but you have to explicitly configure your neighbors using 'neighbors' on each router, because PTMP is generally assumed to be run on non-broadcast media, so there is no autodetection of peers. Your IP is automatically propagated as /32.
Thank you Ondrej, for these pointers. Without having done any additional testing (yet), (3) seems to be pretty close to what I'd want (having to specify all neighbors is both a pro and a con I guess). So I'll be sure to do some experimenting with that tomorrow.
Some detailed documentation regarding the various modes would surely be appreciated indeed ;-)
So far the testing with (3) has shown it to be a rather nice solution problem. It even seems to handle the additional use case where there's floating (high-available) IP addresses involved (as aliases) properly. Just one small question though: Is there some trick to "AdminDown" a session between 2 given peers in a pmtp setup, other than invoking packetfilters and have the sessions time out? Removing the "neighbor" from either end and doing a "configure" keeps the session alive it seems. Regards, Ruben Laban
On Tue, Aug 28, 2012 at 12:09:32PM +0200, Ruben Laban wrote:
So far the testing with (3) has shown it to be a rather nice solution problem. It even seems to handle the additional use case where there's floating (high-available) IP addresses involved (as aliases) properly. Just one small question though: Is there some trick to "AdminDown" a session between 2 given peers in a pmtp setup, other than invoking packetfilters and have the sessions time out?
I don't think there is any.
Removing the "neighbor" from either end and doing a "configure" keeps the session alive it seems.
This is probably a bug. -- Elen sila lumenn' omentielvo Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
On 8/28/2012 11:39 PM, Ondrej Zajicek wrote:
On Tue, Aug 28, 2012 at 12:09:32PM +0200, Ruben Laban wrote:
So far the testing with (3) has shown it to be a rather nice solution problem. It even seems to handle the additional use case where there's floating (high-available) IP addresses involved (as aliases) properly. Just one small question though: Is there some trick to "AdminDown" a session between 2 given peers in a pmtp setup, other than invoking packetfilters and have the sessions time out?
I don't think there is any.
Ok.
Removing the "neighbor" from either end and doing a "configure" keeps the session alive it seems.
This is probably a bug.
What I deduced from my limited testing is that the "neighbors" setting determines which peers to actively "detect". Packets received from peer that are not explicitly mentioned in the config, but reachable network-wise, will be communicated with just fine and will keep doing so. It will even try to re-establish the session after the peer stops sending packets. This could very well be the intended behavior, it's just that it "screws" me when trying to do the "AdminDown" stuff ;-) I don't expect to be needing to "AdminDown" sessions very often, so the use packetfilters is a viable solution for now. Regards, Ruben Laban
On Mon, Aug 27, 2012 at 04:49:06PM +0200, Ondrej Zajicek wrote:
I see three possibilities:
(2) ptp links using ptp addresses for each link ... (2) generally should work and you can use half the addresses than in the first case as you can share local address for each ptp pair, so you can use just four addresses.
Hello I gave a misleading advice here - although it is OK to have several ptp links with the same local IP, it does not work properly if these links are just ptp address pairs on common shared media (ethernet). It is even explained in FAQ: https://git.nic.cz/redmine/projects/bird/wiki/FAQ -- Elen sila lumenn' omentielvo Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
participants (2)
-
Ondrej Zajicek -
Ruben Laban