Re: Multihop BGP cannot use direct gateway mode - misidentification of multihop sessions?
Any ideas on the below? Alex On 8 Aug 2014, at 20:05, Alex Bligh <alex@alex.org.uk> wrote:
When I start bird using 1.4.0-1 (Ubuntu Trusty) I see the error:
warnings.log:Aug 8 13:47:03 node-10-157-128-41 bird: /etc/bird/bird.conf, line 62: Multihop BGP cannot use direct gateway mode
This error did not occur on 1.3.4-1 with the same config on Ubuntu Precise.
The config I am using is pasted below, as is the output of 'ip addr show'.
Line 62 of the config is the closing bracket of a BGP protocol definition. The neighbour statement is on line 29: neighbor 10.157.192.1 as 65000
You will note that interface evrr-000000 has IP address 10.157.192.41/20, and the peer is IP address 10.157.192.1. This should thus not be a multihop session. The output of 'ip route show' (also below) confirms this.
It is an iBGP session (in that both ends have the same address) but I am deliberately hoping to use gateway direct, i.e. want to put the advertised next hop straight into the RIB without doing a recursive lookup. Specifically if something else on the same /20 gets advertised as a next hop, I still want to send the traffic to the advertising router.
The documentation says:
For received routes, their gw (immediate next hop) attribute is computed from received bgp_next_hopattribute. This option specifies how it is computed. Direct mode means that the IP address frombgp_next_hop is used if it is directly reachable, otherwise the neighbor IP address is used. Recursive mode means that the gateway is computed by an IGP routing table lookup for the IP address from bgp_next_hop. Recursive mode is the behavior specified by the BGP standard. Direct mode is simpler, does not require any routes in a routing table, and was used in older versions of BIRD, but does not handle well nontrivial iBGP setups and multihop. Recursive mode is incompatible with sorted tables. Default: direct for direct sessions, recursive for multihop sessions.
From this, gateway direct should be the default for a direct session in any case. It would appear to me that bird is somehow misidentifying the session as a multihop session.
Removing 'gateway direct' fixes the error (but obviously not the behaviour).
This is running in an network container if that's of any interest.
Any ideas?
(apologies for a potential dupe - I sent this to what I think must have been an old list email address)
-- Alex Bligh
root@node-10-157-128-41:/# ip addr show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: dummy2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 5a:17:eb:e4:7b:10 brd ff:ff:ff:ff:ff:ff inet6 fe80::5817:ebff:fee4:7b10/64 scope link valid_lft forever preferred_lft forever 3: dummy: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 12:7f:06:13:e8:91 brd ff:ff:ff:ff:ff:ff inet 10.99.99.1/24 scope global dummy valid_lft forever preferred_lft forever inet6 fe80::107f:6ff:fe13:e891/64 scope link valid_lft forever preferred_lft forever 12: evrr-000000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether ee:19:d3:6f:24:1b brd ff:ff:ff:ff:ff:ff inet 10.157.192.41/20 scope global evrr-000000 valid_lft forever preferred_lft forever inet6 fc00:41c8:10:1dd::29/64 scope global valid_lft forever preferred_lft forever inet6 fe80::ec19:d3ff:fe6f:241b/64 scope link valid_lft forever preferred_lft forever
root@node-10-157-128-41:/# ip route show 10.99.99.0/24 dev dummy proto kernel scope link src 10.99.99.1 10.157.192.0/20 dev evrr-000000 proto kernel scope link src 10.157.192.41
1 # Configure logging 2 log syslog { debug, trace, info, remote, warning, error, auth, fatal, bug }; 3 log stderr all; 4 5 router id 10.157.192.41; 6 7 # Import kernel routes. These come in with RTS_INHERIT 8 protocol kernel evrkernel { 9 persist; # Don't remove routes on bird shutdown 10 scan time 10; # Scan kernel routing table every 20 seconds 11 import all; 12 export all; 13 learn; # Learn all alien routes from the kernel 14 } 15 16 # This pseudo-protocol watches all interface up/down events. 17 protocol device { 18 scan time 10; # Scan interfaces every 10 seconds 19 } 20 21 # Import all directly connected routes. These come in with RTS_DEVICE 22 protocol direct evrdirect { 23 interface "-evrr-000000", "evrr-*"; 24 export all; 25 } 26 27 protocol bgp evrbgp1 { 28 local as 65000; 29 neighbor 10.157.192.1 as 65000; 30 31 gateway direct; 32 export filter { 33 # RTS_INHERIT means routers from the kernel 34 if net = 10.157.192.0/20 then reject; 35 if net = 0.0.0.0/8 then reject; 36 if (source = RTS_INHERIT) || (source = RTS_DEVICE) then { 37 # Add no-export 38 bgp_community.add((65535, 65281)); 39 # Add no-advertise 40 bgp_community.add((65535, 65282)); 41 42 bgp_local_pref = 100; 43 44 accept; 45 } 46 reject; 47 }; 48 import filter { 49 if net = 0.0.0.0/0 then accept; 50 reject; 51 }; 52 next hop self; 53 54 # timers 55 hold time 12; 56 startup hold time 12; 57 keepalive time 4; 58 connect retry time 6; 59 start delay time 1; 60 error wait time 2, 16; 61 error forget time 16; 62 } 63 64 protocol bgp evrbgp2 { 65 local as 65000; 66 neighbor 10.157.192.2 as 65000; 67 68 gateway direct; 69 export filter { 70 # RTS_INHERIT means routers from the kernel 71 if net = 10.157.192.0/20 then reject; 72 if net = 0.0.0.0/8 then reject; 73 if (source = RTS_INHERIT) || (source = RTS_DEVICE) then { 74 # Add no-export 75 bgp_community.add((65535, 65281)); 76 # Add no-advertise 77 bgp_community.add((65535, 65282)); 78 79 bgp_local_pref = 100; 80 81 accept; 82 } 83 reject; 84 }; 85 import filter { 86 if net = 0.0.0.0/0 then accept; 87 reject; 88 }; 89 next hop self; 90 91 # timers 92 hold time 12; 93 startup hold time 12; 94 keepalive time 4; 95 connect retry time 6; 96 start delay time 1; 97 error wait time 2, 16; 98 error forget time 16; 99 } 100
-- Alex Bligh
В письме от 12 августа 2014 20:27:17 пользователь Alex Bligh написал:
Any ideas on the below?
Alex
On 8 Aug 2014, at 20:05, Alex Bligh <alex@alex.org.uk> wrote:
When I start bird using 1.4.0-1 (Ubuntu Trusty) I see the error:
warnings.log:Aug 8 13:47:03 node-10-157-128-41 bird: /etc/bird/bird.conf, line 62: Multihop BGP cannot use direct gateway mode
Things get changed with commit f3e59178506dab9d54cf19ec701f8d9a7fe283f9 Author: Ondrej Zajicek <> Date: Sun Nov 24 12:37:24 2013 +0100 Enables multihop mode for IBGP by default. Now multihop enabled by default for iBGP sessions (which is your case), but can not be turned off with gateway direct so parse error is returned.
This error did not occur on 1.3.4-1 with the same config on Ubuntu Precise.
The config I am using is pasted below, as is the output of 'ip addr show'.
Line 62 of the config is the closing bracket of a BGP protocol definition. The neighbour statement is on line 29: neighbor 10.157.192.1 as 65000
You will note that interface evrr-000000 has IP address 10.157.192.41/20, and the peer is IP address 10.157.192.1. This should thus not be a multihop session. The output of 'ip route show' (also below) confirms this.
It is an iBGP session (in that both ends have the same address) but I am deliberately hoping to use gateway direct, i.e. want to put the advertised next hop straight into the RIB without doing a recursive lookup. Specifically if something else on the same /20 gets advertised as a next hop, I still want to send the traffic to the advertising router.>
You can use following filter code as workaround scenario in your import filter of bgp protocol(s) configuration to be sure that nh is from allowed 10.157.192.0/20: if !(bgp_next_hop ~ 10.157.192.0/20) then bgp_next_hop = from; If bgp_next_hop attribute points to IP not within allowed subnet, use neighbour (from) IP address as bgp_next_hop.
The documentation says:
For received routes, their gw (immediate next hop) attribute is computed from received bgp_next_hopattribute. This option specifies how it is computed. Direct mode means that the IP address frombgp_next_hop is used if it is directly reachable, otherwise the neighbor IP address is used. Recursive mode means that the gateway is computed by an IGP routing table lookup for the IP address from bgp_next_hop. Recursive mode is the behavior specified by the BGP standard. Direct mode is simpler, does not require any routes in a routing table, and was used in older versions of BIRD, but does not handle well nontrivial iBGP setups and multihop. Recursive mode is incompatible with sorted tables. Default: direct for direct sessions, recursive for multihop sessions.> From this, gateway direct should be the default for a direct session in any case. It would appear to me that bird is somehow misidentifying the session as a multihop session.
BIRD by default uses multihop for iBGP sessions since commit f3e59178506dab.
On Wed, Aug 13, 2014 at 10:52:49AM +0300, Sergey Popovich wrote:
commit f3e59178506dab9d54cf19ec701f8d9a7fe283f9 Author: Ondrej Zajicek <> Date: Sun Nov 24 12:37:24 2013 +0100
Enables multihop mode for IBGP by default.
Now multihop enabled by default for iBGP sessions (which is your case), but can not be turned off with gateway direct so parse error is returned.
Hi Multihop is now enabled by default for IBGP, but it *can* be turned off. See below.
From this, gateway direct should be the default for a direct session in any case. It would appear to me that bird is somehow misidentifying the session as a multihop session.
Whether the session is multihop or direct is not detected, but it is configured. In older versions, direct was the universal default. In the current one, direct is the default for EBGP and multihop is the default for IBGP.
BIRD by default uses multihop for iBGP sessions since commit f3e59178506dab.
From my point of view the core of the problem is that multi hop could not be turned off in iBGP configurations, but this is also true on othe platforms (cisco, juniper, ...).
Both IBGP and EBGP can be explicitly configured in BIRD to use either direct or multihop mode (options 'direct' and 'multihop'). The problem here is to mix it with 'gateway direct|recursive' option, which configures something related but slightly different - just the aspect of gateway resolution. You generally don't need to use this ('gateway ...') option as its default depend on direct/multihop session mode. Combination 'multihop' and 'gateway direct' is forbidden, so this option is useful mainly to set 'gateway recursive' together with 'direct' mode. In short, the default values depends in this direction: EBGP|IBGP -> direct|multihop -> gateway direct|recursive Changing former would change the default value for later, but not otherwise. -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
В письме от 13 августа 2014 11:03:21 пользователь Ondrej Zajicek написал:
On Wed, Aug 13, 2014 at 10:52:49AM +0300, Sergey Popovich wrote:
commit f3e59178506dab9d54cf19ec701f8d9a7fe283f9 Author: Ondrej Zajicek <> Date: Sun Nov 24 12:37:24 2013 +0100
Enables multihop mode for IBGP by default.
Now multihop enabled by default for iBGP sessions (which is your case), but can not be turned off with gateway direct so parse error is returned.
Hi
Multihop is now enabled by default for IBGP, but it *can* be turned off. See below.
From this, gateway direct should be the default for a direct session in any case. It would appear to me that bird is somehow misidentifying the session as a multihop session.
Whether the session is multihop or direct is not detected, but it is configured. In older versions, direct was the universal default. In the current one, direct is the default for EBGP and multihop is the default for IBGP.
BIRD by default uses multihop for iBGP sessions since commit f3e59178506dab.>
From my point of view the core of the problem is that multi hop could
not be turned off in iBGP configurations, but this is also true on othe platforms (cisco, juniper, ...).
Both IBGP and EBGP can be explicitly configured in BIRD to use either direct or multihop mode (options 'direct' and 'multihop').
The problem here is to mix it with 'gateway direct|recursive' option, which configures something related but slightly different - just the aspect of gateway resolution. You generally don't need to use this ('gateway ...') option as its default depend on direct/multihop session mode. Combination 'multihop' and 'gateway direct' is forbidden, so this option is useful mainly to set 'gateway recursive' together with 'direct' mode.
In short, the default values depends in this direction:
EBGP|IBGP -> direct|multihop -> gateway direct|recursive
Changing former would change the default value for later, but not otherwise.
Thanks for pointing Ondrej, so using 'direct' (not 'gateway direct') is opposite to the 'multihop' and protocol bgp bgp1 { ... direct; ... } is enought to turn on 'gateway direct' by default as described in docs (but could be turned into 'gateway recursive' on demand). Thanks again. -- SP5474-RIPE Sergey Popovich
Ondrej, On 13 Aug 2014, at 10:03, Ondrej Zajicek <santiago@crfreenet.org> wrote:
In short, the default values depends in this direction:
EBGP|IBGP -> direct|multihop -> gateway direct|recursive
Changing former would change the default value for later, but not otherwise.
Not sure I understand that. The session is IBGP. It is not multihop (by which I mean it's on the same subnet), but AFAICT bird thinks it is. How do I change it to a 'direct' (as opposed to multihop) session? I think you are saying it's defaulting to a multihop configuration (and thus 'gateway' defaults to 'recursive'), as opposed to it detecting a multihop session. Also, you cannot (it would seem) change to 'gateway direct' as you get the error. At the very least I think there is a documentation error as it says 'gateway direct' is the default. -- Alex Bligh
On Wed, Aug 13, 2014 at 10:27:01AM +0100, Alex Bligh wrote:
Ondrej,
On 13 Aug 2014, at 10:03, Ondrej Zajicek <santiago@crfreenet.org> wrote:
In short, the default values depends in this direction:
EBGP|IBGP -> direct|multihop -> gateway direct|recursive
Changing former would change the default value for later, but not otherwise.
Not sure I understand that.
The session is IBGP. It is not multihop (by which I mean it's on the same subnet), but AFAICT bird thinks it is.
Generally, by 'direct BGP session' or 'multihop BGP session' i mean the mode of operation of that BGP sesssion; i.e., whether it is expected to be on the same subnet, not whether it is really on the same subnet. The mode of operation cannot be reliably detected from the network topology, because the topology could change during the operation. E.g.: You have BGP neighbor 192.168.1.10 and two interfaces: eth0 with 192.168.1.1/24 and eth1 with 192.168.2.1/24. If the BGP session is in direct mode, it always uses eth0, packets are sent with TTL 1 and the session automatically shutdowns when eth0 is disabled. If the BGP session is in multihop mode, it reaches 192.168.1.10 based on IGP routes, which is probably a direct route through eth0, but could be rerouted by IGP (without dropout of the BGP session) through indirect hop over eth1 when eth0 is disabled.
How do I change it to a 'direct' (as opposed to multihop) session?
Use 'direct' option: <tag>direct</tag> Specify that the neighbor is directly connected. The IP address of the neighbor must be from a directly reachable IP range (i.e. associated with one of your router's interfaces), otherwise the BGP session wouldn't start but it would wait for such interface to appear. The alternative is the <cf/multihop/ option. Default: enabled for eBGP. I
think you are saying it's defaulting to a multihop configuration (and thus 'gateway' defaults to 'recursive'), as opposed to it detecting a multihop session.
Also, you cannot (it would seem) change to 'gateway direct' as you get the error.
At the very least I think there is a documentation error as it says 'gateway direct' is the default.
It only says that 'gateway direct' is the default for 'direct' sessions, which is true. -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
Ondrej, On 13 Aug 2014, at 10:57, Ondrej Zajicek <santiago@crfreenet.org> wrote:
How do I change it to a 'direct' (as opposed to multihop) session?
Use 'direct' option:
<tag>direct</tag> Specify that the neighbor is directly connected. The IP address of the neighbor must be from a directly reachable IP range (i.e. associated with one of your router's interfaces), otherwise the BGP session wouldn't start but it would wait for such interface to appear. The alternative is the <cf/multihop/ option. Default: enabled for eBGP.
Thanks -- Alex Bligh
participants (3)
-
Alex Bligh -
Ondrej Zajicek -
Sergey Popovich