What is the expected iBGP behaviour for routes with different local_pref?
Hello everyone, I was wondering how bird *should* behave if within one AS a route is received from outside multiple times but with different local_pref set? My expectation would be that using iBGP, the lower local_pref route would show up on all routers, but the higher one is selected. What I observe is that seems the router who initially receives the lower local_pref route does not export it via iBGP, even though it matches the export filter. Example: - server145 has a non-preferable uplink and imports all routes with local_pref = 80 - server137 has a preferable uplink and imports all routes without local_pref modification, thus local_pref = 100 - server145 has both routes - server137 only has the preferable route - It seems that server145 is not exporting its lower local_pref routes, even though they match the export filter Excerpt from server145, importing 2600:: route from outside, with setting local_pref to 80: -------------------------------------------------------------------------------- bird> show route all for 2600:: Table master6: 2600::/48 unicast [ibgp_server137 14:34:07.078 from 2a0a:e5c0:0:b::89] * (100/96) [AS1239i] via fe80::3eec:efff:fed2:d1e4 on eth0 Type: BGP univ BGP.origin: IGP BGP.as_path: 209898 39040 1299 174 1239 BGP.next_hop: 2a0a:e5c0:31:2::1 BGP.local_pref: 100 BGP.community: (1299,20000) (39040,100) (39040,109) (39040,1000) BGP.otc: 209898 unicast [outgoing_server120 14:33:56.123] (100) [AS1239i] via 2a0a:e5c0:31:4::1 on tun1 Type: BGP univ BGP.origin: IGP BGP.as_path: 209898 39040 1299 174 1239 BGP.next_hop: 2a0a:e5c0:31:4::1 fe80::897b:493f:bd44:ec60 BGP.local_pref: 80 BGP.community: (1299,20000) (39040,100) (39040,109) (39040,1000) BGP.otc: 209898 unicast [outgoing_server121 14:34:41.137] (100) [AS1239i] via 2a0a:e5c0:32:4::1 on tun0 Type: BGP univ BGP.origin: IGP BGP.as_path: 209898 15576 3356 174 1239 BGP.next_hop: 2a0a:e5c0:32:4::1 fe80::fa84:82d2:a117:43c BGP.local_pref: 80 BGP.community: (3356,2) (3356,22) (3356,86) (3356,505) (3356,601) (3356,666) (3356,903) (3356,2074) (15576,100) (15576,102) (15576,1000) BGP.otc: 209898 unicast [ibgp_server138 14:34:56.416 from 2a0a:e5c0:0:b::8a] (100/96) [AS1239i] via fe80::3eec:efff:fed2:d0c4 on eth0 Type: BGP univ BGP.origin: IGP BGP.as_path: 209898 15576 3356 174 1239 BGP.next_hop: 2a0a:e5c0:32:2::1 BGP.local_pref: 100 BGP.community: (3356,2) (3356,22) (3356,86) (3356,505) (3356,601) (3356,666) (3356,903) (3356,2074) (15576,100) (15576,102) (15576,1000) BGP.otc: 209898 -------------------------------------------------------------------------------- And on server137, this route is not visible, only the local_pref = 100 are visible: -------------------------------------------------------------------------------- bird> show route all for 2600:: Table master6: 2600::/48 unicast [outgoing_server120 14:41:16.420] * (100) [AS1239i] via 2a0a:e5c0:31:2::1 on oserver120 Type: BGP univ BGP.origin: IGP BGP.as_path: 209898 39040 1299 174 1239 BGP.next_hop: 2a0a:e5c0:31:2::1 BGP.local_pref: 100 BGP.community: (1299,20000) (39040,100) (39040,109) (39040,1000) BGP.otc: 209898 unicast [ibgp_server138 14:42:17.052 from 2a0a:e5c0:0:b::8a] (100/96) [AS1239i] via fe80::3eec:efff:fed2:d0c7 on eth5 Type: BGP univ BGP.origin: IGP BGP.as_path: 209898 15576 3356 174 1239 BGP.next_hop: 2a0a:e5c0:32:2::1 BGP.local_pref: 100 BGP.community: (3356,2) (3356,22) (3356,86) (3356,505) (3356,601) (3356,666) (3356,903) (3356,2074) (15576,100) (15576,102) (15576,1000) BGP.otc: 209898 -------------------------------------------------------------------------------- On the iBGP session we import/export everything that is either static or BGP,excerpt from server137: -------------------------------------------------------------------------------- Channel ipv6 State: UP Table: master6 Preference: 100 Input filter: ACCEPT Output filter: static_and_bgp Routes: 3 imported, 206002 exported, 2 preferred Route change stats: received rejected filtered ignored accepted Import updates: 211444 0 0 70 211374 Import withdraws: 230070 0 --- 24038 206032 Export updates: 645070 137427 98 --- 507545 Export withdraws: 64508 --- --- --- 96296 BGP Next hop: 2a0a:e5c0:0:b::89 fe80::3eec:efff:fed2:d1e4 IGP IPv6 table: master6 -------------------------------------------------------------------------------- But it seems server145 is not exporting them from how it looks: -------------------------------------------------------------------------------- Channel ipv6 State: UP Table: master6 Preference: 100 Input filter: ACCEPT Output filter: static_and_bgp Routes: 206011 imported, 3 exported, 206008 preferred Route change stats: received rejected filtered ignored accepted Import updates: 503843 0 0 445 503398 Import withdraws: 91701 0 --- 78 91623 Export updates: 927807 684226 97 --- 243484 Export withdraws: 33627 --- --- --- 230240 BGP Next hop: 2a0a:e5c0:0:b::91 fe80::2f0:cbff:fefe:b70c IGP IPv6 table: master6 -------------------------------------------------------------------------------- Even though a lot more routes match the filter: -------------------------------------------------------------------------------- bird> show route count filter static_and_bgp 3803744 of 3803746 routes for 950939 networks in table master4 824133 of 824170 routes for 206080 networks in table master6 Total: 4627877 of 4627916 routes for 1157019 networks in 2 tables bird> -------------------------------------------------------------------------------- So is this a misconfiguration or a bug or intended behaviour? Because *if* the upstream of server137 goes down, it starts receiving routes from server145, so failover works, but it seems wrong to me to not have the routes by default. Best regards, Nico p.s.: the config for server145 that does not export the routes is: -------------------------------------------------------------------------------- define pref_tunnel_fiberstream = 80; # used filter static_and_bgp { if source ~ [ RTS_STATIC, RTS_BGP ] then accept; reject; } protocol bgp outgoing_server120 { local as myas; direct; password "..."; bfd on; neighbor 2a0a:e5c0:31:4::1 as 209898; local role customer; default bgp_local_pref pref_tunnel_fiberstream; ipv6 { import filter allow_all; export filter p5_to_world; }; ipv4 { import filter allow_all; export filter p5_to_world; extended next hop on; }; } protocol bgp ibgp_server137 { local as myas; neighbor 2a0a:e5c0:0:b::89 as myas; direct; bfd on; ipv6 { import all; export filter static_and_bgp; gateway recursive; }; ipv4 { import all; export filter static_and_bgp; gateway recursive; extended next hop on; }; } -------------------------------------------------------------------------------- -- Sustainable and modern Infrastructures by ungleich.ch
On Mon, Jul 01, 2024 at 10:26:40AM +0200, Nico Schottelius via Bird-users wrote:
Hello everyone,
I was wondering how bird *should* behave if within one AS a route is received from outside multiple times but with different local_pref set?
My expectation would be that using iBGP, the lower local_pref route would show up on all routers, but the higher one is selected.
What I observe is that seems the router who initially receives the lower local_pref route does not export it via iBGP, even though it matches the export filter.
...
So is this a misconfiguration or a bug or intended behaviour?
Hi In general, only the best route is exported. The route with higher local_pref is the best route, it is not exported to other IBGPs due to being received from an IBGP, but still prevents export of less-prefered route. You can enable the option 'secondary' on IBGP and the option 'sorted' on the routing table, this would change the behavior to 'export the best route from exportable routes', thus the best EBGP route pasing filter when exporting to IBGP. Note that exporting paths that are not best / active in FIB may be problematic, as it may lead to routing loops, but in this specific case (propagating best EBGP) it is likely safe enough. -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) "To err is human -- to blame it on a computer is even more so."
Hello Ondrej, Tim, thanks for pointing out that only the the best route is exported. While I understand that logic, I am not sure if it is smart or really within the intention of the protocol. Let me try to shortly summarise it: - server145 receives an eBGP route for the target that is only intended to be used as a backup, thus has a low local pref - server145 also receives various iBGP routes for the target, which are better than the eBGP route according to our local_pref - Thus server145 does *NOT* export its eBGP learned route via iBGP to other nodes in the same AS <--- this part I doubt whether it is correct or smart - Because of that all other nodes in the same AS are unaware of the routes that server145 received, unless the other eBGP routes / upstreams are disappearing So in a nutshell, the "export only best route" behaviour hides potential other paths by default from the rest of the nodes and they only learn about it after the other eBGP upstream(s) are gone. Wouldn't it be much smarter if server145 would export its routes as well, with the lower local_pref attributes, as they are received by eBGP and thus represent another path outside of the AS? Note: I am not talking about which routes end up in the kernel, this is purely about route exchange within the different bgp speakers of one AS. Does it make sense what I am writing or am I totally off? Or in other words: how do you usually handle redundant upstreams that have different priority? Does everyone in the list "just waits" for one upstream to break until they routes are propagated? That seems incorrect to me. Best regards, Nico -- Sustainable and modern Infrastructures by ungleich.ch
This is how BGP works. If you want to exchange non-best-paths, use “add-paths”. -- Richard
Hi, don't know if i'm right, but i think this has nothing to do with iBGP and local_pref behaviour. As i see you don't use "add path", therefore i understood that only "best paths" are exchanged and for your server137 the route with local_pref 100 is the best path. If you want "all available" routes propagated via (i)BGP you should use add paths switch|rx|tx in my opinion to also transport the "non preferred" paths. HTH, tim On Mon, Jul 01, 2024 at 10:26:40AM GMT, Nico Schottelius via Bird-users wrote:
Hello everyone,
I was wondering how bird *should* behave if within one AS a route is received from outside multiple times but with different local_pref set?
My expectation would be that using iBGP, the lower local_pref route would show up on all routers, but the higher one is selected.
What I observe is that seems the router who initially receives the lower local_pref route does not export it via iBGP, even though it matches the export filter.
Example:
- server145 has a non-preferable uplink and imports all routes with local_pref = 80 - server137 has a preferable uplink and imports all routes without local_pref modification, thus local_pref = 100 - server145 has both routes - server137 only has the preferable route - It seems that server145 is not exporting its lower local_pref routes, even though they match the export filter
Excerpt from server145, importing 2600:: route from outside, with setting local_pref to 80:
-------------------------------------------------------------------------------- bird> show route all for 2600:: Table master6: 2600::/48 unicast [ibgp_server137 14:34:07.078 from 2a0a:e5c0:0:b::89] * (100/96) [AS1239i] via fe80::3eec:efff:fed2:d1e4 on eth0 Type: BGP univ BGP.origin: IGP BGP.as_path: 209898 39040 1299 174 1239 BGP.next_hop: 2a0a:e5c0:31:2::1 BGP.local_pref: 100 BGP.community: (1299,20000) (39040,100) (39040,109) (39040,1000) BGP.otc: 209898 unicast [outgoing_server120 14:33:56.123] (100) [AS1239i] via 2a0a:e5c0:31:4::1 on tun1 Type: BGP univ BGP.origin: IGP BGP.as_path: 209898 39040 1299 174 1239 BGP.next_hop: 2a0a:e5c0:31:4::1 fe80::897b:493f:bd44:ec60 BGP.local_pref: 80 BGP.community: (1299,20000) (39040,100) (39040,109) (39040,1000) BGP.otc: 209898 unicast [outgoing_server121 14:34:41.137] (100) [AS1239i] via 2a0a:e5c0:32:4::1 on tun0 Type: BGP univ BGP.origin: IGP BGP.as_path: 209898 15576 3356 174 1239 BGP.next_hop: 2a0a:e5c0:32:4::1 fe80::fa84:82d2:a117:43c BGP.local_pref: 80 BGP.community: (3356,2) (3356,22) (3356,86) (3356,505) (3356,601) (3356,666) (3356,903) (3356,2074) (15576,100) (15576,102) (15576,1000) BGP.otc: 209898 unicast [ibgp_server138 14:34:56.416 from 2a0a:e5c0:0:b::8a] (100/96) [AS1239i] via fe80::3eec:efff:fed2:d0c4 on eth0 Type: BGP univ BGP.origin: IGP BGP.as_path: 209898 15576 3356 174 1239 BGP.next_hop: 2a0a:e5c0:32:2::1 BGP.local_pref: 100 BGP.community: (3356,2) (3356,22) (3356,86) (3356,505) (3356,601) (3356,666) (3356,903) (3356,2074) (15576,100) (15576,102) (15576,1000) BGP.otc: 209898 --------------------------------------------------------------------------------
And on server137, this route is not visible, only the local_pref = 100 are visible:
-------------------------------------------------------------------------------- bird> show route all for 2600:: Table master6: 2600::/48 unicast [outgoing_server120 14:41:16.420] * (100) [AS1239i] via 2a0a:e5c0:31:2::1 on oserver120 Type: BGP univ BGP.origin: IGP BGP.as_path: 209898 39040 1299 174 1239 BGP.next_hop: 2a0a:e5c0:31:2::1 BGP.local_pref: 100 BGP.community: (1299,20000) (39040,100) (39040,109) (39040,1000) BGP.otc: 209898 unicast [ibgp_server138 14:42:17.052 from 2a0a:e5c0:0:b::8a] (100/96) [AS1239i] via fe80::3eec:efff:fed2:d0c7 on eth5 Type: BGP univ BGP.origin: IGP BGP.as_path: 209898 15576 3356 174 1239 BGP.next_hop: 2a0a:e5c0:32:2::1 BGP.local_pref: 100 BGP.community: (3356,2) (3356,22) (3356,86) (3356,505) (3356,601) (3356,666) (3356,903) (3356,2074) (15576,100) (15576,102) (15576,1000) BGP.otc: 209898 --------------------------------------------------------------------------------
On the iBGP session we import/export everything that is either static or BGP,excerpt from server137:
-------------------------------------------------------------------------------- Channel ipv6 State: UP Table: master6 Preference: 100 Input filter: ACCEPT Output filter: static_and_bgp Routes: 3 imported, 206002 exported, 2 preferred Route change stats: received rejected filtered ignored accepted Import updates: 211444 0 0 70 211374 Import withdraws: 230070 0 --- 24038 206032 Export updates: 645070 137427 98 --- 507545 Export withdraws: 64508 --- --- --- 96296 BGP Next hop: 2a0a:e5c0:0:b::89 fe80::3eec:efff:fed2:d1e4 IGP IPv6 table: master6 --------------------------------------------------------------------------------
But it seems server145 is not exporting them from how it looks:
-------------------------------------------------------------------------------- Channel ipv6 State: UP Table: master6 Preference: 100 Input filter: ACCEPT Output filter: static_and_bgp Routes: 206011 imported, 3 exported, 206008 preferred Route change stats: received rejected filtered ignored accepted Import updates: 503843 0 0 445 503398 Import withdraws: 91701 0 --- 78 91623 Export updates: 927807 684226 97 --- 243484 Export withdraws: 33627 --- --- --- 230240 BGP Next hop: 2a0a:e5c0:0:b::91 fe80::2f0:cbff:fefe:b70c IGP IPv6 table: master6 --------------------------------------------------------------------------------
Even though a lot more routes match the filter:
-------------------------------------------------------------------------------- bird> show route count filter static_and_bgp 3803744 of 3803746 routes for 950939 networks in table master4 824133 of 824170 routes for 206080 networks in table master6 Total: 4627877 of 4627916 routes for 1157019 networks in 2 tables bird> --------------------------------------------------------------------------------
So is this a misconfiguration or a bug or intended behaviour?
Because *if* the upstream of server137 goes down, it starts receiving routes from server145, so failover works, but it seems wrong to me to not have the routes by default.
Best regards,
Nico
p.s.: the config for server145 that does not export the routes is:
-------------------------------------------------------------------------------- define pref_tunnel_fiberstream = 80; # used
filter static_and_bgp { if source ~ [ RTS_STATIC, RTS_BGP ] then accept; reject; }
protocol bgp outgoing_server120 { local as myas; direct; password "..."; bfd on;
neighbor 2a0a:e5c0:31:4::1 as 209898; local role customer; default bgp_local_pref pref_tunnel_fiberstream;
ipv6 { import filter allow_all; export filter p5_to_world; };
ipv4 { import filter allow_all; export filter p5_to_world; extended next hop on; }; }
protocol bgp ibgp_server137 { local as myas; neighbor 2a0a:e5c0:0:b::89 as myas; direct; bfd on;
ipv6 { import all; export filter static_and_bgp;
gateway recursive; };
ipv4 { import all; export filter static_and_bgp;
gateway recursive; extended next hop on; }; } --------------------------------------------------------------------------------
-- Sustainable and modern Infrastructures by ungleich.ch
-- Tim Weippert http://weiti.org - weiti@weiti.org GPG Fingerprint - E704 7303 6FF0 8393 ADB1 398E 67F2 94AE 5995 7DD8
participants (4)
-
Nico Schottelius -
Ondrej Zajicek -
Richard Laager -
Tim Weippert