Receive/export route limit behaviour?
Hellow, On my BIRD router, I have two IPv4 BGP upstream neighbors. From them, my router receive 678,000 and 681,000 routes. On these BGP sessions, I have configured "receive limit 700000 action block". On this same router, I have one IPv4 BGP downstream neighbor. On this one, I have configured "export where proto = "static_allocations" || proto = "bgp_upstream1" || proto = "bgp_upstream2";" and "export limit 700000 action disable". "static_allocations" always contains only one route. BIRD frequently disables downstream neighbor's BGP session because the export limit has been exceeded. I can understand this behaviour: upstream1 may distribute routes that upstream2 does not have, but I see the same behavior with "export limit 750000 action disable". However, my upstreams do not provide me more than 50,000 different routes: # birdc sh route protocol bgp_upstream1 | grep -Po "[0-9./]+(?=[ ]+via)" | sort -n > bgp_upstream1 # birdc sh route protocol bgp_upstream2 | grep -Po "[0-9./]+(?=[ ]+via)" | sort -n > bgp_upstream2 # diff -b -w -y --suppress-common-lines bgp_upstream1 bgp_upstream2 | wc -l 3503 How is it possible to export more routes than I receive? In IPv6, with the same upstream and downstram neighbors and "receive limit" set to 50,000 and "export limit" set to 50,000, I have no problem. Regards.
On Tue, Apr 03, 2018 at 10:52:14PM +0200, Guillaume LUCAS wrote:
Hellow,
On my BIRD router, I have two IPv4 BGP upstream neighbors. From them, my router receive 678,000 and 681,000 routes. On these BGP sessions, I have configured "receive limit 700000 action block".
On this same router, I have one IPv4 BGP downstream neighbor. On this one, I have configured "export where proto = "static_allocations" || proto = "bgp_upstream1" || proto = "bgp_upstream2";" and "export limit 700000 action disable". "static_allocations" always contains only one route.
BIRD frequently disables downstream neighbor's BGP session because the export limit has been exceeded. I can understand this behaviour: upstream1 may distribute routes that upstream2 does not have, but I see the same behavior with "export limit 750000 action disable". However, my upstreams do not provide me more than 50,000 different routes:
# birdc sh route protocol bgp_upstream1 | grep -Po "[0-9./]+(?=[ ]+via)" | sort -n > bgp_upstream1 # birdc sh route protocol bgp_upstream2 | grep -Po "[0-9./]+(?=[ ]+via)" | sort -n > bgp_upstream2 # diff -b -w -y --suppress-common-lines bgp_upstream1 bgp_upstream2 | wc -l 3503
How is it possible to export more routes than I receive?
Hi What do you see in 'show protocols all' in import/export statistics? Do they make sense? -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
Le 04/04/2018 à 15:28, Ondrej Zajicek a écrit :
Hi
What do you see in 'show protocols all' in import/export statistics? Do they make sense?
Hi, # sudo birdc show protocols all direct1 Direct master up 2018-01-12 Routes: 2 imported, 0 exported, 2 preferred Route change stats: received rejected filtered ignored accepted Import updates: 3 0 0 0 3 Import withdraws: 1 0 --- 0 1 Export updates: 0 0 0 --- 0 Export withdraws: 0 --- --- --- 0 kernel1 Kernel master up 2018-01-12 Routes: 31 imported, 681991 exported, 30 preferred Route change stats: received rejected filtered ignored accepted Import updates: 52 0 1 0 51 Import withdraws: 20 0 --- 1 20 Export updates: 83427721 55 0 --- 83427666 Export withdraws: 4163126 --- --- --- 4163135 static_allocations Static master up 2018-01-12 Routes: 1 imported, 0 exported, 1 preferred Route change stats: received rejected filtered ignored accepted Import updates: 1 0 0 0 1 Import withdraws: 0 0 --- 0 0 Export updates: 0 0 0 --- 0 Export withdraws: 0 --- --- --- 0 bgp_upstream1 BGP master up 2018-01-27 Established Routes: 678683 imported, 0 filtered, 1 exported, 569533 preferred Route change stats: received rejected filtered ignored accepted Import updates: 38113294 0 294 12493695 25619305 Import withdraws: 2348477 0 --- 280 2348484 Export updates: 69298824 25943901 43354922 --- 1 Export withdraws: 2932089 --- --- --- 0 bgp_upstream2 BGP master up 2018-04-03 Established Routes: 681762 imported, 77 filtered, 1 exported, 112446 preferred Route change stats: received rejected filtered ignored accepted Import updates: 2828562 0 109 0 2828453 Import withdraws: 162753 0 --- 125 162737 Export updates: 2175525 646411 1529113 --- 1 Export withdraws: 67445 --- --- --- 0 ibgp BGP master up 2018-02-18 Established Routes: 681993 imported, 682012 exported, 10 preferred Route change stats: received rejected filtered ignored accepted Import updates: 29660288 0 0 518955 29141333 Import withdraws: 7164695 0 --- 0 7164695 Export updates: 51000527 12312881 0 --- 38687646 Export withdraws: 1950522 --- --- --- 12715457 bgp_downstream BGP master up 20:17:52 Established Routes: 0 imported, 681980 exported, 0 preferred Route change stats: received rejected filtered ignored accepted Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 2121052 0 17236 --- 2103816 Export withdraws: 4750 --- --- --- 18179 It seems to me it makes sense: in ibgp (no filter in place), this router redistributes 682012 routes (upstream1 + upstream2 + some differences between them, probably). 682012 - 2 (direct1) - 31 (kernel1) = 681979 = downstream. (kernel1 and direct1 are block by the "export where", see my previous mail) Yes, I use iBGP because I have an another router. Quagga. Same two upstreams. No downstreams. Same receive limits (shutdown BGP session if routes > 700,000).
I'm not sure the BIRD logic, but it sounds like you're saying: - send static *and* - send all provider A routes *and* - send all provider B routes These would total more than a million routes. Usually what you share downstream is your own routes; the best-path selection after BGP processes all the routes it learned. That would be a list of about 680k best routes, and be less than your filter. (alternately, you're getting different routes from each of your providers, so if you add them together and remove the duplicates, you have more than 700k routes. I have more than 700k routes in my default-free zone) Jonathan On Wed, Apr 4, 2018 at 4:42 PM, Guillaume LUCAS <glucas+bird@glucas.fr> wrote:
Le 04/04/2018 à 15:28, Ondrej Zajicek a écrit :
Hi
What do you see in 'show protocols all' in import/export statistics? Do they make sense?
Hi,
# sudo birdc show protocols all
direct1 Direct master up 2018-01-12 Routes: 2 imported, 0 exported, 2 preferred Route change stats: received rejected filtered ignored accepted Import updates: 3 0 0 0 3 Import withdraws: 1 0 --- 0 1 Export updates: 0 0 0 --- 0 Export withdraws: 0 --- --- --- 0
kernel1 Kernel master up 2018-01-12 Routes: 31 imported, 681991 exported, 30 preferred Route change stats: received rejected filtered ignored accepted Import updates: 52 0 1 0 51 Import withdraws: 20 0 --- 1 20 Export updates: 83427721 55 0 --- 83427666 Export withdraws: 4163126 --- --- --- 4163135
static_allocations Static master up 2018-01-12 Routes: 1 imported, 0 exported, 1 preferred Route change stats: received rejected filtered ignored accepted Import updates: 1 0 0 0 1 Import withdraws: 0 0 --- 0 0 Export updates: 0 0 0 --- 0 Export withdraws: 0 --- --- --- 0
bgp_upstream1 BGP master up 2018-01-27 Established Routes: 678683 imported, 0 filtered, 1 exported, 569533 preferred Route change stats: received rejected filtered ignored accepted Import updates: 38113294 0 294 12493695 25619305 Import withdraws: 2348477 0 --- 280 2348484 Export updates: 69298824 25943901 43354922 --- 1 Export withdraws: 2932089 --- --- --- 0
bgp_upstream2 BGP master up 2018-04-03 Established Routes: 681762 imported, 77 filtered, 1 exported, 112446 preferred Route change stats: received rejected filtered ignored accepted Import updates: 2828562 0 109 0 2828453 Import withdraws: 162753 0 --- 125 162737 Export updates: 2175525 646411 1529113 --- 1 Export withdraws: 67445 --- --- --- 0
ibgp BGP master up 2018-02-18 Established Routes: 681993 imported, 682012 exported, 10 preferred Route change stats: received rejected filtered ignored accepted Import updates: 29660288 0 0 518955 29141333 Import withdraws: 7164695 0 --- 0 7164695 Export updates: 51000527 12312881 0 --- 38687646 Export withdraws: 1950522 --- --- --- 12715457
bgp_downstream BGP master up 20:17:52 Established Routes: 0 imported, 681980 exported, 0 preferred Route change stats: received rejected filtered ignored accepted Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 2121052 0 17236 --- 2103816 Export withdraws: 4750 --- --- --- 18179
It seems to me it makes sense: in ibgp (no filter in place), this router redistributes 682012 routes (upstream1 + upstream2 + some differences between them, probably). 682012 - 2 (direct1) - 31 (kernel1) = 681979 = downstream. (kernel1 and direct1 are block by the "export where", see my previous mail)
Yes, I use iBGP because I have an another router. Quagga. Same two upstreams. No downstreams. Same receive limits (shutdown BGP session if routes > 700,000).
-- Jonathan
Le 06/04/2018 à 00:23, Jonathan Stewart a écrit :
I'm not sure the BIRD logic, but it sounds like you're saying:
- send static *and* - send all provider A routes *and* - send all provider B routes
I understand: send my static and send all provider A *best* routes and send all provider B *best* routes. For me, the BGP path selection should take place before the redistribution to my peers (iBGP and downstreams). Maybe I'm wrong?
These would total more than a million routes.
If you were right, why does it work when I set "export limit 770000 action disable" but not when I set "export limit 700000" or "export limit 750000"? "It works" means that the BGP session has not been disabled for a week.
(alternately, you're getting different routes from each of your providers, so if you add them together and remove the duplicates, you have more than 700k routes. I have more than 700k routes in my default-free zone)
It seems not: # birdc sh route protocol bgp_upstream1 | grep -Po "[0-9./]+(?=[ ]+via)" > bgp_upstream1 # birdc sh route protocol bgp_upstream2 | grep -Po "[0-9./]+(?=[ ]+via)" > bgp_upstream2 # birdc sh route export bgp_downstream | wc -l 683219 # wc -l bgp_upstream1 bgp_upstream2 679509 bgp_upstream1 682975 bgp_upstream2 1362484 total # sort -n bgp_upstream1 bgp_upstream2 | uniq | wc -l 683218 ^ gap is the static route. As I have said in my first mail, I have a gap of approximately 3500-4000 different routes between my two upstreams (add routes and remove duplicates routes), less than my filter: # sort -n bgp_upstream1 bgp_upstream2 | uniq -c | grep -E "^[ ]+1 " | wc -l 3952 How is it possible that the sum of routes received from my upstreams minus duplicates ones exceeds 750k when I use "receive limit 700000" on each upstream? This would mean that an upstream sometimes removes correct routes before inserting rogue or disaggregated ones. If not, the "receive limit" on upstreams would be effective. I'm stuck on this point.
participants (3)
-
Guillaume LUCAS -
Jonathan Stewart -
Ondrej Zajicek