Out of memory crash of 3000 routes in BGP
Madhu
informmadhu at gmail.com
Tue Oct 11 20:05:29 CEST 2016
Hi everyone,
Now i tested with 16 path and 32 path, I don't see the crash. Below are
the memory usage for each.
*16 path*
bird> show memory
BIRD memory usage
Routing tables: 5109 kB
Route attributes: 13 kB
ROA tables: 192 B
Protocols: 163 kB
Total: 5384 kB
*32 path*
bird> show memory
BIRD memory usage
Routing tables: 10 MB
Route attributes: 16 kB
ROA tables: 192 B
Protocols: 239 kB
Total: 10 MB
Now I increased to* 62 path*, I see that* CPU is going high to 100%*, and
the memory usage is 0.1%. birdc socket got struck. I was not able to access
the socket.
bird> show memory
BIRD memory usage
Routing tables: 5127 kB
Route attributes: 20 kB
ROA tables: 192 B
Protocols: 382 kB
Total: 5641 kB
bird> show memory >>. After this it got struck
# top
top - 17:47:50 up 1:02, 2 users, load average: 1.35, 1.42, 1.10
Tasks: 163 total, 2 running, 161 sleeping, 0 stopped, 0 zombie
%Cpu(s): 29.7 us, 0.5 sy, 0.2 ni, 69.4 id, 0.0 wa, 0.0 hi, 0.2 si,
0.0 st
KiB Mem: 8127384 total, 1279304 used, 6848080 free, 36040 buffers
KiB Swap: 812736 total, 0 used, 812736 free. 456220 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8768 bird 20 0 21728 8808 2352 R* 100.0* 0.1 8:12.97
bird
Attached the bird config file.
Madhu
On Tue, Oct 11, 2016 at 5:56 AM, Justin Cattle <j at ocado.com> wrote:
> It looks like oom-kill from the call trace I think.
>
>
>
> Cheers,
> Just
>
> On 11 October 2016 at 13:07, Ondrej Zajicek <santiago at crfreenet.org>
> wrote:
>
>> On Mon, Oct 10, 2016 at 06:21:44PM -0700, Madhu wrote:
>> > I have 3000 routes with 64 path ecmp. It is crashing . I don't see the
>> coredump . Is there anyway to fix using trace ?
>>
>> Hi
>>
>> I would like to confirm, it is BIRD that was killed by out of memory
>> killer and could be just restarted? Or Linux kernel crashed? (You wrote
>> 'the kernel crashed with out of memory issue' in one prior mail.)
>>
>> In the first case, how much memory BIRD takes after start and before it
>> crashes?
>> How long it takes to crash to crash due to OOM?
>>
>>
>> > On Oct 10, 2016, at 6:08 PM, Jonathan Stewart <
>> jonathan.stewart at gmail.com> wrote:
>> > It must be something specific to your environment.
>>
>> It is likely related to 64 path BGP multipath exported to kernel.
>>
>> --
>> Elen sila lumenn' omentielvo
>>
>> Ondrej 'Santiago' Zajicek (email: santiago at crfreenet.org)
>> OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net)
>> "To err is human -- to blame it on a computer is even more so."
>>
>>
>
> Notice: This email is confidential and may contain copyright material of
> members of the Ocado Group. Opinions and views expressed in this message
> may not necessarily reflect the opinions and views of the members of the
> Ocado Group.
>
>
>
> If you are not the intended recipient, please notify us immediately and
> delete all copies of this message. Please note that it is your
> responsibility to scan this message for viruses.
>
>
>
> Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled
> is a trading name of Marie Claire Beauty Limited, both members of the Ocado
> Group.
>
>
>
> References to the “Ocado Group” are to Ocado Group plc (registered in
> England and Wales with number 7098618) and its subsidiary undertakings (as
> that expression is defined in the Companies Act 2006) from time to time.
> The registered office of Ocado Group plc is Titan Court, 3 Bishops Square,
> Hatfield Business Park, Hatfield, Herts. AL10 9NE.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://trubka.network.cz/pipermail/bird-users/attachments/20161011/742ef689/attachment.html>
-------------- next part --------------
router id 100.0.2.1;
log syslog all;
protocol kernel {
scan time 60;
import all;
merge paths yes limit 255;
export all; # Actually insert routes into the kernel routing table
}
protocol device {
scan time 60;
}
filter bgp_import {
if net ~ [ 120.100.0.0/16+] then accept;
reject;
}
filter kernel_export {
if net ~ [ 120.100.0.0/16+] then accept;
reject;
}
filter bgp_export {
reject;
}
protocol bgp 'AG1-TR1' {
import all;
# export where proto = "static_bgp";
local as 64801;
neighbor 100.0.3.1 as 64801;
source address 100.0.2.1;
rr client;
next hop self;
keepalive time 1;
hold time 3;
}
protocol bgp 'AG1-TR2' {
local as 64801;
neighbor 100.0.3.2 as 64801;
source address 100.0.2.1;
rr client;
next hop self;
keepalive time 1;
hold time 3;
}
protocol bgp 'AG1-CR1-link0' {
local as 64801;
neighbor 100.1.2.0 as 64901;
source address 100.1.2.1;
# import filter bgp_import;
# export filter bgp_export;
}
protocol bgp 'AG1-CR1-link1' {
local as 64801;
neighbor 100.1.3.0 as 64901;
source address 100.1.3.1;
# import filter bgp_import;
# export filter bgp_export;
}
protocol bgp 'AG1-CR1-link2' {
local as 64801;
neighbor 100.1.4.0 as 64901;
source address 100.1.4.1;
# import filter bgp_import;
# export filter bgp_export;
}
protocol bgp 'AG1-CR1-link3' {
local as 64801;
neighbor 100.1.5.0 as 64901;
source address 100.1.5.1;
}
protocol bgp 'AG1-CR1-link4' {
local as 64801;
neighbor 100.1.6.0 as 64901;
source address 100.1.6.1;
}
protocol bgp 'AG1-CR1-link5' {
local as 64801;
neighbor 100.1.7.0 as 64901;
source address 100.1.7.1;
}
protocol bgp 'AG1-CR1-link6' {
local as 64801;
neighbor 100.1.8.0 as 64901;
source address 100.1.8.1;
}
protocol bgp 'AG1-CR1-link7' {
local as 64801;
neighbor 100.1.9.0 as 64901;
source address 100.1.9.1;
}
protocol bgp 'AG1-CR1-link8' {
local as 64801;
neighbor 100.1.10.0 as 64901;
source address 100.1.10.1;
}
protocol bgp 'AG1-CR1-link9' {
local as 64801;
neighbor 100.1.11.0 as 64901;
source address 100.1.11.1;
}
protocol bgp 'AG1-CR1-link10' {
local as 64801;
neighbor 100.1.12.0 as 64901;
source address 100.1.12.1;
}
protocol bgp 'AG1-CR1-link11' {
local as 64801;
neighbor 100.1.13.0 as 64901;
source address 100.1.13.1;
}
protocol bgp 'AG1-CR1-link12' {
local as 64801;
neighbor 100.1.14.0 as 64901;
source address 100.1.14.1;
}
protocol bgp 'AG1-CR1-link13' {
local as 64801;
neighbor 100.1.15.0 as 64901;
source address 100.1.15.1;
}
protocol bgp 'AG1-CR1-link14' {
local as 64801;
neighbor 100.1.16.0 as 64901;
source address 100.1.16.1;
}
protocol bgp 'AG1-CR1-link15' {
local as 64801;
neighbor 100.1.17.0 as 64901;
source address 100.1.17.1;
}
protocol bgp 'AG1-CR1-link16' {
local as 64801;
neighbor 100.1.18.0 as 64901;
source address 100.1.18.1;
}
protocol bgp 'AG1-CR1-link17' {
local as 64801;
neighbor 100.1.19.0 as 64901;
source address 100.1.19.1;
}
protocol bgp 'AG1-CR1-link18' {
local as 64801;
neighbor 100.1.20.0 as 64901;
source address 100.1.20.1;
}
protocol bgp 'AG1-CR1-link19' {
local as 64801;
neighbor 100.1.21.0 as 64901;
source address 100.1.21.1;
}
protocol bgp 'AG1-CR1-link20' {
local as 64801;
neighbor 100.1.22.0 as 64901;
source address 100.1.22.1;
}
protocol bgp 'AG1-CR1-link21' {
local as 64801;
neighbor 100.1.23.0 as 64901;
source address 100.1.23.1;
}
protocol bgp 'AG1-CR1-link22' {
local as 64801;
neighbor 100.1.24.0 as 64901;
source address 100.1.24.1;
}
protocol bgp 'AG1-CR1-link23' {
local as 64801;
neighbor 100.1.25.0 as 64901;
source address 100.1.25.1;
}
protocol bgp 'AG1-CR1-link24' {
local as 64801;
neighbor 100.1.26.0 as 64901;
source address 100.1.26.1;
}
protocol bgp 'AG1-CR1-link25' {
local as 64801;
neighbor 100.1.27.0 as 64901;
source address 100.1.27.1;
}
protocol bgp 'AG1-CR1-link27' {
local as 64801;
neighbor 100.1.28.0 as 64901;
source address 100.1.28.1;
}
protocol bgp 'AG1-CR1-link28' {
local as 64801;
neighbor 100.1.29.0 as 64901;
source address 100.1.29.1;
}
protocol bgp 'AG1-CR1-link29' {
local as 64801;
neighbor 100.1.30.0 as 64901;
source address 100.1.30.1;
}
protocol bgp 'AG1-CR1-link30' {
local as 64801;
neighbor 100.1.31.0 as 64901;
source address 100.1.31.1;
}
protocol bgp 'AG1-CR1-link31' {
local as 64801;
neighbor 100.1.32.0 as 64901;
source address 100.1.32.1;
}
protocol bgp 'AG1-CR1-link32' {
local as 64801;
neighbor 100.1.33.0 as 64901;
source address 100.1.33.1;
}
protocol bgp 'AG1-CR1-link33' {
local as 64801;
neighbor 100.1.34.0 as 64901;
source address 100.1.34.1;
}
protocol bgp 'AG1-CR1-link34' {
local as 64801;
neighbor 100.1.35.0 as 64901;
source address 100.1.35.1;
}
protocol bgp 'AG1-CR1-link35' {
local as 64801;
neighbor 100.1.36.0 as 64901;
source address 100.1.36.1;
}
protocol bgp 'AG1-CR1-link36' {
local as 64801;
neighbor 100.1.37.0 as 64901;
source address 100.1.37.1;
}
protocol bgp 'AG1-CR1-link37' {
local as 64801;
neighbor 100.1.38.0 as 64901;
source address 100.1.38.1;
}
protocol bgp 'AG1-CR1-link38' {
local as 64801;
neighbor 100.1.39.0 as 64901;
source address 100.1.39.1;
}
protocol bgp 'AG1-CR1-link39' {
local as 64801;
neighbor 100.1.40.0 as 64901;
source address 100.1.40.1;
}
protocol bgp 'AG1-CR1-link40' {
local as 64801;
neighbor 100.1.41.0 as 64901;
source address 100.1.41.1;
}
protocol bgp 'AG1-CR1-link41' {
local as 64801;
neighbor 100.1.42.0 as 64901;
source address 100.1.42.1;
}
protocol bgp 'AG1-CR1-link42' {
local as 64801;
neighbor 100.1.43.0 as 64901;
source address 100.1.43.1;
}
protocol bgp 'AG1-CR1-link43' {
local as 64801;
neighbor 100.1.44.0 as 64901;
source address 100.1.44.1;
}
protocol bgp 'AG1-CR1-link44' {
local as 64801;
neighbor 100.1.45.0 as 64901;
source address 100.1.45.1;
}
protocol bgp 'AG1-CR1-link45' {
local as 64801;
neighbor 100.1.46.0 as 64901;
source address 100.1.46.1;
}
protocol bgp 'AG1-CR1-link46' {
local as 64801;
neighbor 100.1.47.0 as 64901;
source address 100.1.47.1;
}
protocol bgp 'AG1-CR1-link47' {
local as 64801;
neighbor 100.1.48.0 as 64901;
source address 100.1.48.1;
}
protocol bgp 'AG1-CR1-link48' {
local as 64801;
neighbor 100.1.49.0 as 64901;
source address 100.1.49.1;
}
protocol bgp 'AG1-CR1-link49' {
local as 64801;
neighbor 100.1.50.0 as 64901;
source address 100.1.50.1;
}
protocol bgp 'AG1-CR1-link50' {
local as 64801;
neighbor 100.1.51.0 as 64901;
source address 100.1.51.1;
}
protocol bgp 'AG1-CR1-link51' {
local as 64801;
neighbor 100.1.52.0 as 64901;
source address 100.1.52.1;
}
protocol bgp 'AG1-CR1-link52' {
local as 64801;
neighbor 100.1.53.0 as 64901;
source address 100.1.53.1;
}
protocol bgp 'AG1-CR1-link53' {
local as 64801;
neighbor 100.1.54.0 as 64901;
source address 100.1.54.1;
}
protocol bgp 'AG1-CR1-link54' {
local as 64801;
neighbor 100.1.55.0 as 64901;
source address 100.1.55.1;
}
protocol bgp 'AG1-CR1-link55' {
local as 64801;
neighbor 100.1.56.0 as 64901;
source address 100.1.56.1;
}
protocol bgp 'AG1-CR1-link56' {
local as 64801;
neighbor 100.1.57.0 as 64901;
source address 100.1.57.1;
}
protocol bgp 'AG1-CR1-link57' {
local as 64801;
neighbor 100.1.58.0 as 64901;
source address 100.1.58.1;
}
protocol bgp 'AG1-CR1-link58' {
local as 64801;
neighbor 100.1.59.0 as 64901;
source address 100.1.59.1;
}
protocol bgp 'AG1-CR1-link59' {
local as 64801;
neighbor 100.1.60.0 as 64901;
source address 100.1.60.1;
}
protocol bgp 'AG1-CR1-link60' {
local as 64801;
neighbor 100.1.61.0 as 64901;
source address 100.1.61.1;
}
protocol bgp 'AG1-CR1-link61' {
local as 64801;
neighbor 100.1.62.0 as 64901;
source address 100.1.62.1;
}
protocol bgp 'AG1-CR1-link62' {
local as 64801;
neighbor 100.1.63.0 as 64901;
source address 100.1.63.1;
}
More information about the Bird-users
mailing list