On Fri, 10 Jul 2026 16:10:09 +0200 Maria Matejka <maria.matejka@nic.cz> wrote:
Hi Mariusz,
On Fri, Jul 10, 2026 at 03:19:03PM +0200, Mariusz Gronczewski wrote:
I've been observing some memory leak (slowly gets to ~30GB across 18 hours then dies to oomkiller, ~500kB/s growth), how can I debug it furter ?
That is not an easy task, and I'm a little worried about your actual memory footprint. This is what I get with one uplink:
BIRD 3.3.1 ready. bird> show route count 1047039 of 1047039 routes for 1047039 networks in table master4 245208 of 245208 routes for 245208 networks in table master6 Total: 1292247 of 1292247 routes for 1292247 networks in 2 tables bird> show mem BIRD memory usage Effective Overhead Routing tables: 191.7 MB 12.8 MB Route attributes: 115.3 MB 70.2 MB Protocols: 371.2 kB 13.6 kB Current config: 17.0 kB 880.0 B Standby memory: 0.0 B 33.2 MB Total: 307.5 MB 116.3 MB
Active pages: 308.2 MB Kept free pages: 33.2 MB Cold free pages: 129.0 MB Hot page cache depleted while in RCU: 2482
we just have 2 peers with full tables so yeah, route count doesn't really match actually used memory
# birdc BIRD 3.3.1 ready. bird> show memory BIRD memory usage Effective Overhead Routing tables: 6059.3 MB 564.4 MB Route attributes: 9510.1 MB 565.8 MB Protocols: 22.3 MB 82.9 kB Current config: 60.8 kB 1232.0 B Standby memory: 0.0 B 32.7 MB Total: 15.2 GB 1163.0 MB
Active pages: 16.2 GB Kept free pages: 32.7 MB Cold free pages: 194.2 MB Hot page cache depleted while in RCU: 66929
bird> show route count 2098594 of 2098594 routes for 1051498 networks in table master4 0 of 0 routes for 0 networks in table master6
This consumption is in itself very fishy. My first look would be into your filters because 10G in route attributes is probably terribly wrong with just 2M routes, and there must be _some_ reason for that, probably impossible deduplication. Another look would be what happens when you slightly change your import filter (e.g. by adding yet another community) and see whether the reported memory footprint raises significantly. Also, it may be handy to set
debug protocols { states }; debug channels { routes, states }; debug tables { routes, states };
and see what is happening over time, but as it is slow, it may yield nothing.
Oh it isn't slow at all, in my case for some reason there is ton of 0.0.0.0/0 route updates like, kernel get good 2 orders of magnitude more updates than any of the BGP peers produces Now 0.0.0.0/0 isnt imported from any peer (dropped on filters), and only generated in static and then distributed to OSPF: protocol static generated { ipv4 { table generated_t; }; check link; igp table master4; route 193.200.227.0/24 blackhole; route 194.50.132.0/22 blackhole; route 0.0.0.0/0 recursive 8.8.8.8 { preference=500; }; route 0.0.0.0/0 recursive 1.1.1.1 { preference=290; }; route 0.0.0.0/0 recursive 13.96.0.1 { preference=280; }; route 0.0.0.0/0 recursive 142.250.0.1 { preference=270; }; route 0.0.0.0/0 recursive 18.244.0.1 { preference=260; }; route 0.0.0.0/0 unreachable { preference=1; }; }; That was my poor attempt at trying to generate one only on router that actually sees internet (and not just have peer up), I wanted to steer traffic to router that have "working" ISP, but I'm not sure why it would generate such flapping (and either way shouldnt really leak memory) I've attached log and full config
[...] learn all;
This may play some role but I don't believe that as the implementation of the kernel protocol almost didn't change between 3.1 and 3.3.
It only started happening once router was enabled (before it was peered but did not push the routes to kernel table), tho it coincided with upgrade from 3.1 to 3.3.1 as I hit different bug there .
I have also tried to disable "export/import table", "import keep filtered on" and "allow as sets" just to narrow it down but appears there is no change
I would expect that with export/import table off, the memory consumption would go away.
No change after disabling, still grows
By any chance, how much are you calling `birdc`? We got reported some memory leaks regarding that.
saw that thread, I've tried disabling montioring (we do have prometheus exporter exporting it quite often) but no change
Also, I expect that your OSPF is not producing way too many routes, but if you could elaborate on that number, it may be a clue.
around 100 or so so probably not
Also, `import keep filtered` in OSPF is not needed, there is no benefit from setting it.
good to know, thanks
Thanks for the report!
-- Mariusz Gronczewski (XANi) <xani+bird@devrandom.pl> GnuPG: 0xEA8ACE64 https://devrandom.eu