On Fri, 10 Jul 2026 16:10:09 +0200 Maria Matejka <maria.matejka@nic.cz> wrote:
Hi Mariusz,
On Fri, Jul 10, 2026 at 03:19:03PM +0200, Mariusz Gronczewski wrote:
I've been observing some memory leak (slowly gets to ~30GB across 18 hours then dies to oomkiller, ~500kB/s growth), how can I debug it furter ?
That is not an easy task, and I'm a little worried about your actual memory footprint. This is what I get with one uplink:
BIRD 3.3.1 ready. bird> show route count 1047039 of 1047039 routes for 1047039 networks in table master4 245208 of 245208 routes for 245208 networks in table master6 Total: 1292247 of 1292247 routes for 1292247 networks in 2 tables bird> show mem BIRD memory usage Effective Overhead Routing tables: 191.7 MB 12.8 MB Route attributes: 115.3 MB 70.2 MB Protocols: 371.2 kB 13.6 kB Current config: 17.0 kB 880.0 B Standby memory: 0.0 B 33.2 MB Total: 307.5 MB 116.3 MB
Active pages: 308.2 MB Kept free pages: 33.2 MB Cold free pages: 129.0 MB Hot page cache depleted while in RCU: 2482
# birdc BIRD 3.3.1 ready. bird> show memory BIRD memory usage Effective Overhead Routing tables: 6059.3 MB 564.4 MB Route attributes: 9510.1 MB 565.8 MB Protocols: 22.3 MB 82.9 kB Current config: 60.8 kB 1232.0 B Standby memory: 0.0 B 32.7 MB Total: 15.2 GB 1163.0 MB
Active pages: 16.2 GB Kept free pages: 32.7 MB Cold free pages: 194.2 MB Hot page cache depleted while in RCU: 66929
bird> show route count 2098594 of 2098594 routes for 1051498 networks in table master4 0 of 0 routes for 0 networks in table master6
This consumption is in itself very fishy. My first look would be into your filters because 10G in route attributes is probably terribly wrong with just 2M routes, and there must be _some_ reason for that, probably impossible deduplication.
Another look would be what happens when you slightly change your import filter (e.g. by adding yet another community) and see whether the reported memory footprint raises significantly. Also, it may be handy to set
debug protocols { states }; debug channels { routes, states }; debug tables { routes, states };
and see what is happening over time, but as it is slow, it may yield nothing.
oh no it's very fast, I think I have possibly found the problem (attached log) I noticed there is many changes in 0.0.0.0/0 route but I: * don't import it from any peer * export it only from OSPF * is generated only as static route : protocol static generated { ipv4 { table generated_t; }; check link; igp table master4; route 193.200.227.0/24 blackhole; route 194.50.132.0/22 blackhole; route 0.0.0.0/0 recursive 8.8.8.8 { preference=500; }; route 0.0.0.0/0 recursive 1.1.1.1 { preference=290; }; route 0.0.0.0/0 recursive 13.96.0.1 { preference=280; }; route 0.0.0.0/0 recursive 142.250.0.1 { preference=270; }; route 0.0.0.0/0 recursive 18.244.0.1 { preference=260; }; route 0.0.0.0/0 unreachable { preference=1; }; }; This was poor attempt (I really wanted to avoid external scripting) at generating a route preference based on whether a given router "sees" wider internet (vs just sending traffic to peer that's). But I don't understand why it causes so many updates
[...] learn all;
This may play some role but I don't believe that as the implementation of the kernel protocol almost didn't change between 3.1 and 3.3.
It only started happening once router was enabled (before it was peered but did not push the routes to kernel table), tho it coincided with upgrade from 3.1 to 3.3.1 as I hit different bug there .
I have also tried to disable "export/import table", "import keep filtered on" and "allow as sets" just to narrow it down but appears there is no change
I would expect that with export/import table off, the memory consumption would go away.
nope
By any chance, how much are you calling `birdc`? We got reported some memory leaks regarding that.
Yes, I saw that report, after stopping all of the tooling that does it doesn't really change so I don't think it is related
Also, I expect that your OSPF is not producing way too many routes, but if you could elaborate on that number, it may be a clue.
few hundred at most
Also, `import keep filtered` in OSPF is not needed, there is no benefit from setting it.
Thanks for the report!
-- Mariusz Gronczewski (XANi) <xani+bird@devrandom.pl> GnuPG: 0xEA8ACE64 https://devrandom.eu