freebsd mbuf cluster leak with bird 2.14

[Thomas Steen Rasmussen]

Hello :)

Yesterday I had one of my FreeBSD routers stop forwarding because it ran 
out of mbuf clusters. It usually operates far from the limit, but there 
is (was) something leaking mbuf clusters bad, and I suspect it might be 
bird.

----

Some background:

Due to a missing/misconfigured kernel export filter bird was repeatedly 
trying to export some routes to the kernel which the kernel already knew 
(from statically configured blackhole routes). So these errors have been 
repeating in the logs for some time (more than a year):

Jan 11 19:09:04 dgncr2a bird[30963]: KRT: Error sending route 
2a09:94c0::/29 to kernel: File exists
Jan 11 19:10:04 dgncr2a syslogd: last message repeated 1 times
Jan 11 19:10:04 dgncr2a bird[30963]: KRT: Error sending route 
85.209.116.0/22 to kernel: File exists
Jan 11 19:11:04 dgncr2a syslogd: last message repeated 1 times

Over the holidays I upgraded from bird 2.0.9 to bird 2.14, as well as 
upgrading FreeBSD from 13-STABLE-384a885111ad to 13-STABLE-2cbd132986a7. 
I suspect one of these two changes made this problem appear. I made no 
changes to bird or router config other than the upgrades.

----

The mbuf cluster leak was pretty bad, like 8-10 clusters per second at a 
pretty steady rate. The kern.ipc.nmbclusters limit on my routers was 
around 2 million and I raised it to 4 million now.

Since I had no idea what was causing the leak and I was desperate for a 
fix I at one point tried adding the missing kernel export filter (as to 
at least silence the noisy warnings in the logs), and imagine my 
surprise when the mbuf cluster leak stopped.

I tried removing the filers again, the leak started again, and stopped 
again when I re-added the filters. It appears some combination of bird 
2.14 and exporting routes already found in the kernel means leaking mbuf 
clusters like crazy.

I have no idea if this is a bird or a freebsd problem but I have to 
start somewhere :) I can to some extent test stuff, but the routers are 
in production (BGP with 1 ebgp and 1 ibgp peer and no full table) so 
nothing too wild.

Thanks! :)

Best regards,

Thomas Steen Rasmussen / Tykling