Hello, we know about this problem and we are discussing in the core team what to do with that as it is not easy to fix correctly in a way which would stay long-term maintainable.
For now, my expectation is that an improvement will land in 3.5.0 and probably won't be backported. The 3.1.x line should be unaffected, the msl code has been added in 3.3.0.
We have seen some patches already trying to fix that but all of them have some drawbacks which we hope to overcome by a different approach. (There are some rather crazy race conditions which we prefer to avoid.)
Sales/marketing note. It would be actually very helpful if the networks running BIRD with lots of BGP sessions had a support contract. Performance analysis and multithreaded development is a laborous and hardware-intensive process, and this is exactly where your money would go – to those people who are familiar with the whole codebase and can fix things. Check
https://bird.nic.cz/commercial-services/ for more info.
I hope this helps.
Happy routing!
Maria
On November 12, 2026 6:26:18 AM GMT+01:00, Marcelo Balbinot <balbinot.marcelo@gmail.com> wrote:
Hello,
On a BIRD 3 route reflector the main thread runs at 100% CPU all the time, and
almost all of it is in msl_cleanup().
Setup:
- ~2,700 BGP sessions (iBGP rr clients and eBGP multihop)
- threads 20 on a 32 vCPU VM; no IGP, only BGP and static reject routes
- worker threads at 5-12% each
Measurements:
- BIRD 3.3.2, process up 2h19m: total CPU 15h26m. perf on the main thread:
94% at a single address in msl_cleanup_full_heads() (lib/mslab.c:723, the
walk over full_heads).
- BIRD 3.4.0, about 10 minutes after a restart, perf on the main thread:
98.7% in msl_cleanup (57k samples in 15 s).
- In 8 of our 12 crash cores, the main thread was inside msl_cleanup(). Two
cores are aborts on asserts in that code: lib/mslab.c:731
(ASSERT_DIE(num_full < s->objs_per_slab)) and lib/event.c:332.
How we read the code: msl_free() sends the cleanup event whenever a page
leaves the full state (or its last object is freed), and every
msl_cleanup_full_heads() run walks the whole full_heads list. With a large
attribute cache and constant churn from thousands of sessions, the cleanup is
rescheduled all the time and each run is O(number of full pages).
We reduced our own churn (empty "igp table" on all BGP channels, since this RR
never resolves next hops), which removed most next-hop re-evaluation. The main
thread is still saturated in msl_cleanup.
Is this a known limitation of the mslab cleanup? Is there a way to tune it, or
a fix we could test? We can run debug builds or collect more perf data.
Disclosure: I used an LLM (Claude) to help analyse the perf and core data.
The numbers above are from our production system.
Best regards,
Marcelo Balbinot