Good day,

While setting up (using Claude’ assistance) an internal PrivateASN “client” setup, we hit some snags on kernel protocols, but also some crashes.
See the report from Claude about the crashes. Any better formats to report the analysis on this group?

Summary of report.

A kernel protocol whose krt_sys_start() fails ("Kernel table N already
registered by X") stays in PS_START with p->scan_timer == NULL. Any later
shutdown of that protocol (reconfiguration that restarts or removes it,
disable, SIGTERM, graceful restart) runs krt_shutdown() ->
krt_scan_timer_stop() -> tm_stop(NULL) and the daemon segfaults.

Reproducible unprivileged with a 12-line config on 3.3.3 and 3.4.0. It took
down two production routers on 2026-10-14 (once on configure, once on
SIGTERM, after the SIGTERM path had already withdrawn all routes).

Versions

Reproducer

a.conf:

log stderr all;
router id 192.0.2.1;
ipv4 table t2;
protocol device { }
protocol kernel k1 {
  kernel table 254;
  ipv4 { table master4; import none; export none; };
}
protocol kernel k2 {
  kernel table 254;
  metric 10;
  ipv4 { table t2; import none; export all; };
}

b.conf = a.conf with k2 changed to kernel table 101; (no metric).

cp a.conf run.conf
unshare -rn sleep 60 &
NSP=$!
nsenter -t $NSP -U -n --preserve-credentials --wd=$PWD ip link set lo up
nsenter -t $NSP -U -n --preserve-credentials --wd=$PWD bird -f -c run.conf -s ctl -P pid &
nsenter -t $NSP -U -n --preserve-credentials --wd=$PWD birdc -s ctl show protocols
cp b.conf run.conf
nsenter -t $NSP -U -n --preserve-credentials --wd=$PWD birdc -s ctl configure

bird -p accepts a.conf. At start:

<ERR> k2: Kernel table 254 already registered by k1

show protocols shows k2 in state start. The configure logs

<INFO> Reconfiguring
<INFO> Restarting protocol k2

and birdc reports "Connection closed by server"; bird is gone (SIGSEGV).
birdc -s ctl disable k2 instead of the configure gives the same result.

version configure (table 254 -> 101) disable k2
3.3.3-cznic.1 SIGSEGV SIGSEGV
3.4.0-cznic.1 SIGSEGV SIGSEGV

Observed in production (3.3.3, two routers, 2026-10-14)

Router A, birdc configure moving the stuck protocol to its own kernel table:

22:52:13 bird: Reconfiguring
22:52:13 bird: Restarting protocol kuls4
22:52:13 kernel: bird[1814]: segfault at 30 ip 0000555b82c5d994 sp 00007ffdf1b62828 error 4 in bird[72994,555b82c00000+116000]

Router B, SIGTERM (init script restart) with the same stuck protocol still present:

23:37:49 bird: Shutting down
23:38:14 kernel: bird[1648]: segfault at 30 ip 0000559ad06c7994 sp 00007ffd3ea32008 error 4 in bird[72994,559ad066a000+116000]

Same text offset 0x72994 both times. With bird3-dbgsym 3.3.3-cznic.1,
addr2line resolves it to tm_stop() lib/timer.c:133 (if (!t->expires)).
Consistent with the fault address: struct timer = resource (32 bytes) +
hook (8) + data (8), so expires is at offset 0x30 of a NULL timer.
Both faults are on the main-thread stack (sp 0x7ffd...).

On router B the 25 s between "Shutting down" and the fault were the regular
shutdown withdrawing ~1M kernel routes and ~100 BGP sessions on the main loop;
the crash came when the stuck protocol's turn came.

Analysis (3.3.3 line numbers)

  1. sysdep/linux/netlink.c:2798 krt_sys_start(): HASH_FIND(nl_table_map, af,
    table id) finds the other kernel protocol, logs "Kernel table %u already
    registered by %s" and returns 0. metric is not part of the key.

  2. sysdep/unix/krt.c:946 krt_start(): line 969 if (!krt_sys_start(p)) { rem_node(&p->krt_node); return PS_START; } runs before line 975
    krt_scan_timer_start(p). p->scan_timer is only ever created inside
    krt_scan_timer_start() (tm_new_init), so it stays NULL. The protocol
    remains in PS_START; nothing else happens, it ran 34 min like that.

  3. sysdep/unix/krt.c:984 krt_shutdown(): line 988 krt_scan_timer_stop(p) is
    the first statement. krt.c:690 krt_scan_timer_stop() calls
    tm_stop(p->scan_timer) with NULL; lib/timer.c:133 reads t->expires.
    The guard if (p->p.proto_state == PS_START) return PS_FLUSH; is at
    line 990, after the call.

  4. On hosts where strict netlink dumps are unavailable (krt_use_shared_scan(),
    krt_scan_all_tables = 1) the same statement takes the other branch:
    krt_scan_all_timer_stop() with ASSERT(krt_scan_all_count > 0) and a
    decrement that this protocol never matched with an increment. Not tested.

Every shutdown path reaches krt_shutdown(): protocol restart or removal on
reconfigure, disable, SIGTERM, and graceful restart (PDC_CMD_GR_DOWN is
only consulted after the timer stop). Once a kernel protocol is in this state
there is no way to end it without the fault.

Tested change

Moving the PS_START check above the timer stop (krt-shutdown.patch, against
the 3.3.3 release tarball):

--- sysdep/unix/krt.c.orig
+++ sysdep/unix/krt.c
@@ -985,11 +985,11 @@
 {
   struct krt_proto *p = (struct krt_proto *) P;

-  krt_scan_timer_stop(p);
-
   if (p->p.proto_state == PS_START)
     return PS_FLUSH;

+  krt_scan_timer_stop(p);
+
   /* FIXME we should flush routes even when persist during reconfiguration */
   if ((p->sync_state != KPS_INIT) && !KRT_CF->persist && (P->down_code != PDC_CMD_GR_DOWN))
   {

With it, the reproducer survives both cases: after configure k2 comes up on
table 101; after disable k2 is down and the daemon keeps running.

Two remarks on how we got there:

Second, separate crash on router A (not reproduced)

After crash 1, router A was started fresh (22:53:54) with the conflict
removed (kuls4/6 on kernel table 101, all four kernel protocols up).
78 s later, with no CLI activity, it segfaulted again:

22:55:12 kernel: bird[13330]: segfault at 30 ip 0000562a8033ea23 sp 00007fc85c2462c0 error 4 in bird[6ea23,562a802e5000+116000]

Text offset 0x6ea23 -> mb_alloc() lib/resource.c:452,
ASSERT_DIE(DG_IS_LOCKED(p->domain)). struct pool = resource (32 bytes) +
TLIST_LIST(resource) inside (16 bytes), so domain is at offset 0x30: the
pool pointer passed to mb_alloc() was NULL. (A NULL p->domain would fault at
0x28, the order field behind the 40-byte mutex in struct domain_generic.)

Unlike crash 1, sp is a thread stack (0x7fc8...), i.e. a protocol or table
loop, not the main loop where the kernel protocols run.

State at the time:

I could not find what called mb_alloc() with a NULL pool: no core dump
(the daemon is not dumpable on that host), no backtrace. nest/proto.c sets
p->pool only in proto_start() (line 1690) and clears it after PS_DOWN_XX
(line 2884). The June 2026 ROA-autoreload / reload-during-GR fixes
(24194edaa, 7a7ebdcc0, 0f5e8e988, b18793719 on stable-v3.3) are in 3.3.3.

The only correlation I have: this configuration (the extra kernel protocol
on table 101, its BIRD table and pipe) was running for the first time. The
same configuration without that kernel protocol (table, pipe and BGP
sessions kept) has been stable on both routers since, through several
restarts and reconfigurations. The 12-line reproducer above does not cover
this case (no BGP, RPKI or graceful restart). I can arrange a core dump for
a future occurrence if useful.

Hendrik Visage
Director/Owner
HeViS.Co Systems t/a Envisage Cloud Solutions
hvisage@hevis.co.za
GSM/SMS/Signal: +27-84-612-5345
InstantMessenger: https://t.me/hvisage