[PATCH] BGP: Fix bucket assert when encoding BMP route monitoring messages
Good afternoon, the 2 issues I reported last week (e038bf6 and 371531d) was resolved on my side in 3.3.1. Thank you! Today in CI, I'm seeing some assertion throws that affects both 3.3.0 and 3.3.1 with BMP (traces also in the linked CI log): - https://github.com/stepbrobd/rfm/actions/runs/27352866052/job/80819677363#st... - https://github.com/stepbrobd/rfm/actions/runs/27347885470/job/80814124422#st... Relevant config: - https://github.com/stepbrobd/rfm/blob/187c61620b3af6ec77e1cc730c6b6b37d174d2... I applied the same patch on 3.3.1 locally to unblock CI, and the current patch is based on thread-next bgp_create_update_bmp() encodes routes for BMP using a temporary bucket allocated on stack. Such buckets are not allocated from the bucket slab and have no ID assigned, so the assert at the beginning of bgp_delete_from_bucket() fails as soon as the NLRI encoder pops the prefix, aborting the daemon on the first route monitoring message that carries a real prefix. On BIRD 3.3.1:
Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020
Temporary BMP buckets always hold exactly one prefix and take the single-prefix path, which does not use the bucket ID at all, so allow them through the assert. Introduced in 9fa3e89476ed53b6a9d41931a8cdf9a8c675be27 "BGP: Compacting export prefix buckets" --- proto/bgp/attrs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proto/bgp/attrs.c b/proto/bgp/attrs.c index 6c38d4955..ee893d2c1 100644 --- a/proto/bgp/attrs.c +++ b/proto/bgp/attrs.c @@ -2063,7 +2063,7 @@ bgp_delete_from_bucket(struct bgp_ptx_private *c, struct bgp_bucket *b, u32 id) bgp_bucket_consistency(b); #endif - ASSERT_DIE(b->my_id); + ASSERT_DIE(b->my_id || b->bmp); u32 row_num = BUCKET_PREFIX_ROW(id); u32 pos = BUCKET_PREFIX_POS(id); ASSERT_DIE(b->last_pref_id); -- 2.54.0
Hii, thanks a lot for the detailed bug report and patch. We will look into it and fix it in the next release. At a first glance it really looks like a regression that sneaked to the release, because of our unfinished BMP test suit. Thanks again for your reporting effort, it helps a lot. Regards, David David Petera (he/him) | BIRD Tech Support | CZ.NIC, z.s.p.o. On 6/11/26 17:43, Yifei Sun wrote:
Good afternoon, the 2 issues I reported last week (e038bf6 and 371531d) was resolved on my side in 3.3.1. Thank you!
Today in CI, I'm seeing some assertion throws that affects both 3.3.0 and 3.3.1 with BMP (traces also in the linked CI log):
- https://github.com/stepbrobd/rfm/actions/runs/27352866052/job/80819677363#st... - https://github.com/stepbrobd/rfm/actions/runs/27347885470/job/80814124422#st...
Relevant config:
- https://github.com/stepbrobd/rfm/blob/187c61620b3af6ec77e1cc730c6b6b37d174d2...
I applied the same patch on 3.3.1 locally to unblock CI, and the current patch is based on thread-next
bgp_create_update_bmp() encodes routes for BMP using a temporary bucket allocated on stack. Such buckets are not allocated from the bucket slab and have no ID assigned, so the assert at the beginning of bgp_delete_from_bucket() fails as soon as the NLRI encoder pops the prefix, aborting the daemon on the first route monitoring message that carries a real prefix.
On BIRD 3.3.1:
Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020 Temporary BMP buckets always hold exactly one prefix and take the single-prefix path, which does not use the bucket ID at all, so allow them through the assert.
Introduced in 9fa3e89476ed53b6a9d41931a8cdf9a8c675be27
"BGP: Compacting export prefix buckets" --- proto/bgp/attrs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/proto/bgp/attrs.c b/proto/bgp/attrs.c index 6c38d4955..ee893d2c1 100644 --- a/proto/bgp/attrs.c +++ b/proto/bgp/attrs.c @@ -2063,7 +2063,7 @@ bgp_delete_from_bucket(struct bgp_ptx_private *c, struct bgp_bucket *b, u32 id) bgp_bucket_consistency(b); #endif
- ASSERT_DIE(b->my_id); + ASSERT_DIE(b->my_id || b->bmp); u32 row_num = BUCKET_PREFIX_ROW(id); u32 pos = BUCKET_PREFIX_POS(id); ASSERT_DIE(b->last_pref_id);
Good day, newly deployed FRR replacement, run into crashes that pointed to BMP and onc eBMP was disabled, BIRD3 stayed up and stable. Can enable bmp and “configure” but then the next restart/reload of a BGP session again triggers an `Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020` start BIRD with bmp enabled: ``` 2026-09-28T01:49:36.925255+02:00 flapping bird: Started 2026-09-28T01:49:46.842780+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020 2026-09-28T02:01:00.187600+02:00 flapping bird: Started 2026-09-28T02:01:11.036402+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020 (protocol bmp commented out here) 2026-09-28T02:01:31.121460+02:00 flapping bird: Started ... stable since, 94 sessions Established, no further assertions ``` enable this BMP config and birdc configure: ``` protocol bmp bgproutesio { station address ip 2a01:4f8:10a:3c21::2 port 45678; monitoring rib in pre_policy; tx buffer limit 512; } ``` then doing a birdc restart peer: ``` root@flapping:~# birdc restart NetWide_nap_v4_1 NetWide_nap_v4_1: restarted 2026-09-28T14:16:21.593306+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020 ``` Claude analysis: 3. The two BMP bugs interact — this is the new part (operator test, 2026-09-28) Test on flapping, BIRD 3.3.2, live: BMP added to a RUNNING BIRD and loaded with birdc configure does not crash — the station comes up and stays up. A fresh start with the same protocol bmp in the config crashes (the attrs.c:2020 assert). That is consistent with the other open BMP bug, and only with it: Thomas Holterbach, 9 Apr 2026, "BMP: no monitoring messages when enabled after BGP sessions (BIRD 3.2.1)": https://trubka.network.cz/archives/list/bird-users@network.cz/thread/YA4I3IX... — if the BGP sessions are Established before BMP is enabled, no monitoring messages are exported at all. Acknowledged by CZ.NIC (David Petera, 17 Apr 2026): "you are unfortunately correct that it is another bug in the BMP protocol in v3... We even found another one related to this while investigating the issue and we will fix them in the new version." So on 3.3.2 the no-refeed bug masks the assert: enable BMP at runtime and nothing is monitored, therefore nothing is encoded, therefore the sham bucket is never popped and the daemon survives. Start with BMP in the config and the initial feed (rt_export_subscribe(tab, all, …) in bmp_add_table(), proto/bmp/bmp.c:546-571) hands the encoder a real prefix, and it aborts on the first one. Operational consequence, which is worth stating on the list: "enable BMP at runtime" is not a workaround. The station connects and looks healthy while receiving no route monitoring at all, and the first restart, refeed or reconfigure that triggers a feed aborts the daemon. Confirmed 2026-09-28 14:11 SAST (diagnostics/2026-09-27-bird-phase1/flapping-bmp-runtime-enable-1411.typescript), on a router carrying ~1.1 M IPv4 + ~255 k IPv6 routes with constant churn: bgproutesio BMP --- up 14:04:45.930 Established Created: 14:04:45.350 Station address: 2a01:4f8:10a:3c21::2 Pending TX: 0 B (limit 512.0 MB) Session TX: 427 B Total TX: 427 B 427 bytes in 7 minutes. That is the Initiation message and nothing else: with 94 Established BGP sessions, the Peer Up messages alone (each carrying both OPEN messages) would be tens of kilobytes. So bmp_startup()'s walk over the existing BGP protocol states produced no peers, hence no streams, hence no route monitoring - and hence no sham bucket, no assert. birdc debug bgproutesio all then logged nothing for 20 s; the only BMP lines in syslog are the two Adding protocol bgproutesio from the two birdc configure runs. No Failed to request pre-policy ... import table disabled either (every channel has import table yes), so the streams were not rejected - they were never created. The fresh-start crash in the same syslog, 2 minutes before that: `2026-09-28T14:02:36.578293+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020` i.e. a seventh occurrence, on a 3.3.2 that had been running for hours. And the mechanism, confirmed at 14:16: with that same runtime-enabled BMP session up and idle at 427 B, restarting one small BGP protocol was enough: ``` root@flapping:~# birdc restart NetWide_nap_v4_1 NetWide_nap_v4_1: restarted 2026-09-28T14:16:21.593306+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020 ``` The protocol coming back up produces a Peer Up through the state-change path (bmp_proto_state_changed -> bmp_peer_up_inout, proto/bmp/bmp.c:1411), which DOES create the stream, which feeds, which encodes the first prefix, which aborts the daemon. So the no-refeed bug does not make a runtime-enabled BMP safe - it makes it quiet until the next session flap, and then it takes BIRD with it. On a busy edge router that is a matter of minutes. Recovery, for anyone who lands here: ASSERT_DIE aborts the process, and with persist yes on the kernel protocols the FIB stays, so forwarding limps on stale routes while BGP is gone. Comment out protocol bmp BEFORE starting BIRD again, or the fresh start aborts immediately. Their patch, for reference (identical in spirit to what the source reading here arrived at independently — they put b->bmp second, and their line number is 2063 because the patch is against thread-next, which has a bgp_bucket_consistency() block above the assert): ``` - ASSERT_DIE(b->my_id); + ASSERT_DIE(b->my_id || b->bmp); ``` --- Hendrik Visage Director/Owner HeViS.Co Systems t/a Envisage Cloud Solutions hvisage@hevis.co.za GSM/SMS/Signal: +27-84-612-5345 InstantMessenger: https://t.me/hvisage
Hi Hendrik, thanks for the report, we think that we have already fixed this bug in our codebase and it will be included in the upcoming patch release `v3.3.3` later this week. The relevant commit is https://gitlab.nic.cz/labs/bird/-/commit/5d5274f6d2ea16a9a93b6477f20aefbbf2c... if you want to take a look or patch your version before release. If you could confirm that the patch stops the crashes from happening in your case it would be appreciated, no worries if you rather wait for the release)) Hope this helps, David David Petera (he/him) | BIRD Tech Support | CZ.NIC, z.s.p.o. On 9/28/26 14:37, Hendrik Visage via Bird-users wrote:
Good day,
newly deployed FRR replacement, run into crashes that pointed to BMP and onc eBMP was disabled, BIRD3 stayed up and stable. Can enable bmp and “configure” but then the next restart/reload of a BGP session again triggers an |Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020|
start BIRD with bmp enabled:
|2026-09-28T01:49:36.925255+02:00 flapping bird: Started 2026-09-28T01:49:46.842780+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020 2026-09-28T02:01:00.187600+02:00 flapping bird: Started 2026-09-28T02:01:11.036402+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020 (protocol bmp commented out here) 2026-09-28T02:01:31.121460+02:00 flapping bird: Started ... stable since, 94 sessions Established, no further assertions |
enable this BMP config and birdc configure:
|protocol bmp bgproutesio { station address ip 2a01:4f8:10a:3c21::2 port 45678; monitoring rib in pre_policy; tx buffer limit 512; } |
then doing a birdc restart peer:
|root@flapping:~# birdc restart NetWide_nap_v4_1 NetWide_nap_v4_1: restarted 2026-09-28T14:16:21.593306+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020 |
Claude analysis:
3. The two BMP bugs interact — this is the new part (operator test, 2026-09-28)
Test on flapping, BIRD 3.3.2, live: BMP added to a RUNNING BIRD and loaded with birdc configure does not crash — the station comes up and stays up. A fresh start with the same protocol bmp in the config crashes (the attrs.c:2020 assert).
That is consistent with the other open BMP bug, and only with it:
Thomas Holterbach, 9 Apr 2026, "BMP: no monitoring messages when enabled after BGP sessions (BIRD 3.2.1)": https://trubka.network.cz/archives/list/bird-users@network.cz/thread/YA4I3IX... — if the BGP sessions are Established before BMP is enabled, no monitoring messages are exported at all. Acknowledged by CZ.NIC (David Petera, 17 Apr 2026): "you are unfortunately correct that it is another bug in the BMP protocol in v3... We even found another one related to this while investigating the issue and we will fix them in the new version." So on 3.3.2 the no-refeed bug masks the assert: enable BMP at runtime and nothing is monitored, therefore nothing is encoded, therefore the sham bucket is never popped and the daemon survives. Start with BMP in the config and the initial feed (rt_export_subscribe(tab, all, …) in bmp_add_table(), proto/bmp/bmp.c:546-571) hands the encoder a real prefix, and it aborts on the first one.
Operational consequence, which is worth stating on the list: "enable BMP at runtime" is not a workaround. The station connects and looks healthy while receiving no route monitoring at all, and the first restart, refeed or reconfigure that triggers a feed aborts the daemon.
Confirmed 2026-09-28 14:11 SAST (diagnostics/2026-09-27-bird-phase1/flapping-bmp-runtime-enable-1411.typescript), on a router carrying ~1.1 M IPv4 + ~255 k IPv6 routes with constant churn:
bgproutesio BMP --- up 14:04:45.930 Established Created: 14:04:45.350 Station address: 2a01:4f8:10a:3c21::2 Pending TX: 0 B (limit 512.0 MB) Session TX: 427 B Total TX: 427 B 427 bytes in 7 minutes. That is the Initiation message and nothing else: with 94 Established BGP sessions, the Peer Up messages alone (each carrying both OPEN messages) would be tens of kilobytes. So bmp_startup()'s walk over the existing BGP protocol states produced no peers, hence no streams, hence no route monitoring - and hence no sham bucket, no assert. birdc debug bgproutesio all then logged nothing for 20 s; the only BMP lines in syslog are the two Adding protocol bgproutesio from the two birdc configure runs. No Failed to request pre-policy ... import table disabled either (every channel has import table yes), so the streams were not rejected - they were never created.
The fresh-start crash in the same syslog, 2 minutes before that:
|2026-09-28T14:02:36.578293+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020| i.e. a seventh occurrence, on a 3.3.2 that had been running for hours.
And the mechanism, confirmed at 14:16: with that same runtime-enabled BMP session up and idle at 427 B, restarting one small BGP protocol was enough:
|root@flapping:~# birdc restart NetWide_nap_v4_1 NetWide_nap_v4_1: restarted 2026-09-28T14:16:21.593306+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020 |
The protocol coming back up produces a Peer Up through the state-change path (bmp_proto_state_changed -> bmp_peer_up_inout, proto/bmp/bmp.c:1411), which DOES create the stream, which feeds, which encodes the first prefix, which aborts the daemon. So the no-refeed bug does not make a runtime-enabled BMP safe - it makes it quiet until the next session flap, and then it takes BIRD with it. On a busy edge router that is a matter of minutes.
Recovery, for anyone who lands here: ASSERT_DIE aborts the process, and with persist yes on the kernel protocols the FIB stays, so forwarding limps on stale routes while BGP is gone. Comment out protocol bmp BEFORE starting BIRD again, or the fresh start aborts immediately.
Their patch, for reference (identical in spirit to what the source reading here arrived at independently — they put b->bmp second, and their line number is 2063 because the patch is against thread-next, which has a bgp_bucket_consistency() block above the assert):
|- ASSERT_DIE(b->my_id); + ASSERT_DIE(b->my_id || b->bmp); | ------------------------------------------------------------------------
Hendrik Visage Director/Owner HeViS.Co Systems t/a Envisage Cloud Solutions hvisage@hevis.co.za GSM/SMS/Signal: +27-84-612-5345 InstantMessenger: https://t.me/hvisage
Thank you David! I’ll try, but might not make it before the end of the week to setup build env and compile :(=) --- Hendrik Visage Director/Owner HeViS.Co Systems t/a Envisage Cloud Solutions hvisage@hevis.co.za GSM/SMS/Signal: +27-84-612-5345 InstantMessenger: https://t.me/hvisage On 29 Sep 2026, at 13:20, David Petera via Bird-users wrote:
Hi Hendrik,
thanks for the report, we think that we have already fixed this bug in our codebase and it will be included in the upcoming patch release `v3.3.3` later this week.
The relevant commit is https://gitlab.nic.cz/labs/bird/-/commit/5d5274f6d2ea16a9a93b6477f20aefbbf2c... if you want to take a look or patch your version before release.
If you could confirm that the patch stops the crashes from happening in your case it would be appreciated, no worries if you rather wait for the release))
Hope this helps, David
David Petera (he/him) | BIRD Tech Support | CZ.NIC, z.s.p.o.
On 9/28/26 14:37, Hendrik Visage via Bird-users wrote:
Good day,
newly deployed FRR replacement, run into crashes that pointed to BMP and onc eBMP was disabled, BIRD3 stayed up and stable. Can enable bmp and “configure” but then the next restart/reload of a BGP session again triggers an |Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020|
start BIRD with bmp enabled:
|2026-09-28T01:49:36.925255+02:00 flapping bird: Started 2026-09-28T01:49:46.842780+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020 2026-09-28T02:01:00.187600+02:00 flapping bird: Started 2026-09-28T02:01:11.036402+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020 (protocol bmp commented out here) 2026-09-28T02:01:31.121460+02:00 flapping bird: Started ... stable since, 94 sessions Established, no further assertions |
enable this BMP config and birdc configure:
|protocol bmp bgproutesio { station address ip 2a01:4f8:10a:3c21::2 port 45678; monitoring rib in pre_policy; tx buffer limit 512; } |
then doing a birdc restart peer:
|root@flapping:~# birdc restart NetWide_nap_v4_1 NetWide_nap_v4_1: restarted 2026-09-28T14:16:21.593306+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020 |
Claude analysis:
3. The two BMP bugs interact — this is the new part (operator test, 2026-09-28)
Test on flapping, BIRD 3.3.2, live: BMP added to a RUNNING BIRD and loaded with birdc configure does not crash — the station comes up and stays up. A fresh start with the same protocol bmp in the config crashes (the attrs.c:2020 assert).
That is consistent with the other open BMP bug, and only with it:
Thomas Holterbach, 9 Apr 2026, "BMP: no monitoring messages when enabled after BGP sessions (BIRD 3.2.1)": https://trubka.network.cz/archives/list/bird-users@network.cz/thread/YA4I3IX... — if the BGP sessions are Established before BMP is enabled, no monitoring messages are exported at all. Acknowledged by CZ.NIC (David Petera, 17 Apr 2026): "you are unfortunately correct that it is another bug in the BMP protocol in v3... We even found another one related to this while investigating the issue and we will fix them in the new version." So on 3.3.2 the no-refeed bug masks the assert: enable BMP at runtime and nothing is monitored, therefore nothing is encoded, therefore the sham bucket is never popped and the daemon survives. Start with BMP in the config and the initial feed (rt_export_subscribe(tab, all, …) in bmp_add_table(), proto/bmp/bmp.c:546-571) hands the encoder a real prefix, and it aborts on the first one.
Operational consequence, which is worth stating on the list: "enable BMP at runtime" is not a workaround. The station connects and looks healthy while receiving no route monitoring at all, and the first restart, refeed or reconfigure that triggers a feed aborts the daemon.
Confirmed 2026-09-28 14:11 SAST (diagnostics/2026-09-27-bird-phase1/flapping-bmp-runtime-enable-1411.typescript), on a router carrying ~1.1 M IPv4 + ~255 k IPv6 routes with constant churn:
bgproutesio BMP --- up 14:04:45.930 Established Created: 14:04:45.350 Station address: 2a01:4f8:10a:3c21::2 Pending TX: 0 B (limit 512.0 MB) Session TX: 427 B Total TX: 427 B 427 bytes in 7 minutes. That is the Initiation message and nothing else: with 94 Established BGP sessions, the Peer Up messages alone (each carrying both OPEN messages) would be tens of kilobytes. So bmp_startup()'s walk over the existing BGP protocol states produced no peers, hence no streams, hence no route monitoring - and hence no sham bucket, no assert. birdc debug bgproutesio all then logged nothing for 20 s; the only BMP lines in syslog are the two Adding protocol bgproutesio from the two birdc configure runs. No Failed to request pre-policy ... import table disabled either (every channel has import table yes), so the streams were not rejected - they were never created.
The fresh-start crash in the same syslog, 2 minutes before that:
|2026-09-28T14:02:36.578293+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020| i.e. a seventh occurrence, on a 3.3.2 that had been running for hours.
And the mechanism, confirmed at 14:16: with that same runtime-enabled BMP session up and idle at 427 B, restarting one small BGP protocol was enough:
|root@flapping:~# birdc restart NetWide_nap_v4_1 NetWide_nap_v4_1: restarted 2026-09-28T14:16:21.593306+02:00 flapping bird: Assertion 'b->my_id' failed at proto/bgp/attrs.c:2020 |
The protocol coming back up produces a Peer Up through the state-change path (bmp_proto_state_changed -> bmp_peer_up_inout, proto/bmp/bmp.c:1411), which DOES create the stream, which feeds, which encodes the first prefix, which aborts the daemon. So the no-refeed bug does not make a runtime-enabled BMP safe - it makes it quiet until the next session flap, and then it takes BIRD with it. On a busy edge router that is a matter of minutes.
Recovery, for anyone who lands here: ASSERT_DIE aborts the process, and with persist yes on the kernel protocols the FIB stays, so forwarding limps on stale routes while BGP is gone. Comment out protocol bmp BEFORE starting BIRD again, or the fresh start aborts immediately.
Their patch, for reference (identical in spirit to what the source reading here arrived at independently — they put b->bmp second, and their line number is 2063 because the patch is against thread-next, which has a bgp_bucket_consistency() block above the assert):
|- ASSERT_DIE(b->my_id); + ASSERT_DIE(b->my_id || b->bmp); | ------------------------------------------------------------------------
Hendrik Visage Director/Owner HeViS.Co Systems t/a Envisage Cloud Solutions hvisage@hevis.co.za GSM/SMS/Signal: +27-84-612-5345 InstantMessenger: https://t.me/hvisage
participants (3)
-
David Petera -
Hendrik Visage -
Yifei Sun