Hi, I have one operator with a full BGP table (one BGP session) and one IX (two BGP sessions, one BGP session each, to two different ISP routers). I need to configure two BGP routers based on Ubuntu + Bird 2/3 with iBGP. Could someone share this configuration and point out what to watch out for when traffic is ~1-2 Gbps? Thanks in advance.
I have run bird1 as an SP router (peering and transit, under 10G), bird2 is prob what you want. I have not touched bird3 yet. Im using bird2+strongswan mostly as an ipsecvti router these days but works well. Typically we like to do pci-pass thru on the host, to a vm, and pass thru a pair of 10G nics or 1 single 10G nic with 8021q and run rtr on a stick for workloads needing > 2Gbit/sec or 200K pps. ubuntu would not be my 1st choice for a router. I guess as the host it would be fine but typically a stripped down debian or devuan vm would be my choice as your rtr vm. for ipv6 esp. there are some kernel sysctls you will want to enable when putting full v6 tables into your fib. for public peering (exchanges) there are l2 proto rules you need to follow (like disabling l2 control protos and lldp) and you might want to practice hardcoding your mac addr so if you ever change anything, you know how to keep your peering fabric MAC addr the same pre and post (otherwise an outage and ticket to update your mac with the peering fabric provider). I have seen with debian 12/13 some ucd snmp package issues where the snmp daemon would hang and need a kill -9 and restart occasionally when taking a few full feeds. bfd works pretty well with bird2 and you want that for indirect loss of lite detection on your bgp sessions. I have had good results with these setting over ipsec+vti tuns: protocol bfd { multihop { interval 750 ms; multiplier 4; }; } cpu-wise my software routers have 2 vpu. To keep jitter to a min, you can allocate dedicated cpus to your vm if you want. The standard design of loopbacks and ip unnumbered for links that we use on h/w routers is a little tricky on debian vms. I use this to make sure my loopback always comes up: auto dummy0 iface dummy0 inet static address 100.120.0.110/32 pre-up modprobe dummy pre-up [ ! -d /sys/class/net/dummy0 ] && ip link add dummy0 type dummy || true pre-up sysctl -p /etc/sysctl.d/zz_local_sysctl.conf up ip link set dummy0 up And then run ospf to get that loopback into IBGP and term all ibgp sessions on the loopback. ip unnumbered works but basically you just put the same address as dummy0 on all your links. Looks odd but works. If ip unn. is not your thing then allocating /31s for your p2p links should work fine. + Jon Larsen: CTO Richweb, Inc. + Richweb.com: Cloud/Route/Switch/MSP Experts since 1995 + GnuPG Public Key: http://jlarsen.richweb.com/jlarsen.gpg + Business: (804) 368-0421 x 101; Mobile: (804) 747-8592
Hoi Mike, On 2025-12-08 17:41, Mike Neo wrote:
Could someone share this configuration and point out what to watch out for when traffic is ~1-2 Gbps? I wrote up the IPng Networks routing policy (including the iBGP and eBGP filtering) which addresses several gotchas in the DFZ: https://ipng.ch/s/articles/2021/11/14/case-study-bgp-routing-policy/
Of course, peering policy is a matter of 'taste' and what works for AS8298 may not work for everybody. But at least a few things like bogon filtering, RPKI, and BGP communities are useful concepts to read up on. groet, Pim -- Pim van Pelt <pim@ipng.ch> PBVP1-RIPE https://ipng.ch/
On 8 Dec 2025, at 17:41, Mike Neo <neomikemac@gmail.com> wrote:
Hi,
I have one operator with a full BGP table (one BGP session) and one IX (two BGP sessions, one BGP session each, to two different ISP routers).
I need to configure two BGP routers based on Ubuntu + Bird 2/3 with iBGP.
Could someone share this configuration and point out what to watch out for when traffic is ~1-2 Gbps?
The exact configuration comes mostly down to taste. You can get inspiration from https://github.com/coloclue/kees which has a good set of examples. And yes, automate from day 1, you and others will thank yourself in the years ahead ;) If you have specific questions do not hesitate to ask, many folks started either from scratch or build up their configs over the years. Most defaults will mostly just work given you have enough CPU, some memory and decent NIC(s). You could for fun look at Pim's VPP setup (https://ipng.ch/s/articles/2023/12/17/debian-on-ipngs-vpp-routers/ and other articles he wrote about it), though that is not needed unless you want to squeeze latency to a minimum. But without VPP one could also dedicate CPUs to interupts (avoid irqbalance for that matter) and to NIC queues. At 1-2 Gbps, it will mostly help you with latency/jitter though, not much with speed. Definitely look at https://bgpfilterguide.nlnog.net <https://bgpfilterguide.nlnog.net/> and implement the policies that make sense for you. Signup for peeringdb.com <http://peeringdb.com/> so that your contacts can be found easily and others can peer better with you. Use bgpq4 for generating BGP prefix lists. Use RPKI ROV and per bgpfilterguide filter these + ensure you got ROA for your own prefixes. Use OTC attribute to avoid accidental route leaks from/to peers/transits. For BCP38 if you can setup a Spoofer node: https://www.caida.org/projects/spoofer/ and then sign up for https://www.manrs.org/ Note that with nftables, if you have full table you can do something like: ``` # Drop when the source route is incorrect fib saddr . iif oif eq 0 log prefix "RPF: " counter drop ``` But be aware the moment you have multiple interfaces where the packets can come from/to and only 1 route is installed, that would cause issues with the other interface. Sign up for https://bgp.tools/ so that you get monitoring from the outside and one can publish BGP Communities there and in NLNOG Ring: https://github.com/NLNOG/lg.ring.nlnog.net and when you send BGP updates there it will nicely show up in their interace: https://lg.ring.nlnog.net <https://lg.ring.nlnog.net/> RIPE Atlas and RIS clients in your network will also allow one to easily spot-check connectivity, or even set up permanent alerts (by having nodes in your network you earn credits, thus you can automate queries with those earned credits) Do not forget monitoring, bgp.tools helps there, but definitely check that all your peers are in a happy state with a monitoring tool. Exposing a looking glass web interface can be useful. I wrote a custom one https://lg.as57777.net <https://lg.as57777.net/> but pre-packaged ones are available and allow easy debugging (I need to publish the code for mine properly one day though). A quick 'connectivity check from multiple points' like I have for https://debug.massars.net <https://debug.massars.net/> can also be useful to spot check if things are working, yes, just some javascript polling a few distributed http servers, but that allows one to quickly see what works and what does not from your client vantage point. Smokeping and alike can be great too for figuring out which links might be showing issues. See https://massars.net/details/ for a long list of things I have done with links to the projects involved. Also https://massars.net/design/ has some more details on how/what I did. Regards, Jeroen
Hello, Thank you very much for all the responses and insights to my previous question — I really appreciate the feedback. At the moment, I am running a very simple Bird v1 setup based on a single physical router, with: - one BGP session to my ISP, and - two BGP sessions to IX route servers. I'm attaching the current configuration for reference. I'm now considering migrating to Bird v2 and deploying two routers, and I am thinking about one of the following designs: 1) Two routers with iBGP In this model: - the BGP session to the ISP would be established on r1, - one BGP session to the IX would be on r1, and another one on r2, - r1 and r2 would be connected via iBGP, - in case of an r1 failure, VRRP would move the IP address to r2 and the BGP session to the ISP would be established on r2. What I'm not fully sure about in this scenario is whether splitting the IX BGP sessions across two routers could cause any issues, such as asymmetric routing or other unexpected behavior. For this option, I have prepared a Bird v2 configuration — I would be very grateful for any comments or suggestions. 2) Two routers without iBGP In this option, I would go with classic active/passive redundancy: - only one router has BGP sessions to the ISP and IXs (similar to the current setup), - in case of an r1 failure, IP addresses are moved to r2 and BGP sessions to the ISP and IXs are established on r2 by birdc enable protocol ISP. In this model, I fully accept that there will be a traffic interruption during failover, including BGP session establishment and table download. For this approach, I am attaching a migrated configuration from Bird v1 to Bird v2. I would also appreciate any feedback on this setup. Which approach would you recommend in practice? Are there any obvious pitfalls or best practices I should consider for either design? Thank you in advance for your advice. Best regards, Mike wt., 9 gru 2025 o 09:43 Jeroen Massar <jeroen@massar.ch> napisał(a):
On 8 Dec 2025, at 17:41, Mike Neo <neomikemac@gmail.com> wrote:
Hi,
I have one operator with a full BGP table (one BGP session) and one IX (two BGP sessions, one BGP session each, to two different ISP routers).
I need to configure two BGP routers based on Ubuntu + Bird 2/3 with iBGP.
Could someone share this configuration and point out what to watch out for when traffic is ~1-2 Gbps?
The exact configuration comes mostly down to taste. You can get inspiration from https://github.com/coloclue/kees which has a good set of examples. And yes, automate from day 1, you and others will thank yourself in the years ahead ;)
If you have specific questions do not hesitate to ask, many folks started either from scratch or build up their configs over the years.
Most defaults will mostly just work given you have enough CPU, some memory and decent NIC(s).
You could for fun look at Pim's VPP setup ( https://ipng.ch/s/articles/2023/12/17/debian-on-ipngs-vpp-routers/ and other articles he wrote about it), though that is not needed unless you want to squeeze latency to a minimum.
But without VPP one could also dedicate CPUs to interupts (avoid irqbalance for that matter) and to NIC queues.
At 1-2 Gbps, it will mostly help you with latency/jitter though, not much with speed.
Definitely look at https://bgpfilterguide.nlnog.net < https://bgpfilterguide.nlnog.net/> and implement the policies that make sense for you.
Signup for peeringdb.com <http://peeringdb.com/> so that your contacts can be found easily and others can peer better with you.
Use bgpq4 for generating BGP prefix lists. Use RPKI ROV and per bgpfilterguide filter these + ensure you got ROA for your own prefixes. Use OTC attribute to avoid accidental route leaks from/to peers/transits.
For BCP38 if you can setup a Spoofer node: https://www.caida.org/projects/spoofer/ and then sign up for https://www.manrs.org/
Note that with nftables, if you have full table you can do something like: ``` # Drop when the source route is incorrect fib saddr . iif oif eq 0 log prefix "RPF: " counter drop ``` But be aware the moment you have multiple interfaces where the packets can come from/to and only 1 route is installed, that would cause issues with the other interface.
Sign up for https://bgp.tools/ so that you get monitoring from the outside and one can publish BGP Communities there and in NLNOG Ring: https://github.com/NLNOG/lg.ring.nlnog.net and when you send BGP updates there it will nicely show up in their interace: https://lg.ring.nlnog.net <https://lg.ring.nlnog.net/>
RIPE Atlas and RIS clients in your network will also allow one to easily spot-check connectivity, or even set up permanent alerts (by having nodes in your network you earn credits, thus you can automate queries with those earned credits)
Do not forget monitoring, bgp.tools helps there, but definitely check that all your peers are in a happy state with a monitoring tool. Exposing a looking glass web interface can be useful. I wrote a custom one https://lg.as57777.net <https://lg.as57777.net/> but pre-packaged ones are available and allow easy debugging (I need to publish the code for mine properly one day though). A quick 'connectivity check from multiple points' like I have for https://debug.massars.net <https://debug.massars.net/> can also be useful to spot check if things are working, yes, just some javascript polling a few distributed http servers, but that allows one to quickly see what works and what does not from your client vantage point. Smokeping and alike can be great too for figuring out which links might be showing issues.
See https://massars.net/details/ for a long list of things I have done with links to the projects involved. Also https://massars.net/design/ has some more details on how/what I did.
Regards, Jeroen
On 28 Jan 2026, at 17:35, Mike Neo <neomikemac@gmail.com> wrote:
Hello,
Thank you very much for all the responses and insights to my previous question — I really appreciate the feedback.
At the moment, I am running a very simple Bird v1 setup based on a single physical router, with: - one BGP session to my ISP, and - two BGP sessions to IX route servers.
I'm attaching the current configuration for reference.
I'm now considering migrating to Bird v2 and deploying two routers, and I am thinking about one of the following designs:
1) Two routers with iBGP
That is the correct model: let BGP handle the failover
In this model: - the BGP session to the ISP would be established on r1, - one BGP session to the IX would be on r1, and another one on r2,
Note that this requires two ports from the IX; often at a separate charge, but depends on the IX.
- r1 and r2 would be connected via iBGP, - in case of an r1 failure, VRRP would move the IP address to r2 and the BGP session to the ISP would be established on r2.
I would not do so. Why not have two transit connections to the upstream ISP?
What I'm not fully sure about in this scenario is whether splitting the IX BGP sessions across two routers could cause any issues, such as asymmetric routing or other unexpected behavior.
One problem can be if you lose iBGP and your internal network is not fully meshed and thus r1 cannot talk to r2 anymore. That can result in traffic arriving at r1 for something needing to go to r2 to not arrive properly. Thus ensure that r1 <-> r2 has multiple physical paths. alternative: stop announcing your prefix from one of the two disconnected routers so that you know where to send traffic. The other possible problem is related to state tracking: do not do state tracking on routers, as packets might one point come from one link and the next from the other, or over r2 instead of r1. If you build state, then those states from the other interface won't work. Hence state tracking on routers is a no-go unless you become really creative, but still state == bad, thus just avoid it. If you have something that needs to be firewalled, do it either on-device; or separately on a VRRP'd and thus state-synced firewall; but not on something that directly speaks BGP with many ports. (a firewall with just a upstream + downstream port makes such a setup a bit more doable; but still state-sync is tricky for active-active; noting that with active-passive and state you break all connections when failover does happen).
For this option, I have prepared a Bird v2 configuration — I would be very grateful for any comments or suggestions.
2) Two routers without iBGP
In this option, I would go with classic active/passive redundancy: - only one router has BGP sessions to the ISP and IXs (similar to the current setup), - in case of an r1 failure, IP addresses are moved to r2 and BGP sessions to the ISP and IXs are established on r2 by birdc enable protocol ISP.
Avoid this. There are no live BGP state-sharing implementations at the moment unfortunately and thus session re-establishment would happen and other such things. The complexity is not worth it and can only cause issues. It would require doing some trickery with either connecting a switch or a bridge to the IX which is likely frowned upon. It would also mean you have to spoof/borrow MAC addresses for VRRP, as most IXs do single-MAC per port. Better to have two separate ports (but for most IXes that means paying for separate port) and do BGP from each, identically (minus the MAC + IP addresses and loopbacks involved). Your example configuration is very very minimal, thus hard to comment but the following: First off I suggest generating the configuration, every body has their own choice of tools for that, many use ansible, others use a bunch of bash scripts. See for many good examples for instance: https://github.com/coloclue/kees/ And of course https://bgpfilterguide.nlnog.net <https://bgpfilterguide.nlnog.net/> which contains snippets for many router platforms. Using OTC to indicate customer/transport peerings, generating prefix filters from bgpq4 (IRR data), using ASPA validation, ROA checks (use stayrtr), max prefix limits based on peeringdb.com <http://peeringdb.com/> data and many other things should be included in a configuration. Greets, Jeroen
participants (4)
-
C. Jon Larsen -
Jeroen Massar -
Mike Neo -
Pim van Pelt