Hi Douglas,

I think one of the main concerns here is also TCP itself. If you want to migrate BGP in such a way that the peer does not notice anything, you need to preserve the TCP session. And this is the task to solve by itself. Otherwise there will be connection reestablish with the remote side, and it will cause route disturbance anyway. And if we able to migrate a TCP, there is a separate problem to migrate internal BGP state. Even if there are some solutions for TCP, I believe there is no support for such in BIRD.
I think I've heard of some VM hypervisor solutions when they run 2 synced VMs which should replicate one another and if one fails for some reason, the other catches up. I.e. they have the same memory state, etc. But it might be not a practical solution.
The other option might be Graceful Restart, when the peer should keep the state for some time as if connection did not failed. So we do not need to sync the internal TCP and BGP states among anycast peers, we can do a brand new BGP session. And if we are able to do it withing the graceful period, and exchange the required routes, it seems it should work.

Regards,
Alexander

On Thu, Oct 29, 2026, 10:44 Douglas Fischer <fischerdouglas@gmail.com> wrote:
Hello,
I had quite a hard time coming up with a subject line for this email.
I hope "BGP Session Stateful Failover across multiple BIRD instances" is sufficiently self-descriptive.

I should mention upfront that I am not aware if this functionality already exists in BIRD (or any other open-source routing daemon).

The goal is to optimize availability in a scenario where BGP sessions are disaggregated from the actual forwarding boxes.

Scenario 1—the most basic one—is straightforward:
- Anycast for a set of IPv4 and IPv6 addresses, propagated via IGP.
- Downstream BGP sessions configured as eBGP multihop, pointing to these Anycast IPs.
- Each node is configured with 100% of the peers and their respective policies.
- Metrics, such as IGP cost, determine which Anycast instance a given BGP client connects to.
- These nodes handle the BGP heavy lifting—managing route exchanges and applying filters for the downstream peers.
- They then propagate these routes to the forwarding devices, providing them in the most streamlined (filtered/optimized) form possible.

Up to this point—as long as the network remains stable—everything works well.

The problem arises when an event causes a specific set of downstream clients to switch from one Route Reflector (RR) node to another.

Initially, I assumed that this kind of "BGP session roaming" would primarily be triggered by the unavailability of a server running the Route Reflector. However, that is rarely the case.
More often, changes within the IGP—for various reasons—cause those BGP clients to shift to a different node.
This leads to session flaps, which in turn cause traffic flaps and significant churn during route recalculations.

I even tried to prevent the IGP from steering traffic from a given downstream client to a different node... but that only made things worse.
The conclusion I’ve reached is that the healthiest approach would be for a BGP session to be resumed from a different node, in such a way that the BGP client remains unaware that this has occurred.

---

Having outlined the problem and the desired solution...
I’m not sure how to actually implement this!
I don’t know if BIRD supports this, and if it does, I’m not sure how to configure it.

I am aware that "graceful restart" is a prerequisite for this kind of implementation.

However, I honestly don’t know how to make this "Anycast TCP" concept work within the context of BGP.
I imagine it would require some form of copying or replication of the active BGP session state table to the nodes that are currently in a passive state.
Logically, this would also likely necessitate a quorum mechanism to determine which node is the active one.

---

I am sharing this scenario—the problem and the intended solution—with you all.
I would welcome any guidance or suggestions as to whether you have encountered a solution for this issue.

P.S.1: I suspect that companies like Google and AWS, which utilize disaggregation in their peering scenarios, likely already implement some form of stateful failover for BGP sessions across multiple boxes.

P.S.2: There is a "Scenario 2" involving IP-in-IP tunnels and PBR (Policy-Based Routing) to make it appear as though the Reflector is directly connected to the EdgePeer.
However, I believe detailing that scenario would only add unnecessary complexity and distract from the core issue: stateful BGP session failover.

Thank you in advance.

--
Douglas Fernando Fischer
Engº de Controle e Automação