<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Monaco;
panose-1:0 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:black">Hi BIRD users,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Does anyone know whether a BGP shared secret can be rotated without incurring any network downtime? I did some testing with the BGP password functionality offered and it appears that any update to the BGP password
configuration incurs a brief network outage with both existing/new connections. It seems like something about the way BIRD is restarting is leading to it pulling down learned routes immediately as opposed to letting them live according to the timeout setting.
Does BIRD flush all routes it has learned when this configuration changes? Here is a brief excerpt to demonstrate the outage. Take note that the network disruption precisely matches the timestamp at which BIRD is reconfigured:<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D"># Logs from calico-node-4w4bv (10.95.14.104)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">2022-06-29 18:26:18.836 [INFO][60] confd/client.go 1415: Trigger to recheck BGP peers following possible password update</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">2022-06-29 18:26:18.836 [INFO][60] confd/client.go 250: Recompute v1 BGP peerings</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">2022-06-29 18:26:18.836 [INFO][60] confd/client.go 949: Recompute BGP peerings:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Reconfiguration requested by SIGHUP</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">2022-06-29 18:26:18.844 [INFO][60] confd/resource.go 278: Target config /etc/calico/confd/config/bird.cfg has been updated due to change in key:
/calico/bgp/v1/global</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Reconfiguring</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: device1: Reconfigured</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: direct1: Reconfigured</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Restarting protocol Mesh_10_95_14_105</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_105: Shutting down</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_105: State changed to stop</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Restarting protocol Mesh_10_95_14_110</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_110: Shutting down</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_110: State changed to stop</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_105: State changed to down</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_105: Initializing</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_105: Starting</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_105: State changed to start</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_110: State changed to down</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_110: Initializing</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_110: Starting</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_110: State changed to start</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Reconfigured</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_110: Connected to table master</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_110: State changed to feed</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_110: State changed to up</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_105: Connected to table master</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_105: State changed to feed</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">bird: Mesh_10_95_14_105: State changed to up</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">2022-06-29 18:26:36.079 [INFO][58] monitor-addresses/autodetection_methods.go 117: Using autodetected IPv4 address 10.95.14.104/26 on matching interface
eth0</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">2022-06-29 18:26:54.537 [INFO][62] felix/summary.go 100: Summarising 9 dataplane reconciliation loops over 1m2.1s: avg=5ms longest=21ms (resync-filter-v4,resync-nat-v4)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D"># Connection Tester Daemonset (hitting an echoserver twice a second or so)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Wed Jun 29 18:26:17 UTC 2022</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Successful echo server connection</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Wed Jun 29 18:26:17 UTC 2022</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Wed Jun 29 18:26:18 UTC 2022</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Successful echo server connection</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Wed Jun 29 18:26:18 UTC 2022</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Wed Jun 29 18:26:18 UTC 2022</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">curl: (28) Connection timed out after 300 milliseconds</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Failed to connect to echo server</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Wed Jun 29 18:26:19 UTC 2022</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Wed Jun 29 18:26:19 UTC 2022</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">curl: (28) Connection timed out after 300 milliseconds</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Failed to connect to echo server</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Wed Jun 29 18:26:20 UTC 2022</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Wed Jun 29 18:26:20 UTC 2022</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Successful echo server connection</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:3.0pt"><span style="font-size:9.0pt;font-family:Monaco;color:#1D1C1D">Wed Jun 29 18:26:20 UTC 2022</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"># For peer /host/10.95.14.81/ip_addr_v4<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">protocol bgp Mesh_10_95_14_81 from bgp_template {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> neighbor 10.95.14.81 as 64512;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> source address 10.95.14.82; # The local address we use for the TCP connection<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> graceful restart time 1800; # This parameter seems to make no difference when changing BGP passwords<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> password "LJiKASiglY+KafEwEn/cSmkiok0zHgpQq5EtYhYgoDcSQwKIpX22Tz7jOzX+";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">I have perused the RFCs for both BGP Graceful Restart (4724) & Secure BGP Sessions (2385) but haven't found a solid answer yet. When the password is changed it makes complete sense that any peers with the new password
will refuse to accept any NEW routes received from peers using the old password and vice versa. I don't see the fundamental reason why TCP segments arriving with some unexpected hash necessitates that previously learned routes from that peer need to be flushed
with no TTL, but the observance of the outage suggests that is what is happening. One would think that, in principle, it could wait to tear down existing routes until a configurable timeout (say the graceful restart) expires, providing a window in which we
can change the password and maintain stable routing.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">I am relatively new to BGP and am using BIRD indirectly via Calico for container networking inside Kubernetes. I will of course take things up with the guys behind Calico, but is there anything in the BGP spec/BIRD
implementation which fundamentally prevents network disruption free secret rotation? Let me know if there is any place I should look for more information on this or any debug logs which would be helpful. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Calvin<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>