<!doctype html><html><head><title></title><style type="text/css">#qt {line-height:1.2;font-family:serif;font-size:0.9em;color:black;background-color:white;}
#qt {margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:auto;max-width:36em;padding-top:1em;padding-right:1em;padding-bottom:1em;padding-left:1em;overflow-wrap:break-word;text-rendering:optimizelegibility;font-kerning:normal;}
@media print{
#qt {background-color:transparent;color:black;font-size:11pt;}
#qt p{orphans:3;widows:3;}
}
#qt p{margin-top:1em;margin-right:0px;margin-bottom:1em;margin-left:0px;}
#qt a:visited,#qt a{color:black;}
#qt blockquote{margin-top:0.5em;margin-right:0.5em;margin-bottom:0.5em;margin-left:0.5em;padding-left:0.5em;border-left-width:2px;border-left-style:solid;border-left-color:rgb(230, 230, 230);color:rgb(68, 68, 68);}
#qt code{font-family:"Lucida Console", monospace;font-size:95%;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;}
#qt pre{margin-top:1em;margin-right:0px;margin-bottom:1em;margin-left:0px;overflow-x:auto;overflow-y:auto;}
#qt pre code{padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;overflow-x:visible;overflow-y:visible;overflow-wrap:normal;}
#qt code{text-wrap:wrap;}
p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div dir="auto">(mistakenly sent off-list)</div><br><br><div style='font-size:10.0pt;font-family:"Tahoma","sans-serif";padding:3.0pt 0in 0in 0in'>
<hr style='border:none;border-top:solid #E1E1E1 1.0pt'>
<b>From:</b> <div dir="auto">Maria Matejka <maria.matejka@nic.cz></div><br>
<b>Sent:</b> 13 April 2024 18:18:05 CEST<br>
<b>To:</b> <div dir="auto">Erin Shepherd <bird-users@erinshepherd.net></div><br>
<b>Subject:</b> <div dir="auto">Re: babel RTT metric false samples</div><br>
</div>
<br>
<div dir="auto">Just quick thought – I think both approaches (timestamping in kernel and in userspace) are actually useful for different purposes. Thus, we shall support both.<br><br>Transferred to our internal issue: <a href="https://gitlab.nic.cz/labs/bird/-/issues/61">https://gitlab.nic.cz/labs/bird/-/issues/61</a><br><br>Maria<br></div><br><br><div class="gmail_quote"><div dir="auto">On 13 April 2024 16:38:47 CEST, Erin Shepherd <bird-users@erinshepherd.net> wrote:</div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div>I guess it might not fit with bird's abstractions (or perhaps the Babel protocol), but has thought been given to using SO_TIMESTAMPING to have the kernel compute TX/RX timestamps? <br></div><div><br></div><div>- Erin</div><div><br></div><div><br></div><div>On Sat, 13 Apr 2024, at 16:14, Maria Matejka via Bird-users wrote:<br></div><blockquote type="cite" id="qt" style=""><p>Hello Stephanie, Toke and list,<br></p><p>On Fri, Apr 12, 2024 at 04:22:50PM +0200, Toke Høiland-Jørgensen via
Bird-users wrote:<br></p><blockquote><p>Stephanie Wilde-Hobbs via Bird-users <a href="mailto:bird-users@network.cz" class="qt-email">bird-users@network.cz</a> writes:<br></p></blockquote><blockquote><blockquote><p>The babel RTT metric measurements provided by bird appears suspect
for my setup. The metric through a tunnel with a latency of about 5ms is
shown in babel as 150+ms.<br></p></blockquote></blockquote><p>[…]<br></p><blockquote><blockquote><p>Debug logs show many RTT samples with approximately correct
timestamps (4-6ms) then the occasional IHU with 800-1200ms calculated
instead. Calculating the RTT metric by hand using babel packet logs
shows that the calculations are correct. By correlating two packet dumps
(the machines have <1ms NTP timekeeping) I can also see that the
packets for which high RTT is calculated have similar transit times
through the tunnel as other packets. Hence, I suspect the accuracy of
the packet timestamps recorded by bird. Is the current packet
timestamping system giving correct timestamps if the packet arrives
while babel is processing another event?<br></p></blockquote><p>Hmm, so Babel implementation in Bird tries to get a timestamp as
early as possible after receiving the packet, and set it as late as
possible before sending out the packet. However, the former in practice
means after returning from poll(), so if the packet has been sitting
around in the OS buffer for a while before Bird gets around to process
it, the timestamp is not set until Bird is done processing it. Likewise,
if the packet sits around in a socket buffer (or in a lower-level buffer
on the sending side) after Bird has sent it out, that time will also be
counted as part of the RTT.<br></p></blockquote><p>I would suspect that the kernel table prune routine may be the case.
It just runs from begin to end synchronously.<br></p><p>I have just fast-tracked Babel in its own thread for BIRD 3, it may
be worth checking. (There should be also artifacts from the build
process for download available.) This should get you rid of most of the
cases of suspiciously high RTT.<br></p><pre><code>https://gitlab.nic.cz/labs/bird/-/tree/babel-in-threads</code><br></pre><p>Just to be noted, updating a route in BIRD 3 is still a locking
process so it may still tamper the RTT measurements. At least it should
happen only in cases where Babel is doing the update. Anyway, with BIRD
3 internals, it should be possible to easily <i>detect</i> such
situations and disregard these single measurements as unreliable. (Not
implemented, though.)<br></p><p>There are even some thoughts on implementing lockless import queues
for routing tables, yet now we have to prioritize BIRD 3 stabilization
to actually release it as a stable version. Import queues must wait.<br></p><p>Also with this testing, feel free to report any weird behavior,
notably crashes of BIRD 3, as bugs. That would be very helpful with
stabilizing BIRD 3. Thanks a lot!<br></p><p>Maria<br></p><p>– Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o.<br></p></blockquote></blockquote></div><div dir="auto"><div class='k9mail-signature'>-- <br>Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o.</div></div><div dir="auto"><div class='k9mail-signature'>-- <br>Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o.</div></div></body></html>