<!DOCTYPE html><html><head><title></title><style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div>send(..., MSG_MORE)<br></div><div>send(..., 0)<br></div><div><br></div><div>Is roughly equivalent to<br></div><div>setsockopt(...<span style="color:var(--ui-page-color-fg);">, TCP_CORK, 1)</span></div><div>send(..., 0)</div><div>send(..., 0)<br></div><div>setsockopt(..., TCP_CORK, 0)<br></div><div><br></div><div>In cases where you know that you're going to write more but you want to be able to discard the user space buffer, MSG_MORE is useful. If holding onto the buffers is easy so they can then be passed to sendmsg(), then you may as well do that. <br></div><div><br></div><div>(And of course you can combine both techniques)<br></div><div><br></div><div>- Erin</div><div><br></div><div>On Thu, 16 May 2024, at 01:32, Job Snijders wrote:<br></div><blockquote type="cite" id="qt" style=""><div>On Wed, May 15, 2024 at 09:09:47PM +0200, Erin Shepherd wrote:<br></div><div>> It seems absent from the BSDs, but on Linux you can pass the MSG_MORE<br></div><div>> flag to send() to override TCP_NODELAY for a specific write<br></div><div><br></div><div>Am I understanding correctly this is a variant on TCP_NOPUSH/TCP_CORK?<br></div><div>"more data is coming, dont push the send button yet!"<br></div><div><br></div><div>In OpenBGPD, TCP_NODELAY is set on the socket (a socket option available<br></div><div>on all platforms, I think?), and then all data is coalesced into<br></div><div>sendmsg(), no need for 'corking'. From my limited testing it seems a<br></div><div>full routing table should fit in ~ TCP 41,000 packets.<br></div><div><br></div><div>BIRD has a code path sk_sendmsg()->sendmsg() called from<br></div><div>sk_maybe_write(); but based my limited testing I'm not sure this path is<br></div><div>followed in all cases, because I see way more than 41K packets for a<br></div><div>full table feed (with TCP_NODELAY enabled).<br></div><div><br></div><div>Perhaps there are two separate questions here:<br></div><div><br></div><div>- are BGP messages (slightly) delayed because of TCP_NODELAY not being<br></div><div> set? (I think yes)<br></div><div>- are BGP messages as efficiently coalesced into as few TCP packets as<br></div><div> possible? (with TCP_NODELAY set, I am not sure)<br></div><div><br></div><div>Kind regards,<br></div><div><br></div><div>Job<br></div><div><br></div><div>ps. To clarify why I started this thread: last week I fell into the TCP<br></div><div>subsystem rabbit hole: why are things the way they are? I started<br></div><div>auditing various programs related to my $dayjob and thought it would be<br></div><div>good to open a conversation with the BIRD developer community. My goal<br></div><div>is not necessarily to get this patch 'as-is' merged, but to learn from<br></div><div>and with friendly and respected BGP developers.<br></div><div><br></div></blockquote></body></html>