<div dir="ltr">Then you can differentiate those routes somehow, for example by marking them with a community, and check the community when exporting to kernel to choose the desired ip for prefsrc.<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Nov 21, 2024 at 2:10 AM hans.heng <<a href="mailto:hans.heng@zoom.us">hans.heng@zoom.us</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div><blockquote><p style="margin:0px;font-size:inherit"><span style="color:rgb(50,53,57);background-color:rgb(255,255,255);font-size:16px;font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen,Ubuntu,Cantarell,Fira Sans,Droid Sans,Helvetica Neue,sans-serif">Or maybe you want to set the source IP based on some external information (for example what provider you received the default route from)?</span></p></blockquote><p style="margin:0px;font-size:inherit"><br></p><p style="margin:0px;font-size:inherit">Yes, maybe this one.</p></div>
<div class="gmail_quote">
<div class="gmail_attr">
On Thu, November 21 2024 at 12:40 AM Alexander Zubkov <a title="mailto:green@qrator.net" href="mailto:green@qrator.net" target="_blank"><green@qrator.net></a> wrote:
</div>
<div>
<blockquote style="margin:5px;padding-left:10px;border-left:thin solid rgb(204,204,204)" type="cite">
<div dir="ltr">It is still unclear what you want to achive. Do you want Linux to use both IPs as src at once in ECMP-like manner? Or maybe you want to set the source IP based on some external information (for example what provider you received the default route from)? Something else?<br></div><br><div class="gmail_quote"><div class="gmail_attr" dir="ltr">On Wed, Nov 20, 2024 at 11:06 AM hans.heng <<a title="mailto:hans.heng@zoom.us" href="mailto:hans.heng@zoom.us" target="_blank">hans.heng@zoom.us</a>> wrote:<br></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div><div><p style="margin:0px;font-size:inherit"><span style="font-size:14px">I just think now these two VIP are unequal, because default routing will use </span>38.145.72.193 instead of 38.145.72.198… No actual problem now, just a question and can we achieve this?</p></div>
<div class="gmail_quote">
<div class="gmail_attr">
On Wed, November 20 2024 at 6:00 PM Alexander Zubkov <a href="mailto:green@qrator.net" title="mailto:green@qrator.net" target="_blank"><green@qrator.net></a> wrote:
</div>
<div>
<blockquote type="cite" style="margin:5px;padding-left:10px;border-left:thin solid rgb(204,204,204)">
<div dir="auto">And what is you intention then? You want to use both sources at once in ecmp-like manner, when the server makes a connection? Or something else?</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Nov 20, 2024, 10:44 hans.heng <<a href="mailto:hans.heng@zoom.us" title="mailto:hans.heng@zoom.us" target="_blank">hans.heng@zoom.us</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div><blockquote><p style="margin:0px;font-size:inherit"><span style="color:rgb(50,53,57);background-color:rgb(255,255,255);font-size:16px;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif">You can set krt_prefsrc in bird. If I remember the option name right.</span></p></blockquote><p style="margin:0px;font-size:inherit"><br></p><p style="margin:0px;font-size:inherit">But we have already this for the first virtual ip 38.145.72.193, I do not know how to additionally add routing I mentioned in the previoius thread for the second virtual ip 38.145.72.198.</p><pre><code>protocol kernel {
scan time 1;
merge paths yes limit 4;
ipv4 {
import none;
export filter {
if proto = direct1 then reject;
krt_prefsrc = 38.145.72.193;
accept;
};
};
}
Thanks,
Hans</code></pre></div>
<div class="gmail_quote">
<div class="gmail_attr">
On Wed, November 20 2024 at 5:40 PM Alexander Zubkov <a title="mailto:green@qrator.net" href="mailto:green@qrator.net" rel="noreferrer" target="_blank"><green@qrator.net></a> wrote:
</div>
<div>
<blockquote style="margin:5px;padding-left:10px;border-left:thin solid rgb(204,204,204)" type="cite">
<div dir="auto">You can set krt_prefsrc in bird. If I remember the option name right.</div><br><div class="gmail_quote"><div class="gmail_attr" dir="ltr">On Wed, Nov 20, 2024, 10:11 hans.heng <<a title="mailto:hans.heng@zoom.us" href="mailto:hans.heng@zoom.us" rel="noreferrer" target="_blank">hans.heng@zoom.us</a>> wrote:<br></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div><div><pre><code>> Hi Hans,
>
> What about just adding it to your LOCAL_NET list? If it works for the first
> IP, why it shouldn't work for the other?
Hi Alexander,
You’re right, simply adding it to LOCAL_NET list does work, other hosts then can test tcp stream on this new virtual ip.
But what confused me is that how can we add export a routing rule like this:
$ ip route
default proto bird src 38.145.72.193 metric 32
nexthop via 10.105.1.10 dev enp4s0f0 weight 1
nexthop via 10.105.1.12 dev enp4s0f1 weight 1
Namely, how this server sends out the traffic using new virtual ip 38.145.72.198 as source addr?
Thanks,
Hans</code></pre></div>
<div class="gmail_quote">
<div class="gmail_attr">
On Wed, November 20 2024 at 4:03 PM Alexander Zubkov <a rel="noreferrer noreferrer" href="mailto:green@qrator.net" title="mailto:green@qrator.net" target="_blank"><green@qrator.net></a> wrote:
</div>
<div>
<blockquote type="cite" style="margin:5px;padding-left:10px;border-left:thin solid rgb(204,204,204)">
<div dir="ltr"><div>Hi Hans,</div><div><br></div><div>What about just adding it to your LOCAL_NET list? If it works for the first IP, why it shouldn't work for the other?<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Nov 20, 2024 at 8:36 AM Hans Heng via Bird-users <<a rel="noreferrer noreferrer" href="mailto:bird-users@network.cz" title="mailto:bird-users@network.cz" target="_blank">bird-users@network.cz</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div><pre><code>Hi all!
I have a dual-home server, whose connection topology and configuration is described as below.
**** Connection Topology and Configuration ****
The server has two physical NICs enp4s0f0 and enp4s0f1, which are connected to two separate ports TOR_A and TOR_B on a Top of Rack (TOR) router, each representing a separate BGP session through bgp_A and bgp_B protocol.
The server also has a dummy interface named em5, which has a private ip <a rel="noreferrer noreferrer" href="https://urldefense.proofpoint.com/v2/url?u=http-3A__38.145.72.193_32&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=jKSN1FCc_6R4dNzWWkk4s9IEPghmnQKefyIIVtVmGsE&e=" title="https://urldefense.proofpoint.com/v2/url?u=http-3A__38.145.72.193_32&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=jKSN1FCc_6R4dNzWWkk4s9IEPghmnQKefyIIVtVmGsE&e=" target="_blank">38.145.72.193/32</a>.
Bird kernel protocol exports an ECMP routing rule to kernel as a kernel default routing, and bgp_A/bgp_B protocol export the private ip to my internet, then <a rel="noreferrer noreferrer" href="https://urldefense.proofpoint.com/v2/url?u=http-3A__38.145.72.193_32&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=jKSN1FCc_6R4dNzWWkk4s9IEPghmnQKefyIIVtVmGsE&e=" title="https://urldefense.proofpoint.com/v2/url?u=http-3A__38.145.72.193_32&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=jKSN1FCc_6R4dNzWWkk4s9IEPghmnQKefyIIVtVmGsE&e=" target="_blank">38.145.72.193/32</a> on em5 acts as a public ip.
**** My goal ****
I want to add another virtual ip on em5, say <a rel="noreferrer noreferrer" href="https://urldefense.proofpoint.com/v2/url?u=http-3A__38.145.72.198_32&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=H85Ro8DXm3uWfnfQU4MX2w7eTYkSllixeGgTN4D3XF8&e=" title="https://urldefense.proofpoint.com/v2/url?u=http-3A__38.145.72.198_32&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=H85Ro8DXm3uWfnfQU4MX2w7eTYkSllixeGgTN4D3XF8&e=" target="_blank">38.145.72.198/32</a>, and let this new virtual ip act as a public ip too.
Can this requirement be met? If yes, how should I modify the configuration to make BGP advertise this ip over my internet?
**** Additional Information on Server ****
## bird setup an ECMP route on 2 phy nics as default route,
## which set source ip to the public ip
$ ip route
default proto bird src 38.145.72.193 metric 32
nexthop via 10.105.1.10 dev enp4s0f0 weight 1
nexthop via 10.105.1.12 dev enp4s0f1 weight 1
<a rel="noreferrer noreferrer" href="https://urldefense.proofpoint.com/v2/url?u=http-3A__10.105.1.10_31&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=-p9Ki6n5W1WV1sy_k1TvuK40E8Bwj5zNUKhIHqRBiws&e=" title="https://urldefense.proofpoint.com/v2/url?u=http-3A__10.105.1.10_31&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=-p9Ki6n5W1WV1sy_k1TvuK40E8Bwj5zNUKhIHqRBiws&e=" target="_blank">10.105.1.10/31</a> dev enp4s0f0 proto kernel scope link src 10.105.1.11
<a rel="noreferrer noreferrer" href="https://urldefense.proofpoint.com/v2/url?u=http-3A__10.105.1.12_31&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=PPYkekks3l9B0WozwhhmanpPyqt-7_BalYLNq0I3QJM&e=" title="https://urldefense.proofpoint.com/v2/url?u=http-3A__10.105.1.12_31&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=PPYkekks3l9B0WozwhhmanpPyqt-7_BalYLNq0I3QJM&e=" target="_blank">10.105.1.12/31</a> dev enp4s0f1 proto kernel scope link src 10.105.1.13
</code></pre><pre><code>bird> show route all
Table master4:
<a rel="noreferrer noreferrer" href="https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=Cvj89uXo924yxK47AGuHlg3g1BKfJdkW6sUvQso_2xE&e=" title="https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=Cvj89uXo924yxK47AGuHlg3g1BKfJdkW6sUvQso_2xE&e=" target="_blank">0.0.0.0/0</a> unicast [bgp_A 2024-11-17] * (100) [AS4212010101i]
via 10.105.1.10 on enp5s0f0
Type: BGP univ
BGP.origin: IGP
BGP.as_path: 4259105001 4212010101
BGP.next_hop: 10.105.1.10
BGP.local_pref: 0
unicast [bgp_B 2024-11-18] (100) [AS4212010101i]
via 10.105.1.12 on enp5s0f1
Type: BGP univ
BGP.origin: IGP
BGP.as_path: 4259205001 4212010101
BGP.next_hop: 10.105.1.12
BGP.local_pref: 0
<a rel="noreferrer noreferrer" href="https://urldefense.proofpoint.com/v2/url?u=http-3A__38.145.72.193_32&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=jKSN1FCc_6R4dNzWWkk4s9IEPghmnQKefyIIVtVmGsE&e=" title="https://urldefense.proofpoint.com/v2/url?u=http-3A__38.145.72.193_32&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=jKSN1FCc_6R4dNzWWkk4s9IEPghmnQKefyIIVtVmGsE&e=" target="_blank">38.145.72.193/32</a> unicast [direct1 2024-11-06] * (240)
dev em5
Type: device univ</code></pre><pre><code>
# bird.conf
router id 172.18.xxx.yyy;
ipv4 table master4;
define LOCAL_NET = [ <a rel="noreferrer noreferrer" href="https://urldefense.proofpoint.com/v2/url?u=http-3A__38.145.72.193_32&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=jKSN1FCc_6R4dNzWWkk4s9IEPghmnQKefyIIVtVmGsE&e=" title="https://urldefense.proofpoint.com/v2/url?u=http-3A__38.145.72.193_32&d=DwMFaQ&c=8lBT5Jra4Bm5rFhLVR7k1wx3__gIUgr523Abjhgq6Gg&r=2bMJ37PQSlB2sRdYUDvVq5IXWp4L1dKQ8V4ZALYNl1I&m=WC9AjD5ZaNv4Fa_mh7nA7q4p-qW7Lj3LBH1uqKj-709dNQOFf9b092u8Xwq5Ago4&s=jKSN1FCc_6R4dNzWWkk4s9IEPghmnQKefyIIVtVmGsE&e=" target="_blank">38.145.72.193/32</a> ];
protocol direct {
ipv4;
interface "em5",-"*";
}
protocol kernel {
scan time 1;
merge paths yes limit 4;
ipv4 {
import none;
export filter {
if proto = "direct1" then reject;
krt_prefsrc = 38.145.72.193;
accept;
};
};
}
protocol device {
scan time 1;
}
protocol bgp bgp_A {
description "TOR A";
local 10.105.1.11 as 4290105101 ; # enp4s0f0
neighbor 10.105.1.10 as 4259105001; # TOR_A
path metric 1;
ipv4 {
import all;
export filter {
if net ~ LOCAL_NET then accept;
else reject;
};
next hop self;
};
}
protocol bgp bgp_B {
bfd;
description "TOR B";
local 10.105.1.13 as 4290105101 ; # enp4s0f1
neighbor 10.105.1.12 as 4259205001; # TOR_B
default bgp_med 0;
default bgp_local_pref 0;
path metric 1;
ipv4 {
import all;
export filter {
if net ~ LOCAL_NET then accept;
else reject;
};
next hop self;
};
}</code></pre></div></div>
</blockquote></div>
</blockquote>
</div>
</div>
</div>
</blockquote></div>
</blockquote>
</div>
</div>
</div>
</blockquote></div>
</blockquote>
</div>
</div>
</div>
</blockquote></div>
</blockquote>
</div>
</div>
</div>
</blockquote></div>