Hi, I am building a solution where I have N HAProxy servers which have BGP and BFD sessions with 2 upstream routers. Bird runs on all HAProxy servers and announces the same prefixes via BGP. I am using ECMP on upstream routes to balance incoming traffic to these HAProxy servers, which balance traffic to N backend servers. I need to advertise/withdraw several IP prefixes within the same range (192.168.200.0/24) based on the health of a service, in this case the frontend on HAProxy. These IPs are assigned only the loopback interface. The problem I am facing is that when I adjust a filter in order to withdraw the route, all routes are withdrawn and the remaining are announced again. According to the doc if a reload on bird brings a new configuration for a protocol, that protocol will be restarted. So, it is expected behavior. Below is the log[1] after I removed '192.168.200.1' from prefix set anycast_prefixes in loopback_ACL filter and reloaded bird. I was monitoring BGP protocols in a loop running bird show protocols all "bgp8" and saw that 'export updates' counter was increased from 2 to 3 and 'export withdraws' counter was increased from 0 to 2. Is there a way to withdraw to a route from a list without effecting the rest routes from the list? The only way I have found so far, it was by removing the specific IP from the loopback interface. But, this solution prevents a reload/restart on HAProxy as it can't bind anymore to the socket Cheers, Pavlos [1] 23:37:52 <INFO> Reconfiguration requested by SIGHUP 23:37:52 <INFO> Reconfiguring 23:37:52 <TRACE> direct1: Reconfigured 23:37:52 <INFO> Reloading protocol direct1 23:37:52 <INFO> Restarting protocol direct1 23:37:52 <TRACE> direct1: Shutting down 23:37:52 <TRACE> direct1: State changed to flush 23:37:52 <TRACE> bgp_peer1: Reconfigured 23:37:52 <TRACE> bgp_peer2: Reconfigured 23:37:52 <INFO> Reconfigured 23:37:52 <TRACE> direct1 > removed [sole] 192.168.200.1/32 dev lo 23:37:52 <TRACE> direct1 > removed [sole] 192.168.200.2/32 dev lo 23:37:52 <TRACE> direct1: State changed to down 23:37:52 <TRACE> direct1: Starting 23:37:52 <TRACE> direct1: Connected to table master 23:37:52 <TRACE> direct1: State changed to feed 23:37:52 <TRACE> direct1 < primary address 192.168.200.1/32 on interface lo added 23:37:52 <TRACE> direct1 > filtered out 192.168.200.1/32 dev lo 23:37:52 <TRACE> direct1 < secondary address 127.0.0.0/8 on interface lo added 23:37:52 <TRACE> direct1 < secondary address 192.168.200.2/32 on interface lo added 23:37:52 <TRACE> direct1 > added [best] 192.168.200.2/32 dev lo 23:37:52 <TRACE> bgp_peer2 < filtered out 192.168.200.2/32 dev lo 23:37:52 <TRACE> bgp_peer1 < filtered out 192.168.200.2/32 dev lo 23:37:52 <TRACE> direct1 < primary address 192.168.80.0/20 on interface eth0 added 23:37:52 <TRACE> direct1: State changed to up 23:37:54 <TRACE> bgp_peer2: Sending KEEPALIVE 23:37:54 <TRACE> bgp_peer1: Sending KEEPALIVE 23:37:54 <TRACE> bgp_peer2: Got KEEPALIVE 23:37:56 <TRACE> bgp_peer1: Got KEEPALIVE bird.conf # Override router ID router id 192.168.88.194; # Listens only on eth0's IP listen bgp address 192.168.88.194; filter loopback_ACL prefix set anycast_prefixes; { anycast_prefixes = [ 192.168.200.1/32, 192.168.200.2/32 ]; if net ~ anycast_prefixes then accept; else reject; } filter anycast_range { if net ~ [ 192.168.200.0/24 ] then accept; else reject; } protocol direct { interface "lo"; debug all; import filter loopback_ACL; export none; } protocol kernel kernel1 { disabled yes; } protocol device { scan time 10; # TODO do I need this? } protocol static { disabled yes; } protocol bfd { interface "eth0" { min rx interval 500 ms; min tx interval 500 ms; idle tx interval 1000 ms; multiplier 3; }; } template bgp BGP_PEERS { bfd on; debug all; import none; export filter anycast_range; direct; hold time 30; startup hold time 240; connect retry time 120; keepalive time 10; start delay time 5; error wait time 60, 300; error forget time 300; disable after error off; next hop self; path metric 1; default bgp_med 0; default bgp_local_pref 0; } protocol bgp bgp_peer1 from BGP_PEERS { disabled no; description "My-BGP-Peer1"; local as 46111; neighbor 192.168.95.252 as 46111; source address 192.168.88.194; } protocol bgp bgp_peer2 from BGP_PEERS { disabled no; description "My-BGP-Peer2"; local as 46111; neighbor 192.168.95.253 as 46111; source address 192.168.88.194; }
Try this: protocol device { scan time 10; # Scan interfaces every 10 seconds primary "lo" 127.0.0.1; } -- Eric Cables On Sun, Aug 3, 2014 at 3:11 PM, Pavlos Parissis <pavlos.parissis@gmail.com> wrote:
Hi,
I am building a solution where I have N HAProxy servers which have BGP and BFD sessions with 2 upstream routers. Bird runs on all HAProxy servers and announces the same prefixes via BGP. I am using ECMP on upstream routes to balance incoming traffic to these HAProxy servers, which balance traffic to N backend servers.
I need to advertise/withdraw several IP prefixes within the same range (192.168.200.0/24) based on the health of a service, in this case the frontend on HAProxy. These IPs are assigned only the loopback interface.
The problem I am facing is that when I adjust a filter in order to withdraw the route, all routes are withdrawn and the remaining are announced again. According to the doc if a reload on bird brings a new configuration for a protocol, that protocol will be restarted. So, it is expected behavior.
Below is the log[1] after I removed '192.168.200.1' from prefix set anycast_prefixes in loopback_ACL filter and reloaded bird. I was monitoring BGP protocols in a loop running bird show protocols all "bgp8" and saw that 'export updates' counter was increased from 2 to 3 and 'export withdraws' counter was increased from 0 to 2.
Is there a way to withdraw to a route from a list without effecting the rest routes from the list?
The only way I have found so far, it was by removing the specific IP from the loopback interface. But, this solution prevents a reload/restart on HAProxy as it can't bind anymore to the socket
Cheers, Pavlos
[1] 23:37:52 <INFO> Reconfiguration requested by SIGHUP 23:37:52 <INFO> Reconfiguring 23:37:52 <TRACE> direct1: Reconfigured 23:37:52 <INFO> Reloading protocol direct1 23:37:52 <INFO> Restarting protocol direct1 23:37:52 <TRACE> direct1: Shutting down 23:37:52 <TRACE> direct1: State changed to flush 23:37:52 <TRACE> bgp_peer1: Reconfigured 23:37:52 <TRACE> bgp_peer2: Reconfigured 23:37:52 <INFO> Reconfigured 23:37:52 <TRACE> direct1 > removed [sole] 192.168.200.1/32 dev lo 23:37:52 <TRACE> direct1 > removed [sole] 192.168.200.2/32 dev lo 23:37:52 <TRACE> direct1: State changed to down 23:37:52 <TRACE> direct1: Starting 23:37:52 <TRACE> direct1: Connected to table master 23:37:52 <TRACE> direct1: State changed to feed 23:37:52 <TRACE> direct1 < primary address 192.168.200.1/32 on interface lo added 23:37:52 <TRACE> direct1 > filtered out 192.168.200.1/32 dev lo 23:37:52 <TRACE> direct1 < secondary address 127.0.0.0/8 on interface lo added 23:37:52 <TRACE> direct1 < secondary address 192.168.200.2/32 on interface lo added 23:37:52 <TRACE> direct1 > added [best] 192.168.200.2/32 dev lo 23:37:52 <TRACE> bgp_peer2 < filtered out 192.168.200.2/32 dev lo 23:37:52 <TRACE> bgp_peer1 < filtered out 192.168.200.2/32 dev lo 23:37:52 <TRACE> direct1 < primary address 192.168.80.0/20 on interface eth0 added 23:37:52 <TRACE> direct1: State changed to up 23:37:54 <TRACE> bgp_peer2: Sending KEEPALIVE 23:37:54 <TRACE> bgp_peer1: Sending KEEPALIVE 23:37:54 <TRACE> bgp_peer2: Got KEEPALIVE 23:37:56 <TRACE> bgp_peer1: Got KEEPALIVE
bird.conf # Override router ID router id 192.168.88.194;
# Listens only on eth0's IP listen bgp address 192.168.88.194;
filter loopback_ACL prefix set anycast_prefixes; { anycast_prefixes = [ 192.168.200.1/32, 192.168.200.2/32 ]; if net ~ anycast_prefixes then accept; else reject; }
filter anycast_range { if net ~ [ 192.168.200.0/24 ] then accept; else reject; }
protocol direct { interface "lo"; debug all; import filter loopback_ACL; export none; }
protocol kernel kernel1 { disabled yes; }
protocol device { scan time 10; # TODO do I need this? }
protocol static { disabled yes; }
protocol bfd { interface "eth0" { min rx interval 500 ms; min tx interval 500 ms; idle tx interval 1000 ms; multiplier 3; }; }
template bgp BGP_PEERS { bfd on; debug all; import none; export filter anycast_range; direct; hold time 30; startup hold time 240; connect retry time 120; keepalive time 10; start delay time 5; error wait time 60, 300; error forget time 300; disable after error off; next hop self; path metric 1; default bgp_med 0; default bgp_local_pref 0; }
protocol bgp bgp_peer1 from BGP_PEERS { disabled no; description "My-BGP-Peer1"; local as 46111; neighbor 192.168.95.252 as 46111; source address 192.168.88.194; }
protocol bgp bgp_peer2 from BGP_PEERS { disabled no; description "My-BGP-Peer2"; local as 46111; neighbor 192.168.95.253 as 46111; source address 192.168.88.194; }
On 04/08/2014 02:01 πμ, Eric Cables wrote:
Try this:
protocol device { scan time 10; # Scan interfaces every 10 seconds primary "lo" 127.0.0.1; }
I read the doc about this and I don't see how it could help me. Nevertheless, I gave it a try but it didn't have any effect, see below the log. Thanks, Pavlos 02:06:33 <INFO> Reconfiguration requested by SIGHUP 02:06:33 <INFO> Reconfiguring 02:06:33 <TRACE> direct1: Reconfigured 02:06:33 <INFO> Reloading protocol direct1 02:06:33 <INFO> Restarting protocol direct1 02:06:33 <TRACE> direct1: Shutting down 02:06:33 <TRACE> direct1: State changed to flush 02:06:33 <TRACE> bgp_peer1: Reconfigured 02:06:33 <TRACE> bgp_peer2: Reconfigured 02:06:33 <INFO> Reconfigured 02:06:33 <TRACE> direct1 > removed [sole] 192.168.200.1/32 dev lo 02:06:33 <TRACE> bgp_peer2 < removed 192.168.200.1/32 dev lo 02:06:33 <TRACE> bgp_peer1 < removed 192.168.200.1/32 dev lo 02:06:33 <TRACE> direct1 > removed [sole] 192.168.200.2/32 dev lo 02:06:33 <TRACE> bgp_peer2 < removed 192.168.200.2/32 dev lo 02:06:33 <TRACE> bgp_peer1 < removed 192.168.200.2/32 dev lo 02:06:33 <TRACE> direct1: State changed to down 02:06:33 <TRACE> direct1: Starting 02:06:33 <TRACE> direct1: Connected to table master 02:06:33 <TRACE> direct1: State changed to feed 02:06:33 <TRACE> bgp_peer2: Sending UPDATE 02:06:33 <TRACE> bgp_peer1: Sending UPDATE 02:06:33 <TRACE> direct1 < primary address 127.0.0.0/8 on interface lo added 02:06:33 <TRACE> direct1 < secondary address 192.168.200.1/32 on interface lo added 02:06:33 <TRACE> direct1 > filtered out 192.168.200.1/32 dev lo 02:06:33 <TRACE> direct1 < secondary address 192.168.200.2/32 on interface lo added 02:06:33 <TRACE> direct1 > added [best] 192.168.200.2/32 dev lo 02:06:33 <TRACE> bgp_peer2 < added 192.168.200.2/32 dev lo 02:06:33 <TRACE> bgp_peer1 < added 192.168.200.2/32 dev lo 02:06:33 <TRACE> direct1 < primary address 192.168.80.0/20 on interface eth0 added 02:06:33 <TRACE> direct1: State changed to up 02:06:33 <TRACE> bgp_peer2: Sending UPDATE 02:06:33 <TRACE> bgp_peer1: Sending UPDATE
Have you looked at putting a protocol device per route, and then having a filter per prefix, so only one of the protocol instances will be restarted? Another alternative would be to use something like ExaBGP, which might be a little better suited for this, and have it peer with BIRD, and have that peer with the opposing device. On Sun, Aug 3, 2014 at 5:12 PM, Pavlos Parissis <pavlos.parissis@gmail.com> wrote:
On 04/08/2014 02:01 πμ, Eric Cables wrote:
Try this:
protocol device { scan time 10; # Scan interfaces every 10 seconds primary "lo" 127.0.0.1; }
I read the doc about this and I don't see how it could help me. Nevertheless, I gave it a try but it didn't have any effect, see below the log.
Thanks, Pavlos
02:06:33 <INFO> Reconfiguration requested by SIGHUP 02:06:33 <INFO> Reconfiguring 02:06:33 <TRACE> direct1: Reconfigured 02:06:33 <INFO> Reloading protocol direct1 02:06:33 <INFO> Restarting protocol direct1 02:06:33 <TRACE> direct1: Shutting down 02:06:33 <TRACE> direct1: State changed to flush 02:06:33 <TRACE> bgp_peer1: Reconfigured 02:06:33 <TRACE> bgp_peer2: Reconfigured 02:06:33 <INFO> Reconfigured 02:06:33 <TRACE> direct1 > removed [sole] 192.168.200.1/32 dev lo 02:06:33 <TRACE> bgp_peer2 < removed 192.168.200.1/32 dev lo 02:06:33 <TRACE> bgp_peer1 < removed 192.168.200.1/32 dev lo 02:06:33 <TRACE> direct1 > removed [sole] 192.168.200.2/32 dev lo 02:06:33 <TRACE> bgp_peer2 < removed 192.168.200.2/32 dev lo 02:06:33 <TRACE> bgp_peer1 < removed 192.168.200.2/32 dev lo 02:06:33 <TRACE> direct1: State changed to down 02:06:33 <TRACE> direct1: Starting 02:06:33 <TRACE> direct1: Connected to table master 02:06:33 <TRACE> direct1: State changed to feed 02:06:33 <TRACE> bgp_peer2: Sending UPDATE 02:06:33 <TRACE> bgp_peer1: Sending UPDATE 02:06:33 <TRACE> direct1 < primary address 127.0.0.0/8 on interface lo added 02:06:33 <TRACE> direct1 < secondary address 192.168.200.1/32 on interface lo added 02:06:33 <TRACE> direct1 > filtered out 192.168.200.1/32 dev lo 02:06:33 <TRACE> direct1 < secondary address 192.168.200.2/32 on interface lo added 02:06:33 <TRACE> direct1 > added [best] 192.168.200.2/32 dev lo 02:06:33 <TRACE> bgp_peer2 < added 192.168.200.2/32 dev lo 02:06:33 <TRACE> bgp_peer1 < added 192.168.200.2/32 dev lo 02:06:33 <TRACE> direct1 < primary address 192.168.80.0/20 on interface eth0 added 02:06:33 <TRACE> direct1: State changed to up 02:06:33 <TRACE> bgp_peer2: Sending UPDATE 02:06:33 <TRACE> bgp_peer1: Sending UPDATE
Hi,
Hello.
The problem I am facing is that when I adjust a filter in order to withdraw the route, all routes are withdrawn and the remaining are announced again. According to the doc if a reload on bird brings a new configuration for a protocol, that protocol will be restarted. So, it is expected behavior.
You have changed import filter of direct1 protocol, thus direct1 gets reconfigured (on HUP or birdc configure). However this protocol does not perform "soft" reconfiguration on filter changes thus direct1 restarts unconditionally to apply your filter changes (IP removal/addition). All routes flushed out from the table (causing BGP withdrawal) and fetched from the protocol (direct) again to the table (causing BGP update). This seems normal behavior. You could simply attach your loopback_ACL filter to the BGP_PEERS template export (which any needed adjustments, or just make loopback_ACL to work as function returning false/true and use it in anycast_range) to solve your case. Also I suggest you to use global prefix set definition, as local variable (in filter/function) is created/assigned each time function is called. For example: ------------ # It is probably good idea to have such protocol configred in most cases. # It fetches list of the interfaces in system and provides them to the other # protocols (except OSPF, it has it's own mechanisms). # protocol device { scan time 10; } # This set should be modified only when new frontend subnet is added. # direct1 would restart on protocol filter change. # define ACAST_PS_DIRECT1 = [ 192.168.200.0/24{32,32} ]; protocol direct { # Import directly connected routes from "lo" interface. interface "lo"; # This may be is needed to properly resolve nexthops from upstream routes. interface "<interface_to_your_routers>"; # Do not import from other sources. interface "-*"; # Import only allowed IPs from the "lo" and route from the upstream # interface. import where net ~ ACAST_PS_DIRECT1 || ifname ="<interface_to_your_routers>"; # No export to the protocol, not necessary export none; } # This is global constant for your prefix set (PS) 192.168.200.0/24. # # Place this definition into external file and source it with include # directive from the global configuration file if you wish to modify # this PS from the outside (e.g. by some HA script). # define ACAST_PS_ADVERTISE = [ 192.168.200.1/32, # Frontend 1 192.168.200.2/32 # Frontend 2 ]; function acast_advertise() { return net ~ ACAST_PS_ADVERTISE; } filter anycast_range { if acase_advertise() then accept; reject; } Tested similar config with BIRD 1.3.11 and seems no unwanted withdrawals spot.
Cheers, Pavlos
[1] 23:37:52 <INFO> Reconfiguration requested by SIGHUP 23:37:52 <INFO> Reconfiguring 23:37:52 <TRACE> direct1: Reconfigured 23:37:52 <INFO> Reloading protocol direct1 23:37:52 <INFO> Restarting protocol direct1 23:37:52 <TRACE> direct1: Shutting down 23:37:52 <TRACE> direct1: State changed to flush 23:37:52 <TRACE> bgp_peer1: Reconfigured 23:37:52 <TRACE> bgp_peer2: Reconfigured 23:37:52 <INFO> Reconfigured 23:37:52 <TRACE> direct1 > removed [sole] 192.168.200.1/32 dev lo 23:37:52 <TRACE> direct1 > removed [sole] 192.168.200.2/32 dev lo 23:37:52 <TRACE> direct1: State changed to down 23:37:52 <TRACE> direct1: Starting 23:37:52 <TRACE> direct1: Connected to table master 23:37:52 <TRACE> direct1: State changed to feed 23:37:52 <TRACE> direct1 < primary address 192.168.200.1/32 on interface lo added 23:37:52 <TRACE> direct1 > filtered out 192.168.200.1/32 dev lo 23:37:52 <TRACE> direct1 < secondary address 127.0.0.0/8 on interface lo added 23:37:52 <TRACE> direct1 < secondary address 192.168.200.2/32 on interface lo added 23:37:52 <TRACE> direct1 > added [best] 192.168.200.2/32 dev lo 23:37:52 <TRACE> bgp_peer2 < filtered out 192.168.200.2/32 dev lo 23:37:52 <TRACE> bgp_peer1 < filtered out 192.168.200.2/32 dev lo 23:37:52 <TRACE> direct1 < primary address 192.168.80.0/20 on interface eth0 added 23:37:52 <TRACE> direct1: State changed to up 23:37:54 <TRACE> bgp_peer2: Sending KEEPALIVE 23:37:54 <TRACE> bgp_peer1: Sending KEEPALIVE 23:37:54 <TRACE> bgp_peer2: Got KEEPALIVE 23:37:56 <TRACE> bgp_peer1: Got KEEPALIVE
bird.conf # Override router ID router id 192.168.88.194;
# Listens only on eth0's IP listen bgp address 192.168.88.194;
filter loopback_ACL prefix set anycast_prefixes; { anycast_prefixes = [ 192.168.200.1/32, 192.168.200.2/32 ]; if net ~ anycast_prefixes then accept; else reject; }
filter anycast_range { if net ~ [ 192.168.200.0/24 ] then accept; else reject; }
protocol direct { interface "lo"; debug all; import filter loopback_ACL; export none; }
protocol kernel kernel1 { disabled yes; }
protocol device { scan time 10; # TODO do I need this? }
protocol static { disabled yes; }
protocol bfd { interface "eth0" { min rx interval 500 ms; min tx interval 500 ms; idle tx interval 1000 ms; multiplier 3; }; }
template bgp BGP_PEERS { bfd on; debug all; import none; export filter anycast_range; direct; hold time 30; startup hold time 240; connect retry time 120; keepalive time 10; start delay time 5; error wait time 60, 300; error forget time 300; disable after error off; next hop self; path metric 1; default bgp_med 0; default bgp_local_pref 0; }
protocol bgp bgp_peer1 from BGP_PEERS { disabled no; description "My-BGP-Peer1"; local as 46111; neighbor 192.168.95.252 as 46111; source address 192.168.88.194; }
protocol bgp bgp_peer2 from BGP_PEERS { disabled no; description "My-BGP-Peer2"; local as 46111; neighbor 192.168.95.253 as 46111; source address 192.168.88.194; }
-- SP5474-RIPE Sergey Popovich
On 4 August 2014 09:53, Sergey Popovich <popovich_sergei@mail.ru> wrote:
Hi,
Hello.
The problem I am facing is that when I adjust a filter in order to withdraw the route, all routes are withdrawn and the remaining are announced again. According to the doc if a reload on bird brings a new configuration for a protocol, that protocol will be restarted. So, it is expected behavior.
You have changed import filter of direct1 protocol, thus direct1 gets reconfigured (on HUP or birdc configure). However this protocol does not perform "soft" reconfiguration on filter changes thus direct1 restarts unconditionally to apply your filter changes (IP removal/addition).
All routes flushed out from the table (causing BGP withdrawal) and fetched from the protocol (direct) again to the table (causing BGP update).
This seems normal behavior.
Sergey, thanks you very much for this very detailed explanation. You could simply attach your loopback_ACL filter to the BGP_PEERS template
export (which any needed adjustments, or just make loopback_ACL to work as function returning false/true and use it in anycast_range) to solve your case.
Also I suggest you to use global prefix set definition, as local variable (in filter/function) is created/assigned each time function is called.
For example: ------------
# It is probably good idea to have such protocol configred in most cases. # It fetches list of the interfaces in system and provides them to the other # protocols (except OSPF, it has it's own mechanisms). # protocol device { scan time 10; }
# This set should be modified only when new frontend subnet is added. # direct1 would restart on protocol filter change. # define ACAST_PS_DIRECT1 = [ 192.168.200.0/24{32,32} ];
protocol direct { # Import directly connected routes from "lo" interface. interface "lo";
# This may be is needed to properly resolve nexthops from upstream routes. interface "<interface_to_your_routers>";
what do you mean by this ? Since I only want to export routes for IPs in the loopback interface, I thought I can safely skip other interfaces for protocol direct. Well, with the filters in place, I don't worry for accidentally announcing routes that I shouldn't. Furthermore, as a safeguard I have set the upstream routers to allow announcements only for the specific prefix(192.168.200.0/24). Oh yes I play very safe here:-)
# Do not import from other sources. interface "-*";
# Import only allowed IPs from the "lo" and route from the upstream # interface. import where net ~ ACAST_PS_DIRECT1 || ifname ="<interface_to_your_routers>";
I don't need to import anything. Bird is used only as an advertisement system.
# No export to the protocol, not necessary export none; }
# This is global constant for your prefix set (PS) 192.168.200.0/24. # # Place this definition into external file and source it with include # directive from the global configuration file if you wish to modify # this PS from the outside (e.g. by some HA script). # define ACAST_PS_ADVERTISE = [ 192.168.200.1/32, # Frontend 1 192.168.200.2/32 # Frontend 2 ];
function acast_advertise() { return net ~ ACAST_PS_ADVERTISE; }
filter anycast_range { if acase_advertise() then accept; reject; }
Tested similar config with BIRD 1.3.11 and seems no unwanted withdrawals spot.
I configured bird as you suggested[1] and after a restart I removed 192.168.200.1/32 from the prefix set[2] See below the log 13:29:18 <INFO> Reconfiguration requested by SIGHUP 13:29:18 <INFO> Reconfiguring 13:29:18 <TRACE> direct1: Reconfigured 13:29:18 <TRACE> bgp_peer1: Reconfigured 13:29:18 <INFO> Reloading protocol bgp_peer1 13:29:18 <TRACE> bgp_peer1: State changed to feed 13:29:18 <TRACE> bgp_peer2: Reconfigured 13:29:18 <INFO> Reloading protocol bgp_peer2 13:29:18 <TRACE> bgp_peer2: State changed to feed 13:29:18 <INFO> Reconfigured 13:29:18 <TRACE> bgp_peer1 < filtered out 192.168.200.1/32 dev lo 13:29:18 <TRACE> bgp_peer1 < removed 192.168.200.1/32 dev lo 13:29:18 <TRACE> bgp_peer1 < replaced 192.168.200.2/32 dev lo 13:29:18 <TRACE> bgp_peer1: State changed to up 13:29:18 <TRACE> bgp_peer2 < filtered out 192.168.200.1/32 dev lo 13:29:18 <TRACE> bgp_peer2 < removed 192.168.200.1/32 dev lo 13:29:18 <TRACE> bgp_peer2 < replaced 192.168.200.2/32 dev lo 13:29:18 <TRACE> bgp_peer2: State changed to up what the meaning of the 'replaced' here? and the output of birdcl show protocols all \"bgp*\" in a loop ### Mon Aug 4 13:29:11 CEST 2014 ### Routes: 0 imported, 2 exported, 0 preferred Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 2 0 0 --- 2 Export withdraws: 0 --- --- --- 0 Routes: 0 imported, 2 exported, 0 preferred Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 2 0 0 --- 2 Export withdraws: 0 --- --- --- 0 ### Mon Aug 4 13:29:16 CEST 2014 ### Routes: 0 imported, 1 exported, 0 preferred Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 4 0 1 --- 3 Export withdraws: 0 --- --- --- 1 Routes: 0 imported, 1 exported, 0 preferred Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 4 0 1 --- 3 Export withdraws: 0 --- --- --- 1 ### Mon Aug 4 13:29:21 CEST 2014 ### Thank you very much for taking the time to provide a complete solution for me. It is very much appreciated. Cheers, Pavlos [1] bird.conf include "/etc/bird.d/anycast_prefixes.conf"; define ACAST_PS_DIRECT1 = [ 192.168.200.0/24{32,32} ]; function anycast_advertise() { return net ~ ACAST_PS_ADVERTISE; } filter anycast_range { if anycast_advertise() then accept; reject; } router id 192.168.88.194; listen bgp address 192.168.88.194; protocol direct { interface "lo"; debug all; import where net ~ ACAST_PS_DIRECT1; export none; } protocol kernel kernel1 { disabled yes; } protocol device { scan time 10; } protocol static { disabled yes; } protocol bfd { interface "eth0" { min rx interval 500 ms; min tx interval 500 ms; idle tx interval 1000 ms; multiplier 3; }; } template bgp BGP_PEERS { bfd on; debug all; import none; export filter anycast_range; direct; hold time 30; startup hold time 240; connect retry time 120; keepalive time 10; start delay time 5; error wait time 60, 300; error forget time 300; disable after error off; next hop self; path metric 1; default bgp_med 0; default bgp_local_pref 0; } protocol bgp bgp_peer1 from BGP_PEERS { disabled no; description "My-BGP-Peer1"; local as 46111; neighbor 192.168.95.252 as 46111; source address 192.168.88.194; } protocol bgp bgp_peer2 from BGP_PEERS { disabled no; description "My-BGP-Peer2"; local as 46111; neighbor 192.168.95.253 as 46111; source address 192.168.88.194; } [2] /etc/bird.d/anycast_prefixes.conf define ACAST_PS_ADVERTISE = [ 192.168.200.1/32, # Frontend 1 192.168.200.2/32 # Frontend 2 ];
Hello.
protocol direct {
# Import directly connected routes from "lo" interface. interface "lo";
# This may be is needed to properly resolve nexthops from upstream
routes.
interface "<interface_to_your_routers>";
what do you mean by this ?
You need this only if you import something from the your routers (for example default from both routers) to let BIRD correctly resolve nexthop. According to your BGP_PEERS template 'import none' you do not import anything from routers.
Since I only want to export routes for IPs in the loopback interface, I thought I can safely skip other interfaces for protocol direct. Well, with the filters in place, I don't worry for accidentally announcing routes that I shouldn't. Furthermore, as a safeguard I have set the upstream routers to allow announcements only for the specific prefix(192.168.200.0/24). Oh yes I play very safe here:-)
# Do not import from other sources. interface "-*";
# Import only allowed IPs from the "lo" and route from the upstream # interface. import where net ~ ACAST_PS_DIRECT1 || ifname
="<interface_to_your_routers>";
I don't need to import anything. Bird is used only as an advertisement system.
Yes, if you use BIRD to avertise prefixes only this is unnecessary and could be skipped.
I configured bird as you suggested[1] and after a restart I removed 192.168.200.1/32 from the prefix set[2]
See below the log
13:29:18 <INFO> Reconfiguration requested by SIGHUP 13:29:18 <INFO> Reconfiguring 13:29:18 <TRACE> direct1: Reconfigured 13:29:18 <TRACE> bgp_peer1: Reconfigured 13:29:18 <INFO> Reloading protocol bgp_peer1 13:29:18 <TRACE> bgp_peer1: State changed to feed 13:29:18 <TRACE> bgp_peer2: Reconfigured 13:29:18 <INFO> Reloading protocol bgp_peer2 13:29:18 <TRACE> bgp_peer2: State changed to feed 13:29:18 <INFO> Reconfigured 13:29:18 <TRACE> bgp_peer1 < filtered out 192.168.200.1/32 dev lo 13:29:18 <TRACE> bgp_peer1 < removed 192.168.200.1/32 dev lo 13:29:18 <TRACE> bgp_peer1 < replaced 192.168.200.2/32 dev lo 13:29:18 <TRACE> bgp_peer1: State changed to up 13:29:18 <TRACE> bgp_peer2 < filtered out 192.168.200.1/32 dev lo 13:29:18 <TRACE> bgp_peer2 < removed 192.168.200.1/32 dev lo 13:29:18 <TRACE> bgp_peer2 < replaced 192.168.200.2/32 dev lo 13:29:18 <TRACE> bgp_peer2: State changed to up
Seems correct, now prefix withdrawn from both peers, without affecting direct1 protocol.
what the meaning of the 'replaced' here?
Prefix is known before filter applied and filter changes does not remove or add such prefix, but could modify one of it's attributes (e.g. bgp_path, community, ...), so this case named 'replce' in routing protocol debugs. This is just trace message has no real impact and thus does not trigger any updates, unless you modify one of prefix attrinutes (bgp_path, community, etc).
and the output of birdcl show protocols all \"bgp*\" in a loop
### Mon Aug 4 13:29:11 CEST 2014 ### Routes: 0 imported, 2 exported, 0 preferred Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 2 0 0 --- 2 Export withdraws: 0 --- --- --- 0 Routes: 0 imported, 2 exported, 0 preferred Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 2 0 0 --- 2 Export withdraws: 0 --- --- --- 0 ### Mon Aug 4 13:29:16 CEST 2014 ### Routes: 0 imported, 1 exported, 0 preferred Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 4 0 1 --- 3 Export withdraws: 0 --- --- --- 1 Routes: 0 imported, 1 exported, 0 preferred Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 4 0 1 --- 3 Export withdraws: 0 --- --- --- 1 ### Mon Aug 4 13:29:21 CEST 2014 ###
Thank you very much for taking the time to provide a complete solution for me. It is very much appreciated.
Cheers, Pavlos
[1] bird.conf
include "/etc/bird.d/anycast_prefixes.conf";
define ACAST_PS_DIRECT1 = [ 192.168.200.0/24{32,32} ];
function anycast_advertise() { return net ~ ACAST_PS_ADVERTISE; }
filter anycast_range { if anycast_advertise() then accept;
reject; } Actually having named filter (anycast_range) also not strictly necessary: you could use 'export where anycast_advertise()' in BGP_PEERS instead of 'export filter anycast_range'. See BGP_PEERS.
This is up to your choice :-).
router id 192.168.88.194; listen bgp address 192.168.88.194;
protocol direct { interface "lo"; debug all; import where net ~ ACAST_PS_DIRECT1; export none; }
protocol kernel kernel1 { disabled yes; } This is not necessary if you do not plan to install routes into kernel routing tables. May be removed safely.
protocol device { scan time 10; }
protocol static { disabled yes; }
protocol bfd { interface "eth0" { min rx interval 500 ms; min tx interval 500 ms; idle tx interval 1000 ms; multiplier 3; }; }
template bgp BGP_PEERS { bfd on; debug all; import none; export filter anycast_range; Simpler version: export where anycast_advertise();
direct; hold time 30; startup hold time 240; connect retry time 120; keepalive time 10; start delay time 5; error wait time 60, 300; error forget time 300; disable after error off; next hop self; path metric 1; default bgp_med 0; default bgp_local_pref 0; }
protocol bgp bgp_peer1 from BGP_PEERS { disabled no; description "My-BGP-Peer1"; local as 46111;
Also could be moved in template (and as many other parameters overwritten in actual protocol configuration).
neighbor 192.168.95.252 as 46111; source address 192.168.88.194;
'source address' could be eliminated by using 'local' extended syntax: local 192.168.88.194 as 46111;
}
protocol bgp bgp_peer2 from BGP_PEERS { disabled no; description "My-BGP-Peer2"; local as 46111; neighbor 192.168.95.253 as 46111; source address 192.168.88.194; }
[2] /etc/bird.d/anycast_prefixes.conf define ACAST_PS_ADVERTISE = [ 192.168.200.1/32, # Frontend 1 192.168.200.2/32 # Frontend 2 ];
-- SP5474-RIPE Sergey Popovich
On 04/08/2014 03:09 μμ, Sergey Popovich wrote: [..snip..]
I configured bird as you suggested[1] and after a restart I removed 192.168.200.1/32 from the prefix set[2]
See below the log
13:29:18 <INFO> Reconfiguration requested by SIGHUP 13:29:18 <INFO> Reconfiguring 13:29:18 <TRACE> direct1: Reconfigured 13:29:18 <TRACE> bgp_peer1: Reconfigured 13:29:18 <INFO> Reloading protocol bgp_peer1 13:29:18 <TRACE> bgp_peer1: State changed to feed 13:29:18 <TRACE> bgp_peer2: Reconfigured 13:29:18 <INFO> Reloading protocol bgp_peer2 13:29:18 <TRACE> bgp_peer2: State changed to feed 13:29:18 <INFO> Reconfigured 13:29:18 <TRACE> bgp_peer1 < filtered out 192.168.200.1/32 dev lo 13:29:18 <TRACE> bgp_peer1 < removed 192.168.200.1/32 dev lo 13:29:18 <TRACE> bgp_peer1 < replaced 192.168.200.2/32 dev lo 13:29:18 <TRACE> bgp_peer1: State changed to up 13:29:18 <TRACE> bgp_peer2 < filtered out 192.168.200.1/32 dev lo 13:29:18 <TRACE> bgp_peer2 < removed 192.168.200.1/32 dev lo 13:29:18 <TRACE> bgp_peer2 < replaced 192.168.200.2/32 dev lo 13:29:18 <TRACE> bgp_peer2: State changed to up
Seems correct, now prefix withdrawn from both peers, without affecting direct1 protocol.
what the meaning of the 'replaced' here?
Prefix is known before filter applied and filter changes does not remove or add such prefix, but could modify one of it's attributes (e.g. bgp_path, community, ...), so this case named 'replce' in routing protocol debugs.
This is just trace message has no real impact and thus does not trigger any updates, unless you modify one of prefix attrinutes (bgp_path, community, etc).
OK, thanks once again for the explanation.
and the output of birdcl show protocols all \"bgp*\" in a loop
### Mon Aug 4 13:29:11 CEST 2014 ### Routes: 0 imported, 2 exported, 0 preferred Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 2 0 0 --- 2 Export withdraws: 0 --- --- --- 0 Routes: 0 imported, 2 exported, 0 preferred Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 2 0 0 --- 2 Export withdraws: 0 --- --- --- 0 ### Mon Aug 4 13:29:16 CEST 2014 ### Routes: 0 imported, 1 exported, 0 preferred Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 4 0 1 --- 3 Export withdraws: 0 --- --- --- 1 Routes: 0 imported, 1 exported, 0 preferred Import updates: 0 0 0 0 0 Import withdraws: 0 0 --- 0 0 Export updates: 4 0 1 --- 3 Export withdraws: 0 --- --- --- 1 ### Mon Aug 4 13:29:21 CEST 2014 ###
Thank you very much for taking the time to provide a complete solution for me. It is very much appreciated.
Cheers, Pavlos
[1] bird.conf
include "/etc/bird.d/anycast_prefixes.conf";
define ACAST_PS_DIRECT1 = [ 192.168.200.0/24{32,32} ];
function anycast_advertise() { return net ~ ACAST_PS_ADVERTISE; }
filter anycast_range { if anycast_advertise() then accept;
reject; } Actually having named filter (anycast_range) also not strictly necessary: you could use 'export where anycast_advertise()' in BGP_PEERS instead of 'export filter anycast_range'. See BGP_PEERS.
This is up to your choice :-).
Guess what? I changed it exactly as you suggest after I sent my previous mail:-)
router id 192.168.88.194; listen bgp address 192.168.88.194;
protocol direct { interface "lo"; debug all; import where net ~ ACAST_PS_DIRECT1; export none; }
protocol kernel kernel1 { disabled yes; } This is not necessary if you do not plan to install routes into kernel routing tables. May be removed safely.
I am not, but I disabled it because export is enabled by default and I don't want 'accidentally' install routes imported by BGP into kernel. OK I know import is None in my BGP template, but once again I play it safe here:-)
protocol device { scan time 10; }
protocol static { disabled yes; }
protocol bfd { interface "eth0" { min rx interval 500 ms; min tx interval 500 ms; idle tx interval 1000 ms; multiplier 3; }; }
template bgp BGP_PEERS { bfd on; debug all; import none; export filter anycast_range; Simpler version: export where anycast_advertise();
direct; hold time 30; startup hold time 240; connect retry time 120; keepalive time 10; start delay time 5; error wait time 60, 300; error forget time 300; disable after error off; next hop self; path metric 1; default bgp_med 0; default bgp_local_pref 0; }
protocol bgp bgp_peer1 from BGP_PEERS { disabled no; description "My-BGP-Peer1"; local as 46111;
Also could be moved in template (and as many other parameters overwritten in actual protocol configuration).
Done. I am wondering why I left it here.
neighbor 192.168.95.252 as 46111; source address 192.168.88.194;
'source address' could be eliminated by using 'local' extended syntax: local 192.168.88.194 as 46111;
}
protocol bgp bgp_peer2 from BGP_PEERS { disabled no; description "My-BGP-Peer2"; local as 46111; neighbor 192.168.95.253 as 46111; source address 192.168.88.194; }
Sergey, thanks a lot for your feedback. Cheers, Pavlos
On Mon, Aug 04, 2014 at 10:53:27AM +0300, Sergey Popovich wrote:
Hi,
Hello.
The problem I am facing is that when I adjust a filter in order to withdraw the route, all routes are withdrawn and the remaining are announced again. According to the doc if a reload on bird brings a new configuration for a protocol, that protocol will be restarted. So, it is expected behavior.
You have changed import filter of direct1 protocol, thus direct1 gets reconfigured (on HUP or birdc configure). However this protocol does not perform "soft" reconfiguration on filter changes thus direct1 restarts unconditionally to apply your filter changes (IP removal/addition).
All routes flushed out from the table (causing BGP withdrawal) and fetched from the protocol (direct) again to the table (causing BGP update).
Yes, that is the reason, direct protocol is limited in this regard. Simple workaround would be to put the filter as export filter to BGP instead of import filter from direct (as in your example). If filtered routes would be needed for more purposes, it is possible to attach direct to separate table and put filters to a pipe connecting these two tables. Just some minor comments:
Also I suggest you to use global prefix set definition, as local variable (in filter/function) is created/assigned each time function is called.
That is not true - (prefix and other) sets are created during parsing of the config file (that is the reason why they are read only and cannot be dynamically parametrized). Although i would usually suggest to put these prefix definitions as global variables anyways.
# This may be is needed to properly resolve nexthops from upstream routes. interface "<interface_to_your_routers>";
You need this only if you import something from the your routers (for example default from both routers) to let BIRD correctly resolve nexthop.
This is not necessary since 1.4.0. In the original configuration there is a 'direct' option in BGP template, which also by default chooses the direct (instead of recursive) resolution of next hops (as done by default in EBGP), -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
participants (5)
-
Eric Cables -
Ondrej Zajicek -
Pavlos Parissis -
Sargun Dhillon -
Sergey Popovich