Bird Centos 8 Replacement recommendations request
Hiya Guys, Apologies if not the right approach/ protocol, do let me know what is if required. Asks: Pls provide.... 1.Details of what OS and version supported by Bird 2.Any OS preferences or best choice of useful. 3.Any views thoughts on below useful Our draft shortlist is Debian 10.7 Buster Ubuntu 20.04 LTS Leap 15.3 Note did not consider any of the BSD variants, which is an oversight so should be included Background We were close to productionising a network stack around number of sites using Bird2.07 as the Routing daemon on Centos 8. In light of the Centos changes are now looking for a replacement and would like to confirm what is supported by Bird as this is a mandatory prerequisite. Happy for call etc if easier. Thks in advance FYI -I did look at wiki FAQs and was semi generic and possibly not up to date Thks Will +44(0)7967 464459
Hi Will, We proudly use BIRD2 on Debian stable in production with full feed BGP sessions and OSPF. Besides some kernel tweaking (needed in other distributions as well) no special requirements. Cheers, Kees On 17-03-2021 11:35, W Templeton wrote:
Our draft shortlist is Debian 10.7 Buster Ubuntu 20.04 LTS Leap 15.3
Hiya Kees, Thks for quick response. Pls can you clarify 1.your OS version 2.Debian kernel tweaking Thks ________________________________ From: Kees Meijs | Nefos <kees@nefos.nl> Sent: 17 March 2021 10:49 To: W Templeton <wnt@btconnect.com>; BIRD Users <bird-users@network.cz> Cc: Will Templeton <wtempleton@tbscg.com> Subject: Re: Bird Centos 8 Replacement recommendations request Hi Will, We proudly use BIRD2 on Debian stable in production with full feed BGP sessions and OSPF. Besides some kernel tweaking (needed in other distributions as well) no special requirements. Cheers, Kees On 17-03-2021 11:35, W Templeton wrote: Our draft shortlist is Debian 10.7 Buster Ubuntu 20.04 LTS Leap 15.3
Hi, On 17-03-2021 11:53, W Templeton wrote:
1.your OS version
Sure:
$ cat /etc/debian_version 10.8
Basicly: Debian stable with current updates.
2.Debian kernel tweaking
Sure:
# cat 99-router.conf # Enable IP forwarding for IPv4 and IPv6 AF. net.ipv4.conf.all.forwarding = 1 net.ipv6.conf.all.forwarding = 1
# Send ICMP errors from receiving interface and be a nice netizen. net.ipv4.icmp_errors_use_inbound_ifaddr = 1
# Disable reverse path filtering. net.ipv4.conf.all.rp_filter = 0 net.ipv4.conf.default.rp_filter = 0
# Allow more IPv6 routes. net.ipv6.route.max_size = 2147483647
# Allow more network neighbours. net.ipv4.neigh.default.gc_thresh1=16384 net.ipv4.neigh.default.gc_thresh2=32768 net.ipv4.neigh.default.gc_thresh3=65536 net.ipv6.neigh.default.gc_thresh1=16384 net.ipv6.neigh.default.gc_thresh2=32768 net.ipv6.neigh.default.gc_thresh3=65536
# cat 99-tengige.conf # 10GB/54MB (56623104) net.core.rmem_max = 56623104 net.core.wmem_max = 56623104 net.core.rmem_default = 56623104 net.core.wmem_default = 56623104 net.core.optmem_max = 40960 net.ipv4.tcp_rmem = 4096 87380 56623104 net.ipv4.tcp_wmem = 4096 65536 56623104
# Increase number of incoming connections. The value can be raised to bursts of request, default is 128 net.core.somaxconn = 1024
# Increase number of incoming connections backlog, default is 1000 net.core.netdev_max_backlog = 50000
# Maximum number of remembered connection requests, default is 128 net.ipv4.tcp_max_syn_backlog = 30000
# Increase the tcp-time-wait buckets pool size to prevent simple DoS attacks, default is 8192 net.ipv4.tcp_max_tw_buckets = 2000000
# Recycle and Reuse TIME_WAIT sockets faster, default is 0 for both net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_tw_reuse = 1
# Decrease TIME_WAIT seconds, default is 30 seconds net.ipv4.tcp_fin_timeout = 10
# Tells the system whether it should start at the default window size only for TCP connections # that have been idle for too long, default is 1 net.ipv4.tcp_slow_start_after_idle = 0
# If your servers talk UDP, also up these limits, default is 4096 net.ipv4.udp_rmem_min = 8192 net.ipv4.udp_wmem_min = 8192
# cat 99-vrrp.conf # For VRRP. net.ipv4.ip_nonlocal_bind = 1
Cybertinus: maybe you can give insight in your kernel tweaks as well? (Probably very overlapping.) Cheers, Kees
Thks, I'm new to Bird and certainly Debian so pls bear with me. So below shows a number of Debian network stack tweaks for your specific use case, is that fair ? With kernel tweaks to follow from Cybertinus @Cybertinus - That would be appreciated. FYI Our setup is Anycast based Bird stacks distributed globally linking directly from virtual image to DC hosting provider who we peer with using BGP, with own ASN & IPv4/24 & IPV6/28. Routing wise 1 IPV4 subnet & 1 IPV6 advertised out to DC Host provider to announce on Internet. So basic routing table. Our host address will grow over time up to max 253/4 ish running as dummy entries (in Centos) which are utilised by HAproxy. So a corner case that appears relatively simple routing Bird wise. Appreciate any thoughts etc. Will ________________________________ From: Kees Meijs | Nefos <kees@nefos.nl> Sent: 17 March 2021 10:59 To: W Templeton <wnt@btconnect.com>; BIRD Users <bird-users@network.cz> Cc: Will Templeton <wtempleton@tbscg.com> Subject: Re: Bird Centos 8 Replacement recommendations request Hi, On 17-03-2021 11:53, W Templeton wrote: 1.your OS version Sure: $ cat /etc/debian_version 10.8 Basicly: Debian stable with current updates. 2.Debian kernel tweaking Sure: # cat 99-router.conf # Enable IP forwarding for IPv4 and IPv6 AF. net.ipv4.conf.all.forwarding = 1 net.ipv6.conf.all.forwarding = 1 # Send ICMP errors from receiving interface and be a nice netizen. net.ipv4.icmp_errors_use_inbound_ifaddr = 1 # Disable reverse path filtering. net.ipv4.conf.all.rp_filter = 0 net.ipv4.conf.default.rp_filter = 0 # Allow more IPv6 routes. net.ipv6.route.max_size = 2147483647 # Allow more network neighbours. net.ipv4.neigh.default.gc_thresh1=16384 net.ipv4.neigh.default.gc_thresh2=32768 net.ipv4.neigh.default.gc_thresh3=65536 net.ipv6.neigh.default.gc_thresh1=16384 net.ipv6.neigh.default.gc_thresh2=32768 net.ipv6.neigh.default.gc_thresh3=65536 # cat 99-tengige.conf # 10GB/54MB (56623104) net.core.rmem_max = 56623104 net.core.wmem_max = 56623104 net.core.rmem_default = 56623104 net.core.wmem_default = 56623104 net.core.optmem_max = 40960 net.ipv4.tcp_rmem = 4096 87380 56623104 net.ipv4.tcp_wmem = 4096 65536 56623104 # Increase number of incoming connections. The value can be raised to bursts of request, default is 128 net.core.somaxconn = 1024 # Increase number of incoming connections backlog, default is 1000 net.core.netdev_max_backlog = 50000 # Maximum number of remembered connection requests, default is 128 net.ipv4.tcp_max_syn_backlog = 30000 # Increase the tcp-time-wait buckets pool size to prevent simple DoS attacks, default is 8192 net.ipv4.tcp_max_tw_buckets = 2000000 # Recycle and Reuse TIME_WAIT sockets faster, default is 0 for both net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_tw_reuse = 1 # Decrease TIME_WAIT seconds, default is 30 seconds net.ipv4.tcp_fin_timeout = 10 # Tells the system whether it should start at the default window size only for TCP connections # that have been idle for too long, default is 1 net.ipv4.tcp_slow_start_after_idle = 0 # If your servers talk UDP, also up these limits, default is 4096 net.ipv4.udp_rmem_min = 8192 net.ipv4.udp_wmem_min = 8192 # cat 99-vrrp.conf # For VRRP. net.ipv4.ip_nonlocal_bind = 1 Cybertinus: maybe you can give insight in your kernel tweaks as well? (Probably very overlapping.) Cheers, Kees
On Mar 17, Kees Meijs | Nefos <kees@nefos.nl> wrote:
# Recycle and Reuse TIME_WAIT sockets faster, default is 0 for both net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_tw_reuse = 1 This is ***BAD***. tcp_tw_recycle is toxic because it breaks NATs, and has even been removed from recent kernels. Do not EVER set it. tcp_tw_reuse is only relevant for outgoing connections.
-- ciao, Marco
Hi Marco, Thanks for pointing out the issue. I'll look into that. However there's no NAT applicable in our specific case. In addition: no trouble so far. But again, good to re-investigate. Maybe it's a nice side thread on the list as well. I'm curious about real life kernel tweaking at other's platform. Cheers, Kees On 17-03-2021 12:55, Marco d'Itri wrote:
This is ***BAD***. tcp_tw_recycle is toxic because it breaks NATs, and has even been removed from recent kernels. Do not EVER set it. tcp_tw_reuse is only relevant for outgoing connections.
On Wed, 17 Mar 2021, W Templeton wrote:
We were close to productionising a network stack around number of sites using Bird2.07 as the Routing daemon on Centos 8. In light of the Centos changes are now looking for a replacement and would like to confirm what is supported by Bird as this is a mandatory prerequisite.
How about some more patience and switching then from CentOS Linux to Rocky Linux? They are making a good progress and the project is driven by the original founder of the CentOS project. Their goal is to provide a drop-in replacement using one or two commands - and without system re-installation. Regards, Robert
Hi, I've a wild mix of CentOS 7 and Arch Linux nodes running bird with haproxy. I'm in the process of migrating the CentOS boxes to Arch Linux. That just works ¯\_(ツ)_/¯ cheers, Tim On 17.03.21 11:35, W Templeton wrote:
Hiya Guys,
Apologies if not the right approach/ protocol, do let me know what is if required.
Asks: Pls provide.... 1.Details of what OS and version supported by Bird 2.Any OS preferences or best choice of useful. 3.Any views thoughts on below useful
Our draft shortlist is Debian 10.7 Buster Ubuntu 20.04 LTS Leap 15.3
Note did not consider any of the BSD variants, which is an oversight so should be included
Background We were close to productionising a network stack around number of sites using Bird2.07 as the Routing daemon on Centos 8. In light of the Centos changes are now looking for a replacement and would like to confirm what is supported by Bird as this is a mandatory prerequisite. Happy for call etc if easier. Thks in advance
FYI -I did look at wiki FAQs and was semi generic and possibly not up to date
Thks Will +44(0)7967 464459
participants (6)
-
Dmitry Melekhov -
Kees Meijs | Nefos -
Marco d'Itri -
Robert Scheck -
Tim Meusel -
W Templeton