- Bird-users - Mailing Lists

Minimalist BGP Anycast Lab Built from VMs?
by Traiano Welcome 05 Oct '16

05 Oct '16

Hi List What is the simplest BGP anycast lab I can set up using Linux VMs running on virtualbox? I'm trying to build the minimum virtual lab to test an Anycast configuration based on BGP. I have enough computing power to run about 4 linux VMs on Virtualbox on a single laptop. I'm using BIRD as a route server (not Quagga, not Zebra) on Linux. Is this even viable? I'd appreciate it if anyone could share their experiences/configurations, diagrams if they've set up a similar lab. Thanks in advance, Traiano

5 5

protocol kernel segfault
by Julien Sansonnens 29 Sep '16

29 Sep '16

Hi list, I'm using bird 1.6.1 on debian Jessie. For some reason (COULD be related to updating to 1.6.1, but not sure), bird6 is not working anymore. It used to work some weeks ago. The problem is related to "protocol kernel". When I set it like this: protocol kernel { import none; export all; } it crashed bird6. Everytime I get this segfault: Sep 29 12:24:11 mymachine kernel: [ 8588.114910] bird6[2974]: segfault at 0 ip (null) sp 00007ffd04645608 error 14 in libnss_files-2.19.so[ 7f91281db000+b000] When I remove this protocol, everything is working fine. Please can you help ? Thanks Julien

2 1

OSPF "hack translation" sought
by Clemens Schrimpe 29 Sep '16

29 Sep '16

Gentlepeople - I was wondering what the "translation" of the usual hack on Quagga/JunOS/etc. for OSPF into a proper BIRD config was to allow OSPF to propagate routes to directly connected networks, which are not really part of an OSPF area. In those other systems I would make those interfaces just "passive" members, but BIRD doesn't know "passive interfaces". Making them stubs has obvious other side effects, which is why I just want the "passive" hack to work under BIRD. So what's the equivalent to do this in BIRD? Thanks, Clemens Schrimpe

2 1

filter all prefixes except a few with 'case' control
by Войнович Андрей Александрович 29 Sep '16

29 Sep '16

Hi all! I try to implement simple filter which rejects all prefixes except a few with 'case' control this way: filter permit_white { case net { 1.1.1.1/32: accept; 2.2.2.2/32: accept; else: reject; } } But syntax check fails on line '1.1.1.1/32: accept;'. I read everything on Internet what was found, but cannot understand what is wrong. ------------------------------------------ Служба поддержки серверов Группа сетевого администрирования ДПП.УТП.СПС ЗАО ПФ <СКБ-Контур>, Тел. +7 (343) 344-11-50 доб. 75352 e-mail: dc-noc(a)skbkontur.ru<mailto:dc-noc@skbkontur.ru> https://www.kontur.ru<https://www.kontur.ru/>

4 5

BIRD patches for IP-in-IP
by Neil Jerram 29 Sep '16

29 Sep '16

Hi BIRD users! Attached are 3 patches that my team has been using for routing through IP-in-IP tunnels, rebased on 1.6.1. I'd like to explain why we find them useful, and start a conversation about whether they or something like them could be upstreamed (or perhaps if there's some better way of achieving our aims). Calico [1] uses BIRD for BGP routing between the hosts in various cloud orchestration systems (Kubernetes, OpenStack etc.), to distribute routes to the pods/VMs/containers in those systems, each of which has its own IP. If all the hosts are directly connected to each other, this is straightforward, but sometimes they are not. For example GCE instances are not directly connected to each other: there is at least one router between them, that knows about routing GCE addresses, and to/from the Internet, and we cannot peer with it or otherwise tell it how to route pod/VM/container IPs. So if we use GCE to create e.g. OpenStack compute hosts, with Calico networking, we need to do something extra to allow VM-addressed data to pass between the compute hosts. One of our solutions is to use IP-in-IP; it works as shown by this diagram: 10.65.0.3 via 10.240.0.5 dev tunl0 onlink default via 10.240.0.1 | +-|----------+ +------------+ | o | | | | Host A | +--------+ | Host B | | |---------| Router |----------| | | 10.240.0.4 | +--------+ | 10.240.0.5 | | |---. | | +------------+ | +------------+ ^ ^ +---v---+ | src 10.65.0.2 | | | tunl0 | | dst 10.65.0.3 | | +-------+ | | \ | v +-----------+ '----' +-----------+ | Pod A | src 10.240.0.4 | Pod B | | 10.65.0.2 | dst 10.240.0.5 | 10.65.0.3 | +-----------+ ------ +-----------+ src 10.65.0.2 dst 10.65.0.3 The diagram shows Pod A sending a packet to Pod B, using IP addresses that are unknown to the 'Router' between the two hosts. Host A has an IP-in-IP device, tunl0, and a route that says to use that device for data to Pod B's address (10.65.0.3). When the packet has passed through that device, it has a new outer IP header, with src 10.240.0.4 and dst 10.240.0.5, and is routed again according to the routing table - so now it can successfully reach Host B. So how is BIRD involved? We statically program the local Pod route on each host: On Host A: 10.65.0.2 dev <interface to Pod A> On Host B: 10.65.0.3 dev <interface to Pod B> then run a BIRD BGP session between Host A and Host B to propagate those routes to the other host - which would normally give us: On Host A: 10.65.0.3 via 10.240.0.5 On Host B: 10.65.0.2 via 10.240.0.4 But we don't want those normal routes, because then the data would get lost at 'Router'. So we enhance and configure BIRD as follows. - In the export filter for protocol kernel, for the relevant routes, we set an attribute 'krt_tunnel = tunl0'. - We modify BIRD, as in the attached patches, to understand that that means that those routes should have 'dev tunl0'. Then instead, we get: On Host A: 10.65.0.3 via 10.240.0.5 dev tunl0 onlink On Host B: 10.65.0.2 via 10.240.0.4 dev tunl0 onlink which allows successful routing of data between the Pods. Thanks for reading this far! I now have three questions: 1. Does the routing approach above make sense? (Or is there some better or simpler or already supported way that we could achieve the same thing?) 2. If (1), would the BIRD team accept patches broadly on the lines of those that are attached? 3. If (2), please let me know if the attached patches are already acceptable, or otherwise what further work is needed for them. Many thanks, Neil

5 9

Problems after upgrade to 1.6.1 from 1.6.0
by Ralf Hildebrandt 28 Sep '16

28 Sep '16

I upgraded BIRD from a working 1.6.0-3+trusty+1 to 1.6.1-1+trusty+1 today (from the Ubuntu PPA https://launchpad.net/~cz.nic-labs/+archive/ubuntu/bird ) Then bird started to segfault: Sep 28 11:46:24 dns-ccm kernel: [4320833.655332] bird[10188]: segfault at 0 ip (null) sp 00007fff3f9624a8 error 14 in bird[5654a2004000+70000] Sep 28 11:46:24 dns-ccm kernel: [4320833.685949] bird[16543]: segfault at 0 ip (null) sp 00007ffe7e13f2c8 error 14 in bird[55d29a334000+70000] I rebuilt the package from the PPA sources without debugging and ran bird in gdb: # gdb ./build-ipv4/bird (gdb) set args -f -u bird -g bird (gdb) run Starting program: /usr/src/bird-1.6.1/build-ipv4/bird -f -u bird -g bird [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". 2016-09-28 15:00:31 <INFO> Chosen router ID 141.42.1.11 according to interface bond0 2016-09-28 15:00:31 <INFO> Started Program received signal SIGSEGV, Segmentation fault. 0x0000000000000000 in ?? () (gdb) bt #0 0x0000000000000000 in ?? () #1 0x00005555555b8812 in sk_err (s=0x5555558077d0, revents=9) at io.c:1867 #2 0x00005555555b93f1 in io_loop () at io.c:2237 #3 0x00005555555bec05 in main (argc=6, argv=0x7fffffffe5e8) at main.c:883 (gdb) bt full #0 0x0000000000000000 in ?? () No symbol table info available. #1 0x00005555555b8812 in sk_err (s=0x5555558077d0, revents=9) at io.c:1867 se = 0 sse = 4 #2 0x00005555555b93f1 in io_loop () at io.c:2237 s = 0x5555558077d0 count = 1 poll_tout = 1000 tout = 4332491 nfds = 3 events= 0 pout = 1 s = 0x55555581ac20 n = 0x5555557f3d88 <sock_list+8> fdmax = 256 pfd = 0x5555558120c0 #3 0x00005555555bec05 in main (argc=6, argv=0x7fffffffe5e8) at main.c:883 use_uid = 115 use_gid = 128 conf = 0x5555557f8b80 (gdb) My config: ========== ---- SNIP ---- log stderr all; router id from "bond0"; # Funktion in der die verwendeten ANYCAST-IPs eingetragen werden muessen function is_anycastIP () { return net ~ [ 141.42.1.1/32, 141.42.1.2/32 ]; } # Direkte Interface-Routen (z.B. die an "lo" gebundene ANYCAST-IP) lernen protocol direct { interface "lo"; } # Exportieren der OSPF-Routen an die Kernel-Routingtable, keine Imports! protocol kernel { scan time 20; import none; export all; preference 100; } # Lokale Devices/Interfaces einbeziehen. # u.a. fuer die automatische Erkennung der RouterID notwendig protocol device { scan time 10; } protocol ospf ANYCAST { rfc1583compat yes; tick 2; preference 50; ecmp yes; import filter { # Default-Route und die von den anderen kommenden ANYCAST-IPs nicht aus OSPF importieren if ((net = 0.0.0.0/0) || is_anycastIP()) then { reject; } accept; }; export filter { # Nur die ANYCAST-IPs ins OSPF propagieren if ((source = RTS_DEVICE) && is_anycastIP()) then { # E1-Route mit Kosten=5000 erzeugen ospf_metric1 = 5000; accept; } reject; }; area 0 { interface "bond0" { type broadcast; hello 10; retransmit 5; wait 40; dead 40; check link; priority 0; cost 5000; authentication cryptographic; password "SECRET"; }; }; } ---- SNIP ---- -- Ralf Hildebrandt Charite Universitätsmedizin Berlin ralf.hildebrandt(a)charite.de Campus Benjamin Franklin http://www.charite.de Hindenburgdamm 30, 12203 Berlin Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155

2 2

Version 1.6.1
by Ondrej Filip 27 Sep '16

27 Sep '16

Dear users, a new version of the 1.6.x branch is out: Version 1.6.1 (2016-09-22) o Support for IPv6 ECMP o Better handling of IPv6 tentative addresses o Several updates and fixes in Babel protocol o Filter: New !~ operator o Filter: ASN ranges in bgpmask o KRT: New kernel protocol option 'metric' o KRT: New route attribute 'krt_scope' o Improved BIRD help messages o Fixes memory leak in BGP multipath o Fixes handling of empty path segments in BGP AS_PATH o Several bug fixes Enjoy it! Ondrej

8 12

Improving Anycast routing with Bird
by Kyle Drake 22 Sep '16

22 Sep '16

Hello! I'm new to using Bird and BGP, but I'm learning. I've recently acquired an IPv4/24, and I've been working on building an Anycast network for CDN use. Right now I have a 15 datacenter network from a VPS provider (which uses Bird as it's recommended daemon: https://www.vultr.com/docs/configuring-bgp-on-vultr) Using their configuration example, I was able to get an anycast network up without any modification. That part is working great! The problem I have now is figuring out how to improve the routing. For example, Comcast users in Portland, OR are being sent to Los Angeles, even though the closest server is in Seattle (confirmed with traceroute). The underlying BGP problem seems to be that Comcast sees multiple routes as 2 AS routes (7922 20473), so it doesn't understand that the Seattle instance is preferred here. There are many more examples like this I've found, and I'm trying to figure out what an approach might be to try to improve this. Is there a way to improve routing using bird export filters or some other mechanism I'm not familiar with? Can I use Bird to steer certain traffic to closer servers, or to only announce routes to regional BGP servers? Or is this largely out of my control and I need to accept these non-ideal routes? I've also been told I might be able to use BGP communities for this, but I've never worked with communities directly, so I don't understand how they would help me here. This is the list of communities I have available to me: https://www.vultr.com/docs/as20473-bgp-customer-guide Thank you in advance for any ideas or code examples here. There is zero documentation on tuning Anycast out there (even the old school BGP books I've skimmed through barely mention anycast). Here is my current bird.conf: log syslog all; router id **MY_IP**; protocol bgp vultr { local as 395409; source address **MY_IP**; import none; export all; graceful restart on; multihop 2; neighbor **BGP_IP** as 64515; password "**OBSCURED**"; } protocol static { route 198.51.233.0/24 via **MY_IP**; } protocol device { scan time 5; } protocol direct { interface "neocities*"; import all; } -Kyle

3 4

BIRD memory usage
by Justin Cattle 19 Sep '16

19 Sep '16

Hi, A colleague of mine reported a memory usage issue with the bird daemon last year, which resulted in a request for a core dump, but we never followed it up. I'd like to re-open this discussion and see if anything can be done to fix it. I'll provide some information regarding a production environment, where the problem is most obvious. But any further details and diagnostics will have to come from our lab environment. Please note, in production we mostly run 1.5, but in the lab we are on 1.6, however we see the same symptoms in both environments on both versions. The symptoms are twofold, but potentially related - greater than expected memory usage reported by the bird daemon itself for the number of routes, but also massively more memory actually used by the daemon process. When the process is started, we see "normal" memory usage, which then seems to grow indefinitely in distinct steps, separated by a period of a few hours. In production, this consumes most of the 32G of memory until the kernel oom-killer to intervenes. Production: BIRD 1.5.0 ready. bird> show memory BIRD memory usage Routing tables: 1405 MB Route attributes: 84 kB ROA tables: 192 B Protocols: 45 kB Total: 1405 MB bird> show route count 2273 of 2273 routes for 1142 networks # ps u -p 3441 USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND bird 3441 0.1 55.4 18275124 18241540 ? Ssl Aug10 73:39 /usr/sbin/bird -f -u bird -g bird ..so that's ~1.4G reported by bird, and ~18G actually consumed by the process. Lab: BIRD 1.6.0 ready. bird> show mem BIRD memory usage Routing tables: 693 MB Route attributes: 28 kB ROA tables: 192 B Protocols: 41 kB Total: 693 MB bird> show route count 175 of 175 routes for 91 networks # ps u -p 29085 USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND bird 29085 0.0 14.9 4994852 4915032 ? Ssl Aug05 19:41 /usr/sbin/bird -f -u bird -g bird ..so that's ~ 0.7G reported by bird, and ~5G actually consumed by the process. I also attached the bird config from the lab. Any help is much appreciated! Thanks. Cheers, Just -- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group. References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

3 15

[PATCH] Netlink: add krt_scope attribute
by Vincent Bernat 16 Sep '16

16 Sep '16

The general-purpose scope attribute is detailed in the documentation as an attribute that BIRD won't set and won't use. Therefore, to expose the scope of a kernel route, this commit adds a new attribute, krt_scope. Its possible values are the numeric values for SCOPE_* variables (1 is SCOPE_LINK for example), not the values from the kernel (253 is RT_SCOPE_LINK). Both import and export are supported. A typical use case is to install "scope link" routes for directly connected destinations, as the kernel won't accept routes with a broader scope when issuing an internal lookup. It would have been neat to be able to use "SCOPE_LINK" in a filter or to show "SCOPE_LINK" when displaying the route, but currently, only the numerical value can be used. Signed-off-by: Vincent Bernat <vincent(a)bernat.im> --- sysdep/linux/krt-sys.h | 1 + sysdep/linux/netlink.Y | 3 ++- sysdep/linux/netlink.c | 40 ++++++++++++++++++++++++++++++++++------ 3 files changed, 37 insertions(+), 7 deletions(-) diff --git a/sysdep/linux/krt-sys.h b/sysdep/linux/krt-sys.h index 96688e34c860..6d6586d10f02 100644 --- a/sysdep/linux/krt-sys.h +++ b/sysdep/linux/krt-sys.h @@ -36,6 +36,7 @@ static inline struct ifa * kif_get_primary_ip(struct iface *i) { return NULL; } #define EA_KRT_PREFSRC EA_CODE(EAP_KRT, 0x10) #define EA_KRT_REALM EA_CODE(EAP_KRT, 0x11) +#define EA_KRT_SCOPE EA_CODE(EAP_KRT, 0x12) #define KRT_METRICS_MAX 0x10 /* RTAX_QUICKACK+1 */ diff --git a/sysdep/linux/netlink.Y b/sysdep/linux/netlink.Y index a1c22f3ece17..c1c9503c33ec 100644 --- a/sysdep/linux/netlink.Y +++ b/sysdep/linux/netlink.Y @@ -10,7 +10,7 @@ CF_HDR CF_DECLS -CF_KEYWORDS(KERNEL, TABLE, METRIC, KRT_PREFSRC, KRT_REALM, KRT_MTU, KRT_WINDOW, +CF_KEYWORDS(KERNEL, TABLE, METRIC, KRT_PREFSRC, KRT_REALM, KRT_SCOPE, KRT_MTU, KRT_WINDOW, KRT_RTT, KRT_RTTVAR, KRT_SSTRESH, KRT_CWND, KRT_ADVMSS, KRT_REORDERING, KRT_HOPLIMIT, KRT_INITCWND, KRT_RTO_MIN, KRT_INITRWND, KRT_QUICKACK, KRT_LOCK_MTU, KRT_LOCK_WINDOW, KRT_LOCK_RTT, KRT_LOCK_RTTVAR, @@ -28,6 +28,7 @@ kern_sys_item: CF_ADDTO(dynamic_attr, KRT_PREFSRC { $$ = f_new_dynamic_attr(EAF_TYPE_IP_ADDRESS, T_IP, EA_KRT_PREFSRC); }) CF_ADDTO(dynamic_attr, KRT_REALM { $$ = f_new_dynamic_attr(EAF_TYPE_INT, T_INT, EA_KRT_REALM); }) +CF_ADDTO(dynamic_attr, KRT_SCOPE { $$ = f_new_dynamic_attr(EAF_TYPE_INT, T_ENUM_SCOPE, EA_KRT_SCOPE); }) CF_ADDTO(dynamic_attr, KRT_MTU { $$ = f_new_dynamic_attr(EAF_TYPE_INT, T_INT, EA_KRT_MTU); }) CF_ADDTO(dynamic_attr, KRT_WINDOW { $$ = f_new_dynamic_attr(EAF_TYPE_INT, T_INT, EA_KRT_WINDOW); }) diff --git a/sysdep/linux/netlink.c b/sysdep/linux/netlink.c index 9bdcc0d2ff19..9f73e0feeeaf 100644 --- a/sysdep/linux/netlink.c +++ b/sysdep/linux/netlink.c @@ -900,6 +900,15 @@ nl_send_route(struct krt_proto *p, rte *e, struct ea_list *eattrs, int op, int d r.r.rtm_dst_len = net->n.pxlen; r.r.rtm_protocol = RTPROT_BIRD; r.r.rtm_scope = RT_SCOPE_UNIVERSE; + if (ea = ea_find(eattrs, EA_KRT_SCOPE)) + switch (ea->u.data) + { + case SCOPE_HOST: r.r.rtm_scope = RT_SCOPE_HOST; break; + case SCOPE_LINK: r.r.rtm_scope = RT_SCOPE_LINK; break; + case SCOPE_SITE: r.r.rtm_scope = RT_SCOPE_SITE; break; + case SCOPE_UNIVERSE: r.r.rtm_scope = RT_SCOPE_UNIVERSE; break; + } + nl_add_attr_ipa(&r.h, sizeof(r), RTA_DST, net->n.prefix); /* @@ -1157,7 +1166,6 @@ nl_parse_route(struct nl_parse_state *s, struct nlmsghdr *h) return; } - if (a[RTA_DST]) { memcpy(&dst, RTA_DATA(a[RTA_DST]), sizeof(dst)); @@ -1195,11 +1203,6 @@ nl_parse_route(struct nl_parse_state *s, struct nlmsghdr *h) if ((c < 0) || !(c & IADDR_HOST) || ((c & IADDR_SCOPE_MASK) <= SCOPE_LINK)) SKIP("strange class/scope\n"); - // ignore rtm_scope, it is not a real scope - // if (i->rtm_scope != RT_SCOPE_UNIVERSE) - // SKIP("scope %u\n", i->rtm_scope); - - switch (i->rtm_protocol) { case RTPROT_UNSPEC: @@ -1304,6 +1307,27 @@ nl_parse_route(struct nl_parse_state *s, struct nlmsghdr *h) return; } + int krt_scope = -1; + switch (i->rtm_scope) + { + case RT_SCOPE_HOST: krt_scope = SCOPE_HOST; break; + case RT_SCOPE_LINK: krt_scope = SCOPE_LINK; break; + case RT_SCOPE_SITE: krt_scope = SCOPE_SITE; break; + case RT_SCOPE_UNIVERSE: krt_scope = SCOPE_UNIVERSE; break; + } + if (krt_scope != -1) + { + ea_list *ea = lp_alloc(s->pool, sizeof(ea_list) + sizeof(eattr)); + ea->next = ra->eattrs; + ra->eattrs = ea; + ea->flags = EALF_SORTED; + ea->count = 1; + ea->attrs[0].id = EA_KRT_SCOPE; + ea->attrs[0].flags = 0; + ea->attrs[0].type = EAF_TYPE_INT; + ea->attrs[0].u.data = krt_scope; + } + if (a[RTA_PREFSRC]) { ip_addr ps; @@ -1633,6 +1657,10 @@ krt_sys_get_attr(eattr *a, byte *buf, int buflen UNUSED) bsprintf(buf, "realm"); return GA_NAME; + case EA_KRT_SCOPE: + bsprintf(buf, "scope"); + return GA_NAME; + case EA_KRT_LOCK: buf += bsprintf(buf, "lock:"); ea_format_bitfield(a, buf, buflen, krt_metrics_names, 2, KRT_METRICS_MAX); -- 2.9.3

3 7