Implementing RTBH filtering / BGP tagging
In working to streamline our utilization of each upstream's RTBH filtering mechanisms (e.g. http://www.he.net/adm/blackhole.html ), I'm having a heck of a time configuring BIRD to "mirror" Cisco product behaviors. Here's the Cisco Way (where X.X.X.X is the ip to blackhole): conf t ip route X.X.X.X 255.255.255.255 Null0 router bgp YourAS network X.X.X.X mask 255.255.255.255 route-map blackhole route-map blackhole permit 10 set community 6939:666 end What I believe needs to happen w/ BIRD, and the Linux Kernel route tables: 1) Create and maintain a non-default Linux kernel route table, to accomplish blackholing at our gateways (working great). Also wanting to have BIRD to monitor this non-default route table, and :666 tag+announce upstream any /32 entries within our 199.127.224.0/22. 2) BIRD imports the non-default kernel route table (seems to be working ok). 3) For each "protocol bgp *Upstream*", use an export filter to identify and tag relevant "blackhole" route entries with a specific RTBH community (e.g. 6939:666)...this is what I'm struggling with. While snippets are below, our entire bird.conf (minus comments) can be found at: http://pastebin.com/xE1AxtGe Here's my breakdown of what seems to be happening: 1) Create/edit/update the non-default kernel route table (works great, the Linux kernel/route config drops this traffic without issue): "... ~$ sudo ip rule add from all table 10 priority 10; sudo ip route add blackhole 199.127.225.73/32 table 10; ..... ~$ sudo ip route show table 10 |wc -l 456 ~$ sudo ip rule 0: from all lookup local 10: from all lookup 10 ..." 2) Read the non-default kernel route table (from bird.conf): "... table blackroutes; protocol kernel blackhole { table blackroutes; kernel table 10; scan time 10; learn; import all; export all; } ..." BIRD seems to import the non-default kernel table without issue: "... ~$ sudo birdc show protocols all |grep blackhole -C 10 blackhole Kernel blackroutes up Mar17 Preference: 10 Input filter: ACCEPT Output filter: ACCEPT Routes: 454 imported, 0 exported, 454 preferred Route change stats: received rejected filtered ignored accepted Import updates: 1363 0 0 0 1363 Import withdraws: 909 0 --- 0 909 Export updates: 1363 1363 0 --- 0 Export withdraws: 909 --- --- --- 0 ~$ sudo ip route show table 10 |wc -l 456 ..." BTW: the bulk of our "blacklist" entries come from http://www.spamhaus.org/drop/drop.lasso, although we only want to tag+announce what we control (e.g. /32's within 199.127.224.0/22). 3) Can't seem to figure out correct tagging method/commands (from bird.conf): "... filter bgp_out_he { if (proto = "blackhole" ) then { bgp_community = -empty-; bgp_community.add((6939,666)); } if net ~ [65.49.94.0/24] then accept; if net ~ [199.127.224.0/22] then accept; reject; } ... protocol bgp HE { local as 14613; source address 65.19.175.14; neighbor 65.19.175.13 as 6939; import all; ... export filter bgp_out_he; } ..." Am I missing something obvious? Going about this the wrong way? Thanks greatly for any help, Gregg Berkholtz
On Mon, 19 Mar 2012, Gregg Berkholtz wrote:
What I believe needs to happen w/ BIRD, and the Linux Kernel route tables: 1) Create and maintain a non-default Linux kernel route table, to accomplish blackholing at our gateways (working great). Also wanting to have BIRD to monitor this non-default route table, and :666 tag+announce upstream any /32 entries within our 199.127.224.0/22.
2) BIRD imports the non-default kernel route table (seems to be working ok).
3) For each "protocol bgp *Upstream*", use an export filter to identify and tag relevant "blackhole" route entries with a specific RTBH community (e.g. 6939:666)...this is what I'm struggling with.
I am not sure I understood exactly what you're trying to do, but... If you create a /32 discard route (using static), say, for 192.0.2.1, and in the import filter, set the nexthop of blackholed routes to that, bird should be smart enough to recursively resolve 192.0.2.1 to discard, and install a discard route instead of one to 192.0.2.1. I did not test the above, but bird certainly was happy enough to install unreachable routes (and not routes to an unreachable gateway) when I set the nexthop to an unreachable ip by mistake... So, if it doesn't do it for discard already, it is probably a valid feature request :-) A less optimized way is to have a dummy0 interface (you need support to the dummy network device compiled in or load the module), assign 192.0.2.1/24 to it, and send blackholed traffic to 192.0.2.2. This _is_ slower than a discard route, the packet will actually go across the entire forwarding stack to be finally sent to an interface that just discards it, instead of being discarded earlier in the stack. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
On Mon, Mar 19, 2012 at 04:59:59PM -0700, Gregg Berkholtz wrote:
In working to streamline our utilization of each upstream's RTBH filtering mechanisms (e.g. http://www.he.net/adm/blackhole.html ), I'm having a heck of a time configuring BIRD to "mirror" Cisco product behaviors.
Here's the Cisco Way (where X.X.X.X is the ip to blackhole): conf t ip route X.X.X.X 255.255.255.255 Null0 router bgp YourAS network X.X.X.X mask 255.255.255.255 route-map blackhole route-map blackhole permit 10 set community 6939:666 end
What I believe needs to happen w/ BIRD, and the Linux Kernel route tables: 1) Create and maintain a non-default Linux kernel route table, to accomplish blackholing at our gateways (working great). Also wanting to have BIRD to monitor this non-default route table, and :666 tag+announce upstream any /32 entries within our 199.127.224.0/22.
2) BIRD imports the non-default kernel route table (seems to be working ok).
3) For each "protocol bgp *Upstream*", use an export filter to identify and tag relevant "blackhole" route entries with a specific RTBH community (e.g. 6939:666)...this is what I'm struggling with.
I am not sure if i understad what you want. I suppose that for each route in blackhole table you want to export that prefix with 6939:666 community through BGP. For that, your current config has a problem that BGP protocol is connected to master table, so there is no way how blackhole routes could go from table blackroutes to the BGP protocol. So you should connect the master and blackroutes table through pipe (and add appropriate filtering to ensure that blackhole routes end only in BGP). Another solution is to add one more table for BGP, connect BGP to that table and add two pipes (master-this_new and blackroutes-this_new), this would make filtering simpler. Or much simpler solution - remove secondary tables, add blackhole routes to bird config as static routes (in static protocol) and have everything in the master table. BTW, in the filter bgp_out_he(), i guess you want accept all routes with proto = "blackhole", otherwise only your routes would be exported (and i suppose blackholed IPs are foreign). -- Elen sila lumenn' omentielvo Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
On Mar 20, 2012, at 10:36 AM, Ondrej Zajicek wrote:
I am not sure if i understad what you want. I suppose that for each route in blackhole table you want to export that prefix with 6939:666 community through BGP.
I think you nailed it on the head, but then again, I do kind of have a better insight on the process he's trying to do, from the other side. ;-)
Or much simpler solution - remove secondary tables, add blackhole routes to bird config as static routes (in static protocol) and have everything in the master table.
Which is exactly what I wound up doing when testing this sort of thing internally. Soft config reloads won't bounce the session when you change items in the static protocol. Good use for an include, generate the static protocol stub from scripts, and you now have a nice persistent state mechanism. Granted, I was also only using this for blackhole injections, and no other routes so the configuration can be really simple.
BTW, in the filter bgp_out_he(), i guess you want accept all routes with proto = "blackhole", otherwise only your routes would be exported (and i suppose blackholed IPs are foreign).
Blackholed IPs would actually have to be local. This mechanism is common for dropping a DDoS at your upstream's borders, and not just your own border, as it's presumably not something you can effectively mitigate internally. Remote IPs would be S/RTBH, and that's not usually seen in transit networks due to the nature of what that would require to affect only the requestor of the blackhole, and not the network as a whole. (VRFs for everyone! Wait... that requires how much RAM? Nevermind...) -- -H U R R I C A N E - E L E C T R I C- Kelly Cochran Sr. Network Engineer 510-580-4100 http://www.he.net/ AS6939
On Tue, Mar 20, 2012 at 11:42:05AM -0700, Kelly Cochran wrote:
Or much simpler solution - remove secondary tables, add blackhole routes to bird config as static routes (in static protocol) and have everything in the master table.
Which is exactly what I wound up doing when testing this sort of thing internally. Soft config reloads won't bounce the session when you change items in the static protocol. Good use for an include, generate
BTW, even 'hard' reconfiguration will not restart the session if there is no change in the BGP session protocol (and its filters). The difference between soft/hard reconfiguration in BIRD is just whether filter changes are considered.
BTW, in the filter bgp_out_he(), i guess you want accept all routes with proto = "blackhole", otherwise only your routes would be exported (and i suppose blackholed IPs are foreign).
Blackholed IPs would actually have to be local. This mechanism is common for dropping a DDoS at your upstream's borders, and not just your own border, as it's presumably not something you can effectively mitigate internally. Remote IPs would be S/RTBH, and that's not usually seen in transit networks due to the nature of what that would require to affect only the requestor of the blackhole, and not the network as a whole. (VRFs for everyone! Wait... that requires how much RAM? Nevermind...)
Yes, i thought more about S/RTBH. BTW, to implement other side of RTBH we would probably need to support explicit change of received BGP route destination to unreachable/blackhole/prohibit type. This currently could by done by some tricks, so explicit filter operator for that would be useful, i suppose. -- Elen sila lumenn' omentielvo Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Thanks for the help and suggestions, we have a working configuration that now imports and uniquely tags/announces blackhole specific kernel route entries. For the archives, here's a basic working "example" config. I opted to get away from statically configuring blackhole routes within BIRD itself, as existing tooling adds/removes kernel route table entries on the linux-based router, and keeping this within kernel routing tables is a simpler approach for us. If using a Linux 2.6 kernel based router, and when tracking blacklisted IPs via a 2nd route table: To blacklist 192.0.2.44/32, run: ~$ sudo ip route add blackhole 192.0.2.44/32 table 10 Using the below configuration, within 10 seconds BIRD will detect a new addition to kernel routing table 10, and: 1) Auto-add a corresponding blackhole entry to your kernel's master routing table...truly blacklisting the IP on your router. 2) Announce 192.0.2.44/32 with a BGP tag of 64665:666 to your upstream provider, enabling you to leverage a customer blackhole community config (e.g. RTBH portion of RFC3882). Relevant portions of bird.conf: "... # Scans and learns of blackhole routes from table 10, every 10 seconds table blackholes; protocol kernel blackhole { table blackholes; kernel table 10; scan time 10; learn; import all; export all; } # Merges kernel route table 10 into the kernel's active/master routing table protocol pipe { table master; peer table blackholes; mode transparent; import filter { print "Importing blackhole list."; accept; }; } filter bgp_out_upstream { # Limit to blackholed routes if (proto = "blackhole" ) then { # Limit to /32 netmask if net.len = 32 then { bgp_community.add((64665,666)); # Replace 64665 with your upstream's ASN printn "Nulling "; print net; accept; } } if net ~ [192.0.2.0/24] then accept; # Replace 192.0.2.0/24 with your local netblock(s) reject; } protocol bgp upstream { ... export filter bgp_out_upstream; ... } ..." Hope this helps others, Gregg Berkholtz Datacenter consulting, hosting & support since 1995 www.tocici.com | 503-488-5461 x17 | AS14613 On Mar 20, 2012, at 3:25 PM, Ondrej Zajicek wrote:
On Tue, Mar 20, 2012 at 11:42:05AM -0700, Kelly Cochran wrote:
Or much simpler solution - remove secondary tables, add blackhole routes to bird config as static routes (in static protocol) and have everything in the master table.
Which is exactly what I wound up doing when testing this sort of thing internally. Soft config reloads won't bounce the session when you change items in the static protocol. Good use for an include, generate
BTW, even 'hard' reconfiguration will not restart the session if there is no change in the BGP session protocol (and its filters). The difference between soft/hard reconfiguration in BIRD is just whether filter changes are considered.
BTW, in the filter bgp_out_he(), i guess you want accept all routes with proto = "blackhole", otherwise only your routes would be exported (and i suppose blackholed IPs are foreign).
Blackholed IPs would actually have to be local. This mechanism is common for dropping a DDoS at your upstream's borders, and not just your own border, as it's presumably not something you can effectively mitigate internally. Remote IPs would be S/RTBH, and that's not usually seen in transit networks due to the nature of what that would require to affect only the requestor of the blackhole, and not the network as a whole. (VRFs for everyone! Wait... that requires how much RAM? Nevermind...)
Yes, i thought more about S/RTBH.
BTW, to implement other side of RTBH we would probably need to support explicit change of received BGP route destination to unreachable/blackhole/prohibit type. This currently could by done by some tricks, so explicit filter operator for that would be useful, i suppose.
-- Elen sila lumenn' omentielvo
Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
-----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAk9zYNkACgkQw+5R43T/iWXgUgCfcH5j2/vF2i40C4RHf5o5gKIV 5C0AoJlSRUJmzzVlAyitQTSWR8NrYmaj =bn0k -----END PGP SIGNATURE-----
participants (4)
-
Gregg Berkholtz -
Henrique de Moraes Holschuh -
Kelly Cochran -
Ondrej Zajicek