$quoted_author = "dspazman@epicup.com" ;

I have tried setting up my routing like: http://lartc.org/howto/lartc.rpdb.multiple-links.html

And my bird setup is like: https://git.nic.cz/redmine/projects/bird/wiki/BGP_filtering (it is down right now, but you can use google cache to still see it)

for the two /29 IP networks I have, one from Cogent, one from HE. I also have a /22 that I am announcing over BGP.

The end result is to have a BGP router using two connections to announce my /22 using my ASN, from the bird router I need to be able to send out requests from my /22 over both interfaces (I have them weighted in the bird config file so one gets more traffic).

I guess what I am asking is if this sounds like it is the correct setup? I don't need someone to babysit me through the whole process, I just want to make sure I'm not going down a rabbit hole and completely missing something.

You're on the right track! :-) My original answer relates only to traffic with a source address in your Cogent /29 as this might be filtered by HE. You might need to do something similar to traffic with a source address in your HE /29 as this might be filtered by Cogent. Traffic with a source address in your /22 shouldn't have that problem.

I'll be done with bugging the whole list now. If I could get a confirmation still that that is the correct route to take, that would be great. Setting up 2 routing tables like the first article, and then setting up bird like the BGP filtering article. I just didn't know if I should be setting up the 2 routing tables, or if the decision on the outbound routing was entirely bird. If anyone wanted to help more and contact me directly I could provide more specific information as well, but that's not required, and I don't want to be any trouble to the group in general.

Bird will accept, filter and sort the BGP routes before exporting them to the kernel. The source based routing mentioned above is just an override at the kernel level so packets from either /29 make it back out the respective link. cheers Marty

On Mon, Dec 05, 2011 at 07:38:30AM +1100, Martin Barry wrote:

...

The end result is to have a BGP router using two connections to announce my /22 using my ASN, from the bird router I need to be able to send out requests from my /22 over both interfaces (I have them weighted in the bird config file so one gets more traffic).

which would balance the outgoing traffic. however the incoming comes over based on bgp preferences(mainly as path) in the internet which results in a lot of asymetric routing. this will result in weird latency and problems with one line affecting all connections.

Can the bgp routing path be set so incoming traffic prefers one connection over the other, as well? I thought setting something like this: protocol ospf { import all; export filter { ospf_metric1 = 1000; if source = RTS_STATIC then accept; else reject; }; area 0 { interface "eth0" { cost 50; type broadcast; hello 5; retransmit 2; wait 10; dead 20; }; interface "eth1" { cost 5; type pointopoint; hello 5; retransmit 2; wait 10; dead 20; }; interface "*" { cost 1000; stub; }; }; } Would cause the path through the connection on eth0 to be 10x more prefered then the path through eth1, so incoming bandwidth should be split in about a 10-1 ratio. So if I have eth0 as a 10 Gig connection, and eth1 as a 1 Gig connection, between using that for incoming traffic, and a multipath rule for outgoing traffic like this: ip route add default scope global nexthop via 1.1.1.153 dev eth0 weight 10 nexthop via 2.2.2.81 dev eth1 weight 1 Would cause the two connections to be pretty uniformly used on incoming and outgoing traffic at a 10/1 ratio, allowing me to pretty fully saturate both lines?

My original answer relates only to traffic with a source address in your Cogent /29 as this might be filtered by HE. You might need to do something similar to traffic with a source address in your HE /29 as this might be filtered by Cogent. Traffic with a source address in your /22 shouldn't have that problem.

the policy routing should probably still be setup to allow testing of a particular line because this way incoming and outgoing traffic do not have to come over the same line. mk

On Sun, Dec 04, 2011 at 06:38:51PM -0500, dspazman@epicup.com wrote:

On Mon, Dec 05, 2011 at 07:38:30AM +1100, Martin Barry wrote:

...

...

The end result is to have a BGP router using two connections to announce my /22 using my ASN, from the bird router I need to be able to send out requests from my /22 over both interfaces (I have them weighted in the bird config file so one gets more traffic).

...

which would balance the outgoing traffic. however the incoming comes over based on bgp preferences(mainly as path) in the internet which results in a lot of asymetric routing. this will result in weird latency and problems with one line affecting all connections.

Can the bgp routing path be set so incoming traffic prefers one connection over the other, as well? I thought setting something like this:

protocol ospf { import all; export filter { ospf_metric1 = 1000; if source = RTS_STATIC then accept; else reject; };

area 0 { interface "eth0" { cost 50; type broadcast; hello 5; retransmit 2; wait 10; dead 20; };

interface "eth1" { cost 5; type pointopoint; hello 5; retransmit 2; wait 10; dead 20; }; interface "*" { cost 1000; stub; }; }; }

Would cause the path through the connection on eth0 to be 10x more prefered then the path through eth1, so incoming bandwidth should be split in about a 10-1 ratio. So if I have eth0 as a 10 Gig connection, and eth1 as a 1 Gig connection, between using that for incoming traffic, and a multipath rule for outgoing traffic like this:

ip route add default scope global nexthop via 1.1.1.153 dev eth0 weight 10 nexthop via 2.2.2.81 dev eth1 weight 1

Would cause the two connections to be pretty uniformly used on incoming and outgoing traffic at a 10/1 ratio, allowing me to pretty fully saturate both lines?

eh. not really. the thing you show here is ospf and the cost is simply the weight of the link that is taken into account when it computes the shortest path. there is no load balancing involved. bgp can loadbalance only by ip networks, and it is a manual process for long term setup, not per packet magic solution. hate to say it but setting it correctly requires a lot of reading and some experience. basically the best thing to do is just to set two bgp peers and then mess with the as path to balance the incoming traffic over those two connections. mk

Let's say you have two routing tables (T1, T2), one for each interface (eth0, eth1), so you can route traffic out each one separately. https://git.nic.cz/redmine/projects/bird/wiki/BGP_example_2 I'm assuming there would have to be some changes to that BGP script to support both routing tables. I'm not sure how to support multiple routing tables with bird. Would someone be able to show that? Seems like it would be a useful example in general, and be a pretty simple modification to that example for someone who knows what they are doing. I didn't see anything on any of the examples on git.nic.cz that showed using multiple tables, as a configuration like http://lartc.org/howto/lartc.rpdb.multiple-links.html shows you'd be using for multiple outgoing connections, which you would typically have in a BGP situation. Though maybe https://git.nic.cz/redmine/projects/bird/wiki/Policy_routing explains it. I plan on keep looking into that example as well. Thanks! -----Original Message----- From: "Martin Barry" <marty@supine.com> Sent: Monday, December 5, 2011 11:53am To: bird-users@atrey.karlin.mff.cuni.cz Subject: Re: Routing issues $quoted_author = "dspazman@epicup.com" ;

Would cause the path through the connection on eth0 to be 10x more prefered then the path through eth1, so incoming bandwidth should be split in about a 10-1 ratio. So if I have eth0 as a 10 Gig connection, and eth1 as a 1 Gig connection, between using that for incoming traffic, and a multipath rule for outgoing traffic like this:

ip route add default scope global nexthop via 1.1.1.153 dev eth0 weight 10 nexthop via 2.2.2.81 dev eth1 weight 1

Would cause the two connections to be pretty uniformly used on incoming and outgoing traffic at a 10/1 ratio, allowing me to pretty fully saturate both lines?

Like the other Martin said, this doesn't work quite how you are thinking. BGP unfortunately has only a limited range of controls one can manipulate and often you have no control over what remote ASes are doing and hence can't control the inbound traffic as much as you would like (if at all). You'll need to read up more on the BGP selection criteria and how these work but the ones to focus on for... Inbound - prefix specificity - AS path and prepending thereof - Communities. Signals to your upstreams for things like 'no export' or prepending to specific ASes - MED. This is works best with multiple links to the same provider. Outbound - filtering - local preference cheers Marty

On Tue, Dec 06, 2011 at 01:58:30PM +0100, Ondrej Zajicek wrote:

- Even if traffic from eth0 and eth1 is sufficiently splited, that will hel just split outgoing traffic on uplinks, splitting incoming is harder.

also the asymetric routing is a bother because problems on one line will affect many more connections and in case one line goes down the whole asymetric traffic dies until the failure gets detected and the internet routes around int. mk

On Tue, Dec 06, 2011 at 10:06:30AM -0500, dspazman@epicup.com wrote:

I tried modifying the example. Here is the situation.

No clients, just have my own block of IPs I want to announce (A.A.A.A/22). I also have two /29s, one from each provider (Y.Y.Y.A/29, Y.Y.Y.B/29) that contains the external IP address for each of my router's two ethernet interfaces, as assigned by each ISP. The two providers I'm trying to just run so one is preferred, but both can be used. The asymetric isn't a huge deal, so if it's simpler to take it out, that's ok also.

In that case i would suggest to forget multiple routing table and keep it simple, like in: https://git.nic.cz/redmine/projects/bird/wiki/BGP_example_2 (or the later filtering example) (and just insert several your ASNs to AS path in export filter to less preferred uplink)

Also, besides the bird setup, I ran the following rules: ip rule add iif eth2 table 1 ip rule add iif eth5 table 2

(This is on Ubuntu, btw, and my two ISP interfaces are eth2 (my less prefered one) and eth5 (my high speed 10 Gig fiber one).

I thought you want to using multiple routing tables to route your traffic from your two internal links. There is no reason to route differently traffic received from each uplink.

The problems / questions I ran into was this.

1) Does the table 1 / table 2 need to be declared in the underlying system first? I didn't think so, like in /etc/iproute2/rt_tables? Because the IP rules seemed to work for pinging out on the interface.

No

3) Is the ospf part needed or helpful?

If you use OSPF in your network, then yes, otherwise no. But your OSPF config seems to be completely pointless. I would suggest to read some general texts about OSPF, BGP and routing.

Is it a good practice to have the BGP router also run OSPF, in general?

I think it is, unless in trivial cases. But in most cases you do not have OSPF session with your provider. -- Elen sila lumenn' omentielvo Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."