bird route selection problem

[Daniel Rimal]

On Fri, Jan 28, 2011 at 10:19 AM, Daniel Rimal <danrimal at gmail.com> wrote:
> On Fri, Jan 28, 2011 at 10:07 AM, Ondrej Zajicek <santiago at crfreenet.org> wrote:
>> On Thu, Jan 27, 2011 at 05:50:42PM +0100, Daniel Rimal wrote:
>>> Hello,
>>>
>>> i have little weird problem with route selection in bird. A have four
>>> routers in
>>> my AS, routers called R1 and R2 is running guagga, routers called R3 and R4
>>> is running bird 1.2.5. I have configured full mesh between all of them.
>>>
>>> IP 81.xxx.yyy.189 and 81.xxx.yyy.190 is ebgp peers. Both,
>>> R3 and R4 running bird have established peer with both ebgp routers.
>>>
>>> IP 109.xxx.yyy.11 and 109.xxx.yyy.12 is IP on R3 respectively R4
>>> (ibgp interconnection)
>>>
>>> After receive this withdraw from ebgp peer 81.xxx.yyy.189 by R3 for example:
>>> 14:19:23.930683 81.xxx.yyy.189  81.x.yyy.185    BGP     UPDATE Message
>>> Withdrawn prefix: 187.120.96.0 (187.120.96.0)
>>>
>>> bird do this: - switch route between ebgp and ibgp peer very fast -
>>> 288 times (in one second) in this case:
>>> 14:19:25 <TRACE> kernel1 < replaced 187.120.96.0/23 via 81.xxx.yyy.190
>>> on bond0.402
>>> 14:19:25 <TRACE> kernel1 < replaced 187.120.96.0/23 via 109.xxx.yyy.12
>>> on bond0.403
>>> 14:19:25 <TRACE> kernel1 < replaced 187.120.96.0/23 via 81.xxx.yyy.190
>>> on bond0.402
>>
>> This is really strange - if R3 has a route from 81.xxx.yyy.190, that
>> route should be preferred. One possible explanation is that ebgp peer
>> 81.xxx.yyy.190 also sent that sequence of updates and withdraws to R3
>> (causing him to switch to ibgp-route when ebgp-route was withdrawn and
>> swich back when it reappears). Are you sure that it is not the case?
>>
>> The switching sequence was on both R3 and R4?
>>
>> Could you also send me an output of 'show route 187.120.96.0/23 all'
>> on R3 and R4?
>
> There is route detail:
>
> 187.120.96.0/23    via 81.xxx.yyy.190 on bond0.402 [casa190 04:45] *
> (100) [AS28202i]
>        Type: BGP unicast univ
>        BGP.origin: IGP
>        BGP.as_path: 15685 29208 3549 4230 28202
>        BGP.next_hop: 81.xxx.yyy.190
>        BGP.local_pref: 700
>        BGP.community:
>                   via 109.xxx.yyy.10 on bond0.403 [r2 09:08] (100) [AS28202i]
>        Type: BGP unicast univ
>        BGP.origin: IGP
>        BGP.as_path: 29208 3549 4230 28202
>        BGP.next_hop: 109.xxx.yyy.10
>        BGP.local_pref: 500
>        BGP.community: (2,100) (2,3549) (3549,4359) (3549,30840)
> (4230,21) (4230,30) (4230,31) (4230,131) (29208,3549)
>                   via 81.xxx.yyy.189 on bond0.402 [casa189 04:45]
> (100) [AS28202i]
>        Type: BGP unicast univ
>        BGP.origin: IGP
>        BGP.as_path: 15685 29208 3549 4230 28202
>        BGP.next_hop: 81.xxx.yyy.189
>        BGP.local_pref: 700
>        BGP.community:
>
> But route list is probably affected by some experiments which i did on
> routers last night. I'm sure that ebgp routers send only one
> update/withdraw, i checked by tcpdump too. Switching sequence was on
> R3 and R4 too.
>
> But i found the problem is probably caused by local_pref settings. I
> did some experiments and found bird weird behaviour when i have
> identical bgp_local_pref value in import filter on both ibgp peers R3
> & R4, like that:
>
> filter bgp_in_sitel
>    {
>            bgp_local_pref=500;
>            accept;
>    }
>
> protocol bgp r4 {
> #       disabled;
>        description "propoj s R4";
>        debug {events, states};
>        local as 35236;
>        neighbor 109.xxx.yyy.12 as 35236;
>        hold time 90;
>        startup hold time 90;
>        connect retry time 30;
>        keepalive time 30;
>        start delay time 5;
>        error wait time 60, 300;
>        error forget time 300;
> #       disable after error;
>        next hop self;
>        path metric 1;
>        default bgp_med 0;
>        default bgp_local_pref 500;
>        import filter bgp_in_sitel;
>        export all;
>
> When i completely disabled import filter on both ibgp peers (set
> import all) and "default_bgp_local_pref" was identical on R3 & R4,
> bird behaviour was normal.
>
> When i enabled import filter on both sides, set different
> "bgp_local_pref" but "default bgp_local_pref" was identical, bird
> behaviour was still bad.
>
> When i enabled import filter on both sides, set different
> "bgp_local_pref" and set different "default bgp_local_pref" , bird
> behaviour was normal.
>
> Dan
>

I just checked bird6 and found same problem:

28-01-2011 12:16:10 <TRACE> kernel1 < replaced 2406:5400::/32 via
2a00:1238:xx:xxx::4 on bond0.403
28-01-2011 12:16:10 <TRACE> kernel1 < replaced 2406:5400::/32 via
2001:1528:yy:yyy::1 on bond0.402
28-01-2011 12:16:10 <TRACE> kernel1 < replaced 2406:5400::/32 via
2a00:1238:xx:xxx::4 on bond0.403
28-01-2011 12:16:10 <TRACE> kernel1 < replaced 2406:5400::/32 via
2001:1528:yy:yyy::1 on bond0.402
28-01-2011 12:16:10 <TRACE> kernel1 < replaced 2406:5400::/32 via
2a00:1238:xx:xxx::4 on bond0.403

Set different local_pref between ibgp peers solved problem, but it
looks like bug, right?

Thank you!
Dan