Hello,

We use BIRD as route server for a long time without any problems. We have both fully symmetric RS for redundancy.

Yesterday we decided to establish "internal BGP session" between both of them. The reason to do this is to achieve prefix symmetric on both RS. Some of our members hold BGP session only with one of our route servers.

But after establishing this BGP session we began to see the following message into our logs on both RS for total random prefixes on absolutely random intervals.

Here is part of my bird.log:
07-03-2014 14:19:41 R0_29: Invalid NEXT_HOP attribute in route 178.254.232.0/21
07-03-2014 14:19:41 R0_30: Invalid NEXT_HOP attribute in route 178.254.232.0/21
07-03-2014 14:18:52 R0_52: Invalid NEXT_HOP attribute in route 79.140.156.0/22
07-03-2014 14:18:52 R0_52: Invalid NEXT_HOP attribute in route 79.140.148.0/22
07-03-2014 14:18:52 R0_52: Invalid NEXT_HOP attribute in route 79.140.144.0/20
07-03-2014 14:18:52 R0_52: Invalid NEXT_HOP attribute in route 79.140.152.0/22
07-03-2014 14:18:52 R0_52: Invalid NEXT_HOP attribute in route 79.140.144.0/22
07-03-2014 14:18:52 R0_52: Invalid NEXT_HOP attribute in route 79.140.156.0/22
07-03-2014 14:18:52 R0_52: Invalid NEXT_HOP attribute in route 79.140.144.0/20
07-03-2014 14:18:52 R0_52: Invalid NEXT_HOP attribute in route 79.140.148.0/22
07-03-2014 14:18:52 R0_52: Invalid NEXT_HOP attribute in route 79.140.152.0/22
07-03-2014 14:18:52 R0_52: Invalid NEXT_HOP attribute in route 79.140.144.0/22
07-03-2014 14:16:39 R0_29: Invalid NEXT_HOP attribute in route 178.254.232.0/21
07-03-2014 14:16:39 R0_30: Invalid NEXT_HOP attribute in route 178.254.232.0/21
07-03-2014 14:01:39 R0_52: Invalid NEXT_HOP attribute in route 95.168.66.0/23
07-03-2014 14:01:39 R0_52: Invalid NEXT_HOP attribute in route 95.168.68.0/23
07-03-2014 13:52:02 R0_14: Invalid NEXT_HOP attribute in route 151.252.198.0/24
07-03-2014 13:51:57 R0_14: Invalid NEXT_HOP attribute in route 151.252.198.0/24
07-03-2014 13:29:36 R0_87: Invalid NEXT_HOP attribute in route 85.11.144.0/20
07-03-2014 13:29:28 R0_87: Invalid NEXT_HOP attribute in route 85.11.144.0/20
07-03-2014 13:29:15 R0_87: Invalid NEXT_HOP attribute in route 85.11.144.0/20
07-03-2014 13:29:06 R0_87: Invalid NEXT_HOP attribute in route 85.11.144.0/20
07-03-2014 13:28:14 R0_87: Invalid NEXT_HOP attribute in route 85.187.120.0/22
07-03-2014 13:28:14 R0_87: Invalid NEXT_HOP attribute in route 85.187.120.0/22
07-03-2014 13:19:36 R0_87: Invalid NEXT_HOP attribute in route 85.187.120.0/22
07-03-2014 13:19:31 R0_87: Invalid NEXT_HOP attribute in route 85.187.120.0/22
07-03-2014 13:19:31 R0_87: Invalid NEXT_HOP attribute in route 85.11.144.0/20
07-03-2014 13:19:31 R0_87: Invalid NEXT_HOP attribute in route 85.187.120.0/22
07-03-2014 13:19:22 R0_87: Invalid NEXT_HOP attribute in route 85.11.144.0/20
07-03-2014 13:13:43 R0_71: Invalid NEXT_HOP attribute in route 46.229.194.0/24
07-03-2014 13:13:43 R0_71: Invalid NEXT_HOP attribute in route 46.229.192.0/23
07-03-2014 13:13:42 R0_29: Invalid NEXT_HOP attribute in route 46.229.194.0/24
07-03-2014 13:13:42 R0_29: Invalid NEXT_HOP attribute in route 46.229.192.0/23
07-03-2014 13:06:22 R0_60: Invalid NEXT_HOP attribute in route 193.104.73.0/24
07-03-2014 13:03:54 R0_52: Invalid NEXT_HOP attribute in route 178.79.61.0/24
07-03-2014 12:54:56 R0_60: Invalid NEXT_HOP attribute in route 89.35.114.0/24
07-03-2014 12:54:56 R0_60: Invalid NEXT_HOP attribute in route 86.105.150.0/24
07-03-2014 12:54:56 R0_60: Invalid NEXT_HOP attribute in route 46.102.181.0/24
07-03-2014 12:54:56 R0_60: Invalid NEXT_HOP attribute in route 46.102.176.0/24
07-03-2014 12:54:56 R0_60: Invalid NEXT_HOP attribute in route 37.156.69.0/24
07-03-2014 12:54:25 R0_60: Invalid NEXT_HOP attribute in route 188.240.70.0/24
07-03-2014 12:54:25 R0_60: Invalid NEXT_HOP attribute in route 188.212.157.0/24
07-03-2014 12:34:43 R0_52: Invalid NEXT_HOP attribute in route 91.148.106.0/24
07-03-2014 12:27:48 R0_52: Invalid NEXT_HOP attribute in route 91.148.118.0/24
07-03-2014 12:16:33 R0_29: Invalid NEXT_HOP attribute in route 178.254.232.0/21
07-03-2014 12:16:33 R0_30: Invalid NEXT_HOP attribute in route 178.254.232.0/21
07-03-2014 12:09:39 R0_87: Invalid NEXT_HOP attribute in route 85.11.144.0/20
07-03-2014 12:09:10 R0_87: Invalid NEXT_HOP attribute in route 85.11.144.0/20
07-03-2014 12:08:57 R0_87: Invalid NEXT_HOP attribute in route 85.187.120.0/22
07-03-2014 12:08:50 R0_87: Invalid NEXT_HOP attribute in route 85.187.120.0/22
07-03-2014 12:08:49 R0_87: Invalid NEXT_HOP attribute in route 85.187.120.0/22
07-03-2014 12:02:41 R0_52: Invalid NEXT_HOP attribute in route 212.200.0.0/21
07-03-2014 11:47:01 R0_29: Invalid NEXT_HOP attribute in route 178.254.232.0/21
07-03-2014 11:47:01 R0_30: Invalid NEXT_HOP attribute in route 178.254.232.0/21
07-03-2014 10:55:50 R0_60: Invalid NEXT_HOP attribute in route 91.206.209.0/24

I had checked many times and make many tests if the next-hop is correct and everything looks
fine but this messages comes permanently :)

We managed to troubleshoot the following behavior and relationship and see the following effect:

Our scenario:

0.100                 0.200
RS1 <- iBGP -> RS2
|                       |
|                       |
TEST.AS -----------
0.252

TEST.AS announce: 31.13.244.0/24

RS2 learn this prefix as:

31.13.244.0/24     via 10.1.0.252 on eth0 [R0_252 14:44] * (100) [AS60230i]
                             via 10.1.0.252 on eth0 [R0_100 15:12 from 10.1.0.100] (100) [AS60230i]

As soon as we stop to announce 31.13.244.0/24 from TEST.AS to RS1, in the logs of RS1 begin to come the following message:

07-03-2014 15:10:09 R0_252: Invalid NEXT_HOP attribute in route 31.13.244.0/24

But when we try to shutdown (administratively ) the session between TEST.AS and RS1 this behavior does not occur and everything seems normal.

I hope this information will be useful for resolving this strange issue.

Here is our configuration set on both RS. The real AS number & IPs has been changed for security reason :)

# RS2:
table T65535

protocol pipe P65535 from iBGP_PIPES {
description "RS1";
peer table T65535;
export where RS_PIPE_OUT();
}

protocol bgp R0_100 from iBGP {
description "0.100_iBGP_RS1";
source address 10.1.0.200;
neighbor 10.1.0.100 as 65535;
import all;
export all;
passive off;
table T65535;
route limit 10000;
gateway direct;
}

# RS1:
table T65535

protocol pipe P65535 from iBGP_PIPES {
description "RS2";
peer table T65535;
export where RS_PIPE_OUT();
}

protocol bgp R0_200 from iBGP {
description "0.200_iBGP_RS1";
source address 10.1.0.100;
neighbor 10.1.0.200 as 65535;
import all;
export all;
passive off;
table T65535;
route limit 10000;
gateway direct;
}

# show bgp summary shows the following:
# RS2:
---------------------------------------------------------------- adv / rcv / limit ---
0.100_iBGP_RS1    10.1.0.100    65535    Mar06    Established    4527/3235/10000
--------------------------------------------------------------------------------------

# RS1:
---------------------------------------------------------------- adv / rcv / limit ---
0.200_iBGP_RS2    10.1.0.200    65535    Mar06    Established    3235/4527/10000
--------------------------------------------------------------------------------------

Any ideas or thoughts are highly appreciated!

Thanks in advance!

--

---
Find out about our new Cloud service - Cloudware.bg

Access anywhere. Manage it yourself. Pay as you go.

Javor Kliachev
IP Engineer

Neterra Ltd.
Telephone: +359 2 975 16 16
Fax: +359 2 975 34 36