Toke Høiland-Jørgensen <toke@toke.dk> writes:
Nico Schottelius <nico.schottelius@ungleich.ch> writes:
Hello,
we are using direct + babel as an IGP and today something "funky" happened: the routes of one router disappeared and on other routers the babel entries contained empty router IDs in babel:
Hmm, the obvious reason for this would be if Babel no longer considers that router reachable. The output of 'show babel neighbors' and 'show babel routes' from both sides (while this is going on) might shed some light on this.
Luckily (?) I kept one broken system running, looking at the entries for 2a0a:e5c0:0:2::/64 and 2a0a:e5c0:0:5::/64 on a router1.place6, we see an empty router id. bird> show babel entries babel1: Prefix Router ID Metric Seqno Routes Sources 185.203.114.0/23 00:00:00:00:93:4e:c3:fb 0 2204 4 0 195.141.230.102/31 <none> - - 1 0 185.203.112.0/24 <none> - - 2 0 147.78.195.224/27 00:00:00:00:93:4e:c3:fb 0 2204 7 0 193.192.225.72/31 <none> - - 1 0 94.78.2.168/29 00:00:00:00:93:4e:c3:fb 0 2204 4 0 147.78.195.0/29 00:00:00:00:93:4e:c3:e3 96 1 1 0 185.155.31.0/24 00:00:00:00:93:4e:c3:fb 0 2204 4 0 45.134.132.0/29 00:00:00:00:93:4e:c3:fb 0 2204 4 0 185.155.31.0/26 00:00:00:00:93:4e:c3:fb 0 2204 4 0 2a0a:e5c0:1:f::/64 00:00:00:00:93:4e:c3:e3 96 1 1 0 2a0a:e5c0:1:e::/64 00:00:00:00:93:4e:c3:e3 96 1 1 0 2a0a:e5c0:1:d::/64 00:00:00:00:93:4e:c3:fb 0 2204 5 0 2a0a:e5c0:0:9::/64 00:00:00:00:00:00:00:2f 96 1 1 0 2a0a:e5c0:1:8::/64 00:00:00:00:93:4e:c3:fb 0 2204 10 0 2a0a:e5c0:0:6::/64 00:00:00:00:00:00:00:2e 96 2 1 0 2a0a:e5c0:0:5::/64 <none> - - 2 0 2a0a:e5c0:0:4::/64 00:00:00:00:00:00:00:2f 96 1 1 0 2a0a:e5c0:1:4::/64 00:00:00:00:93:4e:c3:fb 0 2204 5 0 2a0a:e5c0:0:3::/64 00:00:00:00:00:00:00:2e 96 2 1 0 2a0a:e5c0:0:2::/64 <none> - - 2 0 2a0a:e5c0:2:2::/64 00:00:00:00:93:4e:c3:fb 0 2204 5 0 2a0a:e5c0:10:1::/64 00:00:00:00:93:4e:c3:e3 96 1 1 0 2a0a:e5c0::/64 00:00:00:00:00:00:00:2f 96 1 2 0 2a0a:e5c0:1b::/64 00:00:00:00:93:4e:c3:fb 0 2204 5 0 2a0a:e5c0:10::/64 00:00:00:00:93:4e:c3:e3 96 1 1 0 2a01:330:1000:1003::/64 00:00:00:00:93:4e:c3:fb 0 2204 5 0 2a0a:e5c0:2:12::/64 00:00:00:00:93:4e:c3:fb 0 2204 5 0 2001:1700:3500::10/124 <none> - - 1 0 2a0a:e5c0:10:bee::/64 00:00:00:00:93:4e:c3:e3 96 1 1 0 2001:1700:3500::/124 <none> - - 1 0 2a0a:e5c0:1::/124 00:00:00:00:93:4e:c3:fb 0 2204 5 0 bird> Output from the same router "router1.place6": bird> show babel neighbors babel1: IP address Interface Metric Routes Hellos Expires fe80::21b:21ff:febc:bf36 bond0.8 96 8 12 0.000 fe80::21b:21ff:febc:bfe0 bond0.8 65535 14 9 0.000 fe80::20d:b9ff:fe49:a705 bond0.8 96 4 12 0.000 fe80::a236:9fff:fe08:a780 bond0.8 65535 7 10 0.000 fe80::20d:b9ff:fe57:2f91 bond0.8 96 4 12 0.000 fe80::21b:21ff:febc:bfd8 bond0.8 65535 7 10 0.000 fe80::21b:21ff:febc:bfe0 bond0.35 65535 14 9 0.000 fe80::21b:21ff:febc:bfe0 bond0.100 65535 14 9 0.000 fe80::21b:21ff:febc:bfe0 bond0.12 65535 14 9 5.999 fe80::21b:21ff:febc:bfe0 bond0.38 65535 8 10 5.999 bird> This time I actually see the routes, even though the rotuer id is none: 2a0a:e5c0:0:5::/64 fe80::a236:9fff:fe08:a780 bond0.8 65535 + 1401 41.400 2a0a:e5c0:0:5::/64 fe80::21b:21ff:febc:bfd8 bond0.8 65535 + 1 36.722 Full output on [0] So it seems that a missing router ID does not always result into the routes not being sent. However on router2.place6 I should also receive the route 147.78.195.0/29 from router1.place10 via bond0.8. Output from router2.place6, which is fe80::21b:21ff:febc:bfe0: bird> show babel neighbors babel1: IP address Interface Metric Routes Hellos Expires fe80::21b:21ff:febc:bf36 bond0.8 65535 8 12 0.000 fe80::20d:b9ff:fe49:a705 bond0.8 65535 4 12 0.000 fe80::20d:b9ff:fe57:2f91 bond0.8 96 4 12 0.000 fe80::a236:9fff:fe08:a780 bond0.8 65535 7 9 0.000 fe80::a236:9fff:fe08:a150 bond0.8 65535 14 8 0.000 fe80::21b:21ff:febc:bfd8 bond0.8 96 7 12 0.000 fe80::a236:9fff:fe08:a150 bond0.35 65535 14 8 0.000 fe80::a236:9fff:fe08:a150 bond0.100 65535 14 7 0.000 fe80::a236:9fff:fe08:a150 bond0.38 65535 8 7 5.999 fe80::a236:9fff:fe08:a150 bond0.12 65535 14 10 5.999 bird> fe80::21b:21ff:febc:bf36 is router1.place10, checking the neighbors from router1.place10: bird> show babel neighbors babel1: IP address Interface Metric Routes Hellos Expires fe80::20d:b9ff:fe49:a705 bond0.8 96 4 16 4.188 fe80::21b:21ff:febc:bfe0 bond0.8 65535 14 9 1.955 fe80::20d:b9ff:fe57:2f91 bond0.8 96 4 15 2.551 fe80::a236:9fff:fe08:a780 bond0.8 65535 7 10 0.473 fe80::21b:21ff:febc:bfd8 bond0.8 96 7 15 5.519 fe80::a236:9fff:fe08:a150 bond0.8 96 14 13 3.445 bird> So both routers see each other via babel. However router2.place6 does *not* see the route 147.78.195.0/29, see [1]. The babel entries on router2.place6 again show the route, but an empty router id: bird> show babel entries babel1: Prefix Router ID Metric Seqno Routes Sources 185.203.114.0/23 00:00:00:00:93:4e:c3:fc 0 1 4 0 195.141.230.102/31 00:00:00:00:93:4e:c3:fa 96 1 1 0 185.203.112.0/24 00:00:00:00:93:4e:c3:fa 96 1 2 0 147.78.195.224/27 00:00:00:00:93:4e:c3:fc 0 1 7 0 193.192.225.72/31 <none> - - 1 0 94.78.2.168/29 00:00:00:00:93:4e:c3:fc 0 1 4 0 147.78.195.0/29 <none> - - 1 0 185.155.31.0/24 00:00:00:00:93:4e:c3:fc 0 1 4 0 45.134.132.0/29 00:00:00:00:93:4e:c3:fc 0 1 4 0 185.155.31.0/26 00:00:00:00:93:4e:c3:fc 0 1 4 0 2a0a:e5c0:1:f::/64 <none> - - 1 0 2a0a:e5c0:1:e::/64 <none> - - 1 0 2a0a:e5c0:1:d::/64 00:00:00:00:93:4e:c3:fc 0 1 5 0 2a0a:e5c0:0:9::/64 <none> - - 1 0 2a0a:e5c0:1:8::/64 00:00:00:00:93:4e:c3:fc 0 1 10 0 2a0a:e5c0:0:6::/64 00:00:00:00:00:00:00:2e 96 2 1 0 2a0a:e5c0:0:5::/64 00:00:00:00:93:4e:c3:fa 96 1 2 0 2a0a:e5c0:0:4::/64 <none> - - 1 0 2a0a:e5c0:1:4::/64 00:00:00:00:93:4e:c3:fc 0 1 5 0 2a0a:e5c0:0:3::/64 00:00:00:00:00:00:00:2e 96 2 1 0 2a0a:e5c0:0:2::/64 00:00:00:00:93:4e:c3:fa 96 1 2 0 2a0a:e5c0:2:2::/64 00:00:00:00:93:4e:c3:fc 0 1 5 0 2a0a:e5c0:10:1::/64 <none> - - 1 0 2a0a:e5c0::/64 00:00:00:00:00:00:00:2e 96 2 2 0 2a0a:e5c0:1b::/64 00:00:00:00:93:4e:c3:fc 0 1 5 0 2a0a:e5c0:10::/64 <none> - - 1 0 2a01:330:1000:1003::/64 00:00:00:00:93:4e:c3:fc 0 1 5 0 2a0a:e5c0:2:12::/64 00:00:00:00:93:4e:c3:fc 0 1 5 0 2001:1700:3500::10/124 00:00:00:00:93:4e:c3:fa 96 1 1 0 2a0a:e5c0:10:bee::/64 <none> - - 1 0 2001:1700:3500::/124 <none> - - 1 0 2a0a:e5c0:1::/124 00:00:00:00:93:4e:c3:fc 0 1 5 0 bird> Both routers are connected via an L2 link (10km) via two Arista 7050 switches over a vlan tagged bond. Communication between the routers works fine, besides the babel protocol issue.
After restarting the router, the entries are correct now (router id 00:00:00:00:93:4e:c3:e3):
How long did you wait before and after restarting?
Maybe about 1h - it's hard to say, as we noticed it by the loss of routes that used to be there. Just this moment I restarted bird on router1.place10 and the IPv4 route 147.78.195.0/29 instantly pops up again on router2.place6: bird> show babel routes ... 147.78.195.0/29 147.78.195.227 bond0.8 65535 + 1 51.477 ... This time however it took only seconds or a minute until that route was lost again on router2.place6.
The "funny" part is that bird 2.0.8 on that router was up and running, it did see babel neighbors, it did BGP, however only after restarting bird, the routes were correctly received be neighboring routers.
How did you determine that it "sees babel neighbors"?
Using show babel neighbors
As you can see in above table there is at least another router that is affected. I tried `disable babel1` and `enable babel1` on it, however that did not fix the problem.
We did some amount of `configure` commands on the bird process, but my understanding was that it should be very similar to restarting it, without the loss of sessions. The babel protocol *did* previously export the routes correctly to other routers, but I am 99% sure that it did stop doing it, until we hard restarted bird.
What did you do to determine that? What's the output of the 'babel show *' commands on the affected router itself?
So right now after restarting bird on router1.place10, the routes appeared and `mtr` to a destination address instantly worked. For a few seconds, until router2.place6 went back into not having the route.
The babel configuration we use on most of our routers looks like this:
protocol direct { ipv4; ipv6; interface "bond0.*"; }
protocol babel { interface "bond0.*" { type wired; };
ipv4 { export where (source = RTS_DEVICE); }; ipv6 { export where (source = RTS_DEVICE); }; }
So we are not re-babling like babeld, but only inject local device routes, as we have 2 redundant routers per network.
Is there a conceptual problem or is this a bug in bird 2.0.8 and in either way, is there anything we can do to fix it besides restarting bird on router propagation error?
If you're not re-exporting the babel routes you lose the resiliency of going through additional hops, of course (i.e., if you re-export, the routers can re-route through one another as long as one of them has a valid route to the destination), but I assume that's what you're trying to avoid(?), so in that case there should not be any conceptual issue with what you're doing AFAICT...
So the reason for not re-exporting is that our setup looks as follows: [ net x ]---------- router1.place5 -------x----- router1.place6 ----- [ net y ] |--------------- router2.place5 ----x/--\--- router2.place6 -----/ So basically each network in every place has 2 routers, both of which peer (bgp) / communicate (babel) with the routers of the other networks. Thus there is no need for rebabling something the other side already knows. I hope my description above is enough for pin pointing to the source of the problem. Best regards, Nico [0] from router1.place6 bird> show babel routes babel1: Prefix Nexthop Interface Metric F Seqno Expires 185.203.114.0/23 185.203.114.8 bond0.12 65535 + 1 44.244 185.203.114.0/23 147.78.195.252 bond0.8 65535 + 1 44.244 185.203.114.0/23 94.78.2.171 bond0.100 65535 + 1 44.244 185.203.114.0/23 45.134.132.6 bond0.35 65535 + 1 44.244 195.141.230.102/31 147.78.195.250 bond0.8 65535 + 1 36.722 185.203.112.0/24 147.78.195.249 bond0.8 65535 + 1401 41.400 185.203.112.0/24 147.78.195.250 bond0.8 65535 + 1 36.722 147.78.195.224/27 147.78.195.227 bond0.8 96 * 1 41.012 147.78.195.224/27 147.78.195.249 bond0.8 65535 + 1401 41.400 147.78.195.224/27 185.203.114.8 bond0.12 65535 + 1 44.244 147.78.195.224/27 147.78.195.252 bond0.8 65535 + 1 44.244 147.78.195.224/27 94.78.2.171 bond0.100 65535 + 1 44.244 147.78.195.224/27 45.134.132.6 bond0.35 65535 + 1 44.244 147.78.195.224/27 147.78.195.250 bond0.8 65535 + 1 36.722 193.192.225.72/31 147.78.195.249 bond0.8 65535 + 1401 41.400 94.78.2.168/29 185.203.114.8 bond0.12 65535 + 1 44.244 94.78.2.168/29 147.78.195.252 bond0.8 65535 + 1 44.244 94.78.2.168/29 94.78.2.171 bond0.100 65535 + 1 44.244 94.78.2.168/29 45.134.132.6 bond0.35 65535 + 1 44.244 147.78.195.0/29 147.78.195.227 bond0.8 96 * 1 41.012 185.155.31.0/24 185.203.114.8 bond0.12 65535 + 1 44.244 185.155.31.0/24 147.78.195.252 bond0.8 65535 + 1 44.244 185.155.31.0/24 94.78.2.171 bond0.100 65535 + 1 44.244 185.155.31.0/24 45.134.132.6 bond0.35 65535 + 1 44.244 45.134.132.0/29 185.203.114.8 bond0.12 65535 + 1 44.244 45.134.132.0/29 147.78.195.252 bond0.8 65535 + 1 44.244 45.134.132.0/29 94.78.2.171 bond0.100 65535 + 1 44.244 45.134.132.0/29 45.134.132.6 bond0.35 65535 + 1 44.244 185.155.31.0/26 185.203.114.8 bond0.12 65535 + 1 44.244 185.155.31.0/26 147.78.195.252 bond0.8 65535 + 1 44.244 185.155.31.0/26 94.78.2.171 bond0.100 65535 + 1 44.244 185.155.31.0/26 45.134.132.6 bond0.35 65535 + 1 44.244 2a0a:e5c0:1:f::/64 fe80::21b:21ff:febc:bf36 bond0.8 96 * 1 41.012 2a0a:e5c0:1:e::/64 fe80::21b:21ff:febc:bf36 bond0.8 96 * 1 41.012 2a0a:e5c0:1:d::/64 fe80::21b:21ff:febc:bfe0 bond0.12 65535 + 1 44.244 2a0a:e5c0:1:d::/64 fe80::21b:21ff:febc:bfe0 bond0.8 65535 + 1 44.244 2a0a:e5c0:1:d::/64 fe80::21b:21ff:febc:bfe0 bond0.38 65535 + 1 44.244 2a0a:e5c0:1:d::/64 fe80::21b:21ff:febc:bfe0 bond0.100 65535 + 1 44.244 2a0a:e5c0:1:d::/64 fe80::21b:21ff:febc:bfe0 bond0.35 65535 + 1 44.244 2a0a:e5c0:0:9::/64 fe80::20d:b9ff:fe49:a705 bond0.8 96 * 1 36.419 2a0a:e5c0:1:8::/64 fe80::21b:21ff:febc:bf36 bond0.8 96 + 1 41.012 2a0a:e5c0:1:8::/64 fe80::a236:9fff:fe08:a780 bond0.8 65535 + 1401 41.400 2a0a:e5c0:1:8::/64 fe80::20d:b9ff:fe57:2f91 bond0.8 96 + 2 36.001 2a0a:e5c0:1:8::/64 fe80::20d:b9ff:fe49:a705 bond0.8 96 * 1 36.419 2a0a:e5c0:1:8::/64 fe80::21b:21ff:febc:bfe0 bond0.12 65535 + 1 44.244 2a0a:e5c0:1:8::/64 fe80::21b:21ff:febc:bfe0 bond0.8 65535 + 1 44.244 2a0a:e5c0:1:8::/64 fe80::21b:21ff:febc:bfe0 bond0.38 65535 + 1 44.244 2a0a:e5c0:1:8::/64 fe80::21b:21ff:febc:bfe0 bond0.100 65535 + 1 44.244 2a0a:e5c0:1:8::/64 fe80::21b:21ff:febc:bfe0 bond0.35 65535 + 1 44.244 2a0a:e5c0:1:8::/64 fe80::21b:21ff:febc:bfd8 bond0.8 65535 + 1 36.722 2a0a:e5c0:0:6::/64 fe80::20d:b9ff:fe57:2f91 bond0.8 96 * 2 36.001 2a0a:e5c0:0:5::/64 fe80::a236:9fff:fe08:a780 bond0.8 65535 + 1401 41.400 2a0a:e5c0:0:5::/64 fe80::21b:21ff:febc:bfd8 bond0.8 65535 + 1 36.722 2a0a:e5c0:0:4::/64 fe80::20d:b9ff:fe49:a705 bond0.8 96 * 1 36.419 2a0a:e5c0:1:4::/64 fe80::21b:21ff:febc:bfe0 bond0.12 65535 + 1 44.244 2a0a:e5c0:1:4::/64 fe80::21b:21ff:febc:bfe0 bond0.8 65535 + 1 44.244 2a0a:e5c0:1:4::/64 fe80::21b:21ff:febc:bfe0 bond0.38 65535 + 1 44.244 2a0a:e5c0:1:4::/64 fe80::21b:21ff:febc:bfe0 bond0.100 65535 + 1 44.244 2a0a:e5c0:1:4::/64 fe80::21b:21ff:febc:bfe0 bond0.35 65535 + 1 44.244 2a0a:e5c0:0:3::/64 fe80::20d:b9ff:fe57:2f91 bond0.8 96 * 2 36.001 2a0a:e5c0:0:2::/64 fe80::a236:9fff:fe08:a780 bond0.8 65535 + 1401 41.400 2a0a:e5c0:0:2::/64 fe80::21b:21ff:febc:bfd8 bond0.8 65535 + 1 36.722 2a0a:e5c0:2:2::/64 fe80::21b:21ff:febc:bfe0 bond0.12 65535 + 1 44.244 2a0a:e5c0:2:2::/64 fe80::21b:21ff:febc:bfe0 bond0.8 65535 + 1 44.244 2a0a:e5c0:2:2::/64 fe80::21b:21ff:febc:bfe0 bond0.38 65535 + 1 44.244 2a0a:e5c0:2:2::/64 fe80::21b:21ff:febc:bfe0 bond0.100 65535 + 1 44.244 2a0a:e5c0:2:2::/64 fe80::21b:21ff:febc:bfe0 bond0.35 65535 + 1 44.244 2a0a:e5c0:10:1::/64 fe80::21b:21ff:febc:bf36 bond0.8 96 * 1 41.012 2a0a:e5c0::/64 fe80::20d:b9ff:fe57:2f91 bond0.8 96 + 2 36.001 2a0a:e5c0::/64 fe80::20d:b9ff:fe49:a705 bond0.8 96 * 1 36.419 2a0a:e5c0:1b::/64 fe80::21b:21ff:febc:bfe0 bond0.12 65535 + 1 44.244 2a0a:e5c0:1b::/64 fe80::21b:21ff:febc:bfe0 bond0.8 65535 + 1 44.244 2a0a:e5c0:1b::/64 fe80::21b:21ff:febc:bfe0 bond0.38 65535 + 1 44.244 2a0a:e5c0:1b::/64 fe80::21b:21ff:febc:bfe0 bond0.100 65535 + 1 44.244 2a0a:e5c0:1b::/64 fe80::21b:21ff:febc:bfe0 bond0.35 65535 + 1 44.244 2a0a:e5c0:10::/64 fe80::21b:21ff:febc:bf36 bond0.8 96 * 1 41.012 2a01:330:1000:1003::/64 fe80::21b:21ff:febc:bfe0 bond0.12 65535 + 1 44.244 2a01:330:1000:1003::/64 fe80::21b:21ff:febc:bfe0 bond0.8 65535 + 1 44.244 2a01:330:1000:1003::/64 fe80::21b:21ff:febc:bfe0 bond0.38 65535 + 1 44.244 2a01:330:1000:1003::/64 fe80::21b:21ff:febc:bfe0 bond0.100 65535 + 1 44.244 2a01:330:1000:1003::/64 fe80::21b:21ff:febc:bfe0 bond0.35 65535 + 1 44.244 2a0a:e5c0:2:12::/64 fe80::21b:21ff:febc:bfe0 bond0.12 65535 + 1 44.244 2a0a:e5c0:2:12::/64 fe80::21b:21ff:febc:bfe0 bond0.8 65535 + 1 44.244 2a0a:e5c0:2:12::/64 fe80::21b:21ff:febc:bfe0 bond0.38 65535 + 1 44.244 2a0a:e5c0:2:12::/64 fe80::21b:21ff:febc:bfe0 bond0.100 65535 + 1 44.244 2a0a:e5c0:2:12::/64 fe80::21b:21ff:febc:bfe0 bond0.35 65535 + 1 44.244 2001:1700:3500::10/124 fe80::21b:21ff:febc:bfd8 bond0.8 65535 + 1 36.722 2a0a:e5c0:10:bee::/64 fe80::21b:21ff:febc:bf36 bond0.8 96 * 1 41.012 2001:1700:3500::/124 fe80::a236:9fff:fe08:a780 bond0.8 65535 + 1401 41.400 2a0a:e5c0:1::/124 fe80::21b:21ff:febc:bfe0 bond0.12 65535 + 1 44.244 2a0a:e5c0:1::/124 fe80::21b:21ff:febc:bfe0 bond0.8 65535 + 1 44.244 2a0a:e5c0:1::/124 fe80::21b:21ff:febc:bfe0 bond0.38 65535 + 1 44.244 2a0a:e5c0:1::/124 fe80::21b:21ff:febc:bfe0 bond0.100 65535 + 1 44.244 2a0a:e5c0:1::/124 fe80::21b:21ff:febc:bfe0 bond0.35 65535 + 1 44.244 [1] from router2.place6 bird> show babel routes babel1: Prefix Nexthop Interface Metric F Seqno Expires 185.203.114.0/23 45.134.132.5 bond0.35 65535 + 2204 36.139 185.203.114.0/23 94.78.2.170 bond0.100 65535 + 2204 36.139 185.203.114.0/23 147.78.195.251 bond0.8 65535 + 2204 36.139 185.203.114.0/23 185.203.114.7 bond0.12 65535 + 2204 36.138 195.141.230.102/31 147.78.195.250 bond0.8 65535 + 1 40.620 185.203.112.0/24 147.78.195.249 bond0.8 65535 + 1401 36.890 185.203.112.0/24 147.78.195.250 bond0.8 65535 + 1 40.620 147.78.195.224/27 147.78.195.227 bond0.8 65535 + 1 44.353 147.78.195.224/27 185.203.114.7 bond0.12 65535 + 2204 36.138 147.78.195.224/27 94.78.2.170 bond0.100 65535 + 2204 36.139 147.78.195.224/27 45.134.132.5 bond0.35 65535 + 2204 36.139 147.78.195.224/27 147.78.195.251 bond0.8 65535 + 2204 36.139 147.78.195.224/27 147.78.195.249 bond0.8 65535 + 1401 36.890 147.78.195.224/27 147.78.195.250 bond0.8 65535 + 1 40.620 193.192.225.72/31 147.78.195.249 bond0.8 65535 + 1401 36.890 94.78.2.168/29 45.134.132.5 bond0.35 65535 + 2204 36.139 94.78.2.168/29 94.78.2.170 bond0.100 65535 + 2204 36.139 94.78.2.168/29 147.78.195.251 bond0.8 65535 + 2204 36.139 94.78.2.168/29 185.203.114.7 bond0.12 65535 + 2204 36.138 147.78.195.0/29 147.78.195.227 bond0.8 65535 + 1 44.353 185.155.31.0/24 45.134.132.5 bond0.35 65535 + 2204 36.139 185.155.31.0/24 94.78.2.170 bond0.100 65535 + 2204 36.139 185.155.31.0/24 147.78.195.251 bond0.8 65535 + 2204 36.139 185.155.31.0/24 185.203.114.7 bond0.12 65535 + 2204 36.138 45.134.132.0/29 45.134.132.5 bond0.35 65535 + 2204 36.139 45.134.132.0/29 94.78.2.170 bond0.100 65535 + 2204 36.139 45.134.132.0/29 147.78.195.251 bond0.8 65535 + 2204 36.139 45.134.132.0/29 185.203.114.7 bond0.12 65535 + 2204 36.138 185.155.31.0/26 45.134.132.5 bond0.35 65535 + 2204 36.139 185.155.31.0/26 94.78.2.170 bond0.100 65535 + 2204 36.139 185.155.31.0/26 147.78.195.251 bond0.8 65535 + 2204 36.139 185.155.31.0/26 185.203.114.7 bond0.12 65535 + 2204 36.138 2a0a:e5c0:1:f::/64 fe80::21b:21ff:febc:bf36 bond0.8 65535 + 1 44.353 2a0a:e5c0:1:e::/64 fe80::21b:21ff:febc:bf36 bond0.8 65535 + 1 44.353 2a0a:e5c0:1:d::/64 fe80::a236:9fff:fe08:a150 bond0.38 65535 + 2204 36.139 2a0a:e5c0:1:d::/64 fe80::a236:9fff:fe08:a150 bond0.35 65535 + 2204 36.139 2a0a:e5c0:1:d::/64 fe80::a236:9fff:fe08:a150 bond0.100 65535 + 2204 36.139 2a0a:e5c0:1:d::/64 fe80::a236:9fff:fe08:a150 bond0.8 65535 + 2204 36.139 2a0a:e5c0:1:d::/64 fe80::a236:9fff:fe08:a150 bond0.12 65535 + 2204 36.138 2a0a:e5c0:0:9::/64 fe80::20d:b9ff:fe49:a705 bond0.8 96 * 1 40.196 2a0a:e5c0:1:8::/64 fe80::20d:b9ff:fe57:2f91 bond0.8 96 * 2 36.140 2a0a:e5c0:1:8::/64 fe80::20d:b9ff:fe49:a705 bond0.8 96 + 1 40.196 2a0a:e5c0:1:8::/64 fe80::21b:21ff:febc:bf36 bond0.8 65535 + 1 44.353 2a0a:e5c0:1:8::/64 fe80::a236:9fff:fe08:a150 bond0.12 65535 + 2204 36.138 2a0a:e5c0:1:8::/64 fe80::a236:9fff:fe08:a150 bond0.38 65535 + 2204 36.139 2a0a:e5c0:1:8::/64 fe80::a236:9fff:fe08:a150 bond0.100 65535 + 2204 36.139 2a0a:e5c0:1:8::/64 fe80::a236:9fff:fe08:a150 bond0.35 65535 + 2204 36.139 2a0a:e5c0:1:8::/64 fe80::a236:9fff:fe08:a150 bond0.8 65535 + 2204 36.139 2a0a:e5c0:1:8::/64 fe80::a236:9fff:fe08:a780 bond0.8 65535 + 1401 36.890 2a0a:e5c0:1:8::/64 fe80::21b:21ff:febc:bfd8 bond0.8 65535 + 1 40.620 2a0a:e5c0:0:6::/64 fe80::20d:b9ff:fe57:2f91 bond0.8 96 * 2 36.140 2a0a:e5c0:0:5::/64 fe80::a236:9fff:fe08:a780 bond0.8 65535 + 1401 36.890 2a0a:e5c0:0:5::/64 fe80::21b:21ff:febc:bfd8 bond0.8 65535 + 1 40.620 2a0a:e5c0:0:4::/64 fe80::20d:b9ff:fe49:a705 bond0.8 96 * 1 40.196 2a0a:e5c0:1:4::/64 fe80::a236:9fff:fe08:a150 bond0.38 65535 + 2204 36.139 2a0a:e5c0:1:4::/64 fe80::a236:9fff:fe08:a150 bond0.35 65535 + 2204 36.139 2a0a:e5c0:1:4::/64 fe80::a236:9fff:fe08:a150 bond0.100 65535 + 2204 36.139 2a0a:e5c0:1:4::/64 fe80::a236:9fff:fe08:a150 bond0.8 65535 + 2204 36.139 2a0a:e5c0:1:4::/64 fe80::a236:9fff:fe08:a150 bond0.12 65535 + 2204 36.138 2a0a:e5c0:0:3::/64 fe80::20d:b9ff:fe57:2f91 bond0.8 96 * 2 36.140 2a0a:e5c0:0:2::/64 fe80::a236:9fff:fe08:a780 bond0.8 65535 + 1401 36.890 2a0a:e5c0:0:2::/64 fe80::21b:21ff:febc:bfd8 bond0.8 65535 + 1 40.620 2a0a:e5c0:2:2::/64 fe80::a236:9fff:fe08:a150 bond0.38 65535 + 2204 36.139 2a0a:e5c0:2:2::/64 fe80::a236:9fff:fe08:a150 bond0.35 65535 + 2204 36.139 2a0a:e5c0:2:2::/64 fe80::a236:9fff:fe08:a150 bond0.100 65535 + 2204 36.139 2a0a:e5c0:2:2::/64 fe80::a236:9fff:fe08:a150 bond0.8 65535 + 2204 36.139 2a0a:e5c0:2:2::/64 fe80::a236:9fff:fe08:a150 bond0.12 65535 + 2204 36.138 2a0a:e5c0:10:1::/64 fe80::21b:21ff:febc:bf36 bond0.8 65535 + 1 44.353 2a0a:e5c0::/64 fe80::20d:b9ff:fe57:2f91 bond0.8 96 * 2 36.140 2a0a:e5c0::/64 fe80::20d:b9ff:fe49:a705 bond0.8 96 + 1 40.196 2a0a:e5c0:1b::/64 fe80::a236:9fff:fe08:a150 bond0.38 65535 + 2204 36.139 2a0a:e5c0:1b::/64 fe80::a236:9fff:fe08:a150 bond0.35 65535 + 2204 36.139 2a0a:e5c0:1b::/64 fe80::a236:9fff:fe08:a150 bond0.100 65535 + 2204 36.139 2a0a:e5c0:1b::/64 fe80::a236:9fff:fe08:a150 bond0.8 65535 + 2204 36.139 2a0a:e5c0:1b::/64 fe80::a236:9fff:fe08:a150 bond0.12 65535 + 2204 36.138 2a0a:e5c0:10::/64 fe80::21b:21ff:febc:bf36 bond0.8 65535 + 1 44.353 2a01:330:1000:1003::/64 fe80::a236:9fff:fe08:a150 bond0.38 65535 + 2204 36.139 2a01:330:1000:1003::/64 fe80::a236:9fff:fe08:a150 bond0.35 65535 + 2204 36.139 2a01:330:1000:1003::/64 fe80::a236:9fff:fe08:a150 bond0.100 65535 + 2204 36.139 2a01:330:1000:1003::/64 fe80::a236:9fff:fe08:a150 bond0.8 65535 + 2204 36.139 2a01:330:1000:1003::/64 fe80::a236:9fff:fe08:a150 bond0.12 65535 + 2204 36.138 2a0a:e5c0:2:12::/64 fe80::a236:9fff:fe08:a150 bond0.38 65535 + 2204 36.139 2a0a:e5c0:2:12::/64 fe80::a236:9fff:fe08:a150 bond0.35 65535 + 2204 36.139 2a0a:e5c0:2:12::/64 fe80::a236:9fff:fe08:a150 bond0.100 65535 + 2204 36.139 2a0a:e5c0:2:12::/64 fe80::a236:9fff:fe08:a150 bond0.8 65535 + 2204 36.139 2a0a:e5c0:2:12::/64 fe80::a236:9fff:fe08:a150 bond0.12 65535 + 2204 36.138 2001:1700:3500::10/124 fe80::21b:21ff:febc:bfd8 bond0.8 65535 + 1 40.620 2a0a:e5c0:10:bee::/64 fe80::21b:21ff:febc:bf36 bond0.8 65535 + 1 44.353 2001:1700:3500::/124 fe80::a236:9fff:fe08:a780 bond0.8 65535 + 1401 36.890 2a0a:e5c0:1::/124 fe80::a236:9fff:fe08:a150 bond0.38 65535 + 2204 36.139 2a0a:e5c0:1::/124 fe80::a236:9fff:fe08:a150 bond0.35 65535 + 2204 36.139 2a0a:e5c0:1::/124 fe80::a236:9fff:fe08:a150 bond0.100 65535 + 2204 36.139 2a0a:e5c0:1::/124 fe80::a236:9fff:fe08:a150 bond0.8 65535 + 2204 36.139 2a0a:e5c0:1::/124 fe80::a236:9fff:fe08:a150 bond0.12 65535 + 2204 36.138 bird> -- Sustainable and modern Infrastructures by ungleich.ch