Hi. Anyone knows if Bird documentation has any information about BGP table size vs. number of routes? Thanks. Emerson
Hello! BIRD has in concept no limit on number of routes and number of prefices. It should scale quite well. If not, please let us know. Maria On 8/12/19 2:15 PM, Emerson Barea wrote:
Hi. Anyone knows if Bird documentation has any information about BGP table size vs. number of routes? Thanks. Emerson
Sorry, I think that I didn't explaim well in my first message. I want to know if exist any information that tell me how much RAM memory 100k BGP prefix uses in Bird, for example. I know Cisco uses between 28MB and 71MB of memory by 100k prefix [1]. So, I want to know if Bird has some information like this. Thank you Emerson [1] hhttps:// www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/12512-41.html Em seg, 12 de ago de 2019 às 09:33, Maria Matejka <jan.matejka@nic.cz> escreveu:
Hello! BIRD has in concept no limit on number of routes and number of prefices. It should scale quite well. If not, please let us know. Maria
On 8/12/19 2:15 PM, Emerson Barea wrote:
Hi. Anyone knows if Bird documentation has any information about BGP table size vs. number of routes? Thanks. Emerson
Basically, we eat about 64 bytes per prefix, 64 bytes per route and (112 + variable) deduplicated route attributes. Therefore in the most basic case, you can have 6.4MB for 100k prefices + 6.4MB for 100k routes (one per prefix) + 112 bytes when all the prefices have the same nexthop and nothing else, in total ~13 MB. If you get a full 4k packet of BGP attributes for each route, the latest value is not 112 bytes but ~4k*100k ~ 400MB in (probably) worst case. Looking into our testbed having full IPv4 BGP feed loaded (760k prefices), BIRD eats 89MB for route table entries and 48MB for the deduplicated attributes, in total 137 MB. After normalization to 100k routes, you get 137 MB / 7.6 ~ 18 MB. Maria On 8/12/19 3:56 PM, Emerson Barea wrote:
Sorry, I think that I didn't explaim well in my first message.
I want to know if exist any information that tell me how much RAM memory 100k BGP prefix uses in Bird, for example.
I know Cisco uses between 28MB and 71MB of memory by 100k prefix [1]. So, I want to know if Bird has some information like this.
Thank you
Emerson
[1] hhttps://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/12... <http://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/12512-41.html>
Em seg, 12 de ago de 2019 às 09:33, Maria Matejka <jan.matejka@nic.cz <mailto:jan.matejka@nic.cz>> escreveu:
Hello! BIRD has in concept no limit on number of routes and number of prefices. It should scale quite well. If not, please let us know. Maria
On 8/12/19 2:15 PM, Emerson Barea wrote: > Hi. Anyone knows if Bird documentation has any information about BGP > table size vs. number of routes? > Thanks. > Emerson
Hi, Emerson, For example, this is an actual Bird 1.6.7: 82968 of 82968 routes for 41553 networks bird 22191 1.6 2.6 123740 107788 And this is the same In a different environment: 544703 of 544703 routes for 503219 networks bird 22191 1.7 5.5 236172 220244 It is not a memory you should be concerned about. The main issue of bird with huge BGP tables is CPU – bird uses only one core. -- Best regards, Valery Lutoshkin From: Bird-users [mailto:bird-users-bounces@network.cz] On Behalf Of Emerson Barea Sent: Monday, August 12, 2019 8:57 PM To: Maria Matejka Cc: bird-users@network.cz Subject: Re: BGP table size Sorry, I think that I didn't explaim well in my first message. I want to know if exist any information that tell me how much RAM memory 100k BGP prefix uses in Bird, for example. I know Cisco uses between 28MB and 71MB of memory by 100k prefix [1]. So, I want to know if Bird has some information like this. Thank you Emerson [1] hhttps://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/12... Em seg, 12 de ago de 2019 às 09:33, Maria Matejka <jan.matejka@nic.cz> escreveu: Hello! BIRD has in concept no limit on number of routes and number of prefices. It should scale quite well. If not, please let us know. Maria On 8/12/19 2:15 PM, Emerson Barea wrote:
Hi. Anyone knows if Bird documentation has any information about BGP table size vs. number of routes? Thanks. Emerson
It is not a memory you should be concerned about. The main issue of bird with huge BGP tables is CPU – bird uses only one core.
Which is what is going to be fixed in a short time (about a month to first test release, I hope), at least if you don't use Kernel protocol. Merge to master is planned to early 2020, which is a bit optimistic estimation but still reasonable. Maria
On 2019-08-13 09:58, Maria Matějka wrote:
It is not a memory you should be concerned about. The main issue of bird with huge BGP tables is CPU – bird uses only one core.
Which is what is going to be fixed in a short time (about a month to first test release, I hope), at least if you don't use Kernel protocol.
Merge to master is planned to early 2020, which is a bit optimistic estimation but still reasonable.
Maria
Hello Maria, This is very interesting news! Can you tell me more about this (maybe in a separate thread, to not pollute this one)? first few questions I do have: 1. Will this be done only in the 2.x branch, or also in the 1.6.x? 2. If you do use the Kernel protocol, is the entire Bird process still single threaded, or only the part regarding the Kernel protocol? So, calculating which route is best could still be multi threaded, but sending the end result to the kernel is single threaded 3. If you do use the Kernel protocol to send routes to the kernel, is there any alternative to get Bird fully multi threaded? I run Bird on 4 routers, most of them have 8 cores, one of them even has 12 cores, so I would love to be able to use all those cores for Bird! And I could even double the amount of cores if I would enable Hyper Threading again, but from a security point of view I don't want to do that. Kind regards, Cybertinus
This is very interesting news! Can you tell me more about this (maybe in a separate thread, to not pollute this one)? first few questions I do have:
Changing the subject, it should be enough.
1. Will this be done only in the 2.x branch, or also in the 1.6.x?
Only in 2.x; these changes include sh*tload of changes in route propagation which we won't do twice.
2. If you do use the Kernel protocol, is the entire Bird process still single threaded, or only the part regarding the Kernel protocol? So, calculating which route is best could still be multi threaded, but sending the end result to the kernel is single threaded
The multithreaded BIRD will be in a separate code branch, a separate version, separate binary. If you opt for multithreaded, no kernel protocol will be available at all.
3. If you do use the Kernel protocol to send routes to the kernel, is there any alternative to get Bird fully multi threaded?
To wait until we rework the kernel-nest interface; then we'll merge multithreaded execution into master.
I run Bird on 4 routers, most of them have 8 cores, one of them even has 12 cores, so I would love to be able to use all those cores for Bird! And I could even double the amount of cores if I would enable Hyper Threading again, but from a security point of view I don't want to do that.
At least if you have some non-trivial filters, it will help you. The first code to be multithreaded is filter code, then probably other parts will follow. Maria
Hello, Do you have aby plans for integrating BIRD with non kernel data planes? I mean something along the lines of FIB protocol which dumps routes into data plane. Best regards, Łukasz Jarosz wt., 13 sie 2019, 13:21 użytkownik Maria Jan Matejka <jan.matejka@nic.cz> napisał:
This is very interesting news! Can you tell me more about this (maybe in a separate thread, to not pollute this one)? first few questions I do have:
Changing the subject, it should be enough.
1. Will this be done only in the 2.x branch, or also in the 1.6.x?
Only in 2.x; these changes include sh*tload of changes in route propagation which we won't do twice.
2. If you do use the Kernel protocol, is the entire Bird process still single threaded, or only the part regarding the Kernel protocol? So, calculating which route is best could still be multi threaded, but sending the end result to the kernel is single threaded
The multithreaded BIRD will be in a separate code branch, a separate version, separate binary. If you opt for multithreaded, no kernel protocol will be available at all.
3. If you do use the Kernel protocol to send routes to the kernel, is there any alternative to get Bird fully multi threaded?
To wait until we rework the kernel-nest interface; then we'll merge multithreaded execution into master.
I run Bird on 4 routers, most of them have 8 cores, one of them even has 12 cores, so I would love to be able to use all those cores for Bird! And I could even double the amount of cores if I would enable Hyper Threading again, but from a security point of view I don't want to do that.
At least if you have some non-trivial filters, it will help you. The first code to be multithreaded is filter code, then probably other parts will follow.
Maria
On 8/13/19 1:24 PM, Łukasz Jarosz wrote:
Hello, Do you have aby plans for integrating BIRD with non kernel data planes? I mean something along the lines of FIB protocol which dumps routes into data plane.
What non-kernel data planes would you like? We have currently no plan to implement anything like this. Anyway there are some plans to have a python/lua binding to write your own protocol and stabilize the internal API. For the stabilisation and documentation, it may be handy to know what kind of route import/export you need to include it there. Maria
I am not an expert here, but VPP seems like an obvious example. I will probably have to write some proof concept and come back with it. Due to multithreading, how big changes to internal API are planned? wt., 13 sie 2019 o 13:31 Maria Jan Matejka <jan.matejka@nic.cz> napisał(a):
On 8/13/19 1:24 PM, Łukasz Jarosz wrote:
Hello, Do you have aby plans for integrating BIRD with non kernel data planes? I mean something along the lines of FIB protocol which dumps routes into data plane.
What non-kernel data planes would you like? We have currently no plan to implement anything like this. Anyway there are some plans to have a python/lua binding to write your own protocol and stabilize the internal API. For the stabilisation and documentation, it may be handy to know what kind of route import/export you need to include it there.
Maria
On 8/13/19 1:43 PM, Łukasz Jarosz wrote:
I am not an expert here, but VPP seems like an obvious example. I will probably have to write some proof concept and come back with it. Due to multithreading, how big changes to internal API are planned?
If you are working in BGP-like mode (single prefix updates/withdraws), the changes are trivial. If you need full table sync every other while, like kernel does, there will be completely different API, not implemented yet. If you eventually come to Prague, let us know to have a short meeting to discuss this. Or we're going to Rotterdam to RIPE 79 in October, feel free to reach us there. Maria
On 2019-08-13 13:08, Maria Jan Matejka wrote:
1. Will this be done only in the 2.x branch, or also in the 1.6.x?
Only in 2.x; these changes include sh*tload of changes in route propagation which we won't do twice.
Right, makes sense indeed. Then I have to upgrade from 1.6 to 2.0 :)
2. If you do use the Kernel protocol, is the entire Bird process still single threaded, or only the part regarding the Kernel protocol? So, calculating which route is best could still be multi threaded, but sending the end result to the kernel is single threaded
The multithreaded BIRD will be in a separate code branch, a separate version, separate binary. If you opt for multithreaded, no kernel protocol will be available at all.
OK, that's a clear and conscious decision for the users then indeed.
3. If you do use the Kernel protocol to send routes to the kernel, is there any alternative to get Bird fully multi threaded?
To wait until we rework the kernel-nest interface; then we'll merge multithreaded execution into master.
OK, then I have to wait for that, because I do use the Kernel protocol to place the best routes from all my peers into the kernel routing table.
I run Bird on 4 routers, most of them have 8 cores, one of them even has 12 cores, so I would love to be able to use all those cores for Bird! And I could even double the amount of cores if I would enable Hyper Threading again, but from a security point of view I don't want to do that.
At least if you have some non-trivial filters, it will help you. The first code to be multithreaded is filter code, then probably other parts will follow.
I filter a lot. Currently my Bird config is 78 MB in size, most of it is filtering. That should be noticeable then. Thanks for your answers. Is there some easy way to follow the process for this, apart from regularly check out the git repo and reading through the commit messages?
Maria
Kind regards, Cybertinus
On 8/13/19 2:51 PM, Cybertinus wrote:
On 2019-08-13 13:08, Maria Jan Matejka wrote:
1. Will this be done only in the 2.x branch, or also in the 1.6.x?
Only in 2.x; these changes include sh*tload of changes in route propagation which we won't do twice.
Right, makes sense indeed. Then I have to upgrade from 1.6 to 2.0 :)
Of course. After we fix a stupid bug in 2.0.5, it should also yield faster filters.
3. If you do use the Kernel protocol to send routes to the kernel, is there any alternative to get Bird fully multi threaded?
To wait until we rework the kernel-nest interface; then we'll merge multithreaded execution into master.
OK, then I have to wait for that, because I do use the Kernel protocol to place the best routes from all my peers into the kernel routing table.
You can use a dirty hack, run one multithreaded BIRD for filtering and another BIRD at the same machine for route export to kernel, connected locally by a BGP session.
I run Bird on 4 routers, most of them have 8 cores, one of them even has 12 cores, so I would love to be able to use all those cores for Bird! And I could even double the amount of cores if I would enable Hyper Threading again, but from a security point of view I don't want to do that.
At least if you have some non-trivial filters, it will help you. The first code to be multithreaded is filter code, then probably other parts will follow.
I filter a lot. Currently my Bird config is 78 MB in size, most of it is filtering. That should be noticeable then.
Thanks for your answers. Is there some easy way to follow the process for this, apart from regularly check out the git repo and reading through the commit messages?
I'll keep this mailing list updated as we need people to test it. Parallel execution of legacy code is always a can of worms ... a truck full of cans of worms. Maria
participants (7)
-
Cybertinus -
Emerson Barea -
Maria Jan Matejka -
Maria Matejka
-
Maria Matějka
-
Łukasz Jarosz -
Валерий Н. Лутошкин