comparing open source BGP stacks performance

[Justin Pietsch]

(added bird-users@ back, I didn't mean to remove them before)

Sorry that I missed that you'd already added the pull request. I merged it.
we do need a way to specify exabgp vs bird, but that's just another thing
to add.

I agree about containers and versions. Just haven't had time to think it
through carefully.

The policy support is what bgperf had before I forked it. I don't use it, I
don't know how to usefully test with it.

One more feature I'd like to see, yet I'm not sure how to do this
> properly – to make the measurement more accurate, the target should not
> only run on its own machine, but bgperf with all the testers should also
> watch their load, issuing warning when the bgperf suite may be the
> bottleneck. This will probably help testing some hardware routers in a
> somehow comparable way.
>

I agree. My best quick hack for this is to make sure that there are extra
resources on the CPU when running. That's why bgperf now tracks %idle.

As you can probably tell,  bgperf isn't a great architecture now, and I'm
trying to get useful results as quickly as possible. So much could be done
here.

thanks a lot!

Justin

On Tue, Aug 31, 2021 at 2:30 AM Maria Matejka <maria.matejka at nic.cz> wrote:

> Hello!
>
> On 8/31/21 1:29 AM, Justin Pietsch wrote:
> > I meant to ask if you'd compared exabgp to bird as testers? I'll try to
> > compare later if I have some time.
> Well, I have no exact numbers, I just implemented the bird tester and it
> seems to eat less memory and less time. You could probably run the same
> batch first with exabgp testers and then with bird testers, comparing
> how much this affects the results.
>
> > On Mon, Aug 30, 2021 at 4:27 PM Justin Pietsch <jpietsch at gmail.com
> > <mailto:jpietsch at gmail.com>> wrote:
> >
> >     This is fantastic!!!! Can you submit those changes as PRs to my
> >     bgperf? I'm working on moving it to a more permanent place. Then we
> >     can track issues! :)
>
> Well, yes, the tester exchange is already there pending:
>
>         https://github.com/jopietsch/bgperf/pull/1
>
> and it seems that there are two more from somebody else.
>
> The tester exchange is anyway quite stupid and may need some similar
> code to MRT testers instead of what is there now.
>
> >     your suggestion for specifying branches used to work in bgperf, but
> >     I broke it in trying to get everything working. I moved some of the
> >     targets to just download prebuilt docker images, which makes it
> >     harder. Also with batch I'm not sure how to have multiple versions
> >     of the same container. Just more things to figure out, but that all
> >     makes sense.
>
> I think it would be better to have separate containers for different
> branches, yet I couldn't figure out how to do that, except for brutal
> solutions which were too disgusting to implement.
>
> Anyway, when BIRD is used as tester, it shouldn't imho change with the
> target, which is probably the main reason why I prefer separate
> containers for branches.
>
> One more thing – when running update -c, the container should also pull,
> not only checkout, imho. (Haven't had time to fix this yet.)
>
> >         * fixed policy configuration in BIRD (no clue whether it works
> with
> >         other stacks)
>
> This may need thorough examination first to formally define what the
> policies should mean. It would be really stupid to compare route
> filtering with different policies in each stack.
>
> One more feature I'd like to see, yet I'm not sure how to do this
> properly – to make the measurement more accurate, the target should not
> only run on its own machine, but bgperf with all the testers should also
> watch their load, issuing warning when the bgperf suite may be the
> bottleneck. This will probably help testing some hardware routers in a
> somehow comparable way.
>
> Maria
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://trubka.network.cz/pipermail/bird-users/attachments/20210901/d31db0a4/attachment.htm>