(added bird-users@ back, I didn't mean to remove them before) Sorry that I missed that you'd already added the pull request. I merged it. we do need a way to specify exabgp vs bird, but that's just another thing to add. I agree about containers and versions. Just haven't had time to think it through carefully. The policy support is what bgperf had before I forked it. I don't use it, I don't know how to usefully test with it. One more feature I'd like to see, yet I'm not sure how to do this
properly – to make the measurement more accurate, the target should not only run on its own machine, but bgperf with all the testers should also watch their load, issuing warning when the bgperf suite may be the bottleneck. This will probably help testing some hardware routers in a somehow comparable way.
I agree. My best quick hack for this is to make sure that there are extra resources on the CPU when running. That's why bgperf now tracks %idle. As you can probably tell, bgperf isn't a great architecture now, and I'm trying to get useful results as quickly as possible. So much could be done here. thanks a lot! Justin On Tue, Aug 31, 2021 at 2:30 AM Maria Matejka <maria.matejka@nic.cz> wrote:
Hello!
On 8/31/21 1:29 AM, Justin Pietsch wrote:
I meant to ask if you'd compared exabgp to bird as testers? I'll try to compare later if I have some time. Well, I have no exact numbers, I just implemented the bird tester and it seems to eat less memory and less time. You could probably run the same batch first with exabgp testers and then with bird testers, comparing how much this affects the results.
On Mon, Aug 30, 2021 at 4:27 PM Justin Pietsch <jpietsch@gmail.com <mailto:jpietsch@gmail.com>> wrote:
This is fantastic!!!! Can you submit those changes as PRs to my bgperf? I'm working on moving it to a more permanent place. Then we can track issues! :)
Well, yes, the tester exchange is already there pending:
https://github.com/jopietsch/bgperf/pull/1
and it seems that there are two more from somebody else.
The tester exchange is anyway quite stupid and may need some similar code to MRT testers instead of what is there now.
your suggestion for specifying branches used to work in bgperf, but I broke it in trying to get everything working. I moved some of the targets to just download prebuilt docker images, which makes it harder. Also with batch I'm not sure how to have multiple versions of the same container. Just more things to figure out, but that all makes sense.
I think it would be better to have separate containers for different branches, yet I couldn't figure out how to do that, except for brutal solutions which were too disgusting to implement.
Anyway, when BIRD is used as tester, it shouldn't imho change with the target, which is probably the main reason why I prefer separate containers for branches.
One more thing – when running update -c, the container should also pull, not only checkout, imho. (Haven't had time to fix this yet.)
* fixed policy configuration in BIRD (no clue whether it works
with
other stacks)
This may need thorough examination first to formally define what the policies should mean. It would be really stupid to compare route filtering with different policies in each stack.
One more feature I'd like to see, yet I'm not sure how to do this properly – to make the measurement more accurate, the target should not only run on its own machine, but bgperf with all the testers should also watch their load, issuing warning when the bgperf suite may be the bottleneck. This will probably help testing some hardware routers in a somehow comparable way.
Maria