BIRD Testing With Ansible?
TL;DR: Should we test BIRD with Ansible? Hello! Dear fellow users and anybody else reading this message, please take a deep breath and read the story of testing BIRD before you shout NO, YES, DEFINITELY MAYBE or whatever else. Long time ago, when Linux implemented Network Namespaces feature, even before anybody knew anything about "ip netns", Santiago created a simple but quite powerful tool called "netlab" to test BIRD on one machine, using this exact feature. It has grown to quite a decent set of automatic functionality tests which you can see in the bird-tools repository[1] in the "netlab" directory. After some time, when I got to BIRD development, we wanted to test also BIRD on different platforms. We went to virtualization, setup a decent testing setup with QEMU-KVM and OpenVSwitch and then I tried to run there a simple MPLS setup when the OpenVSwitch suddenly crashed on SegFault. I solved it by using just plain old bridges and veth's. When I met some friends several days later, one of them being a Linux kernel developer working also on OpenVSwitch, I mentioned these problems, thinking that he may give me a pointer how to solve it. Instead, he interrupted me, saying "hey, shut up, this is an embargoed bug". I had missed that this segfault may be a security problem leading at least to DoS. This was happening when the OpenStack and LibVirt and others were still in an early phase of development and we couldn't find any suitable solution for us so we just developed something for ourselves. Anyway, now it is 2022 and when I reach out to almost any user of BIRD, I hear about Ansible. I'm thinking whether it is a good idea to leave our old tooling in a museum and to migrate to Ansible. Facts: * Our team is familiar with the current tooling, not Ansible. * It's probably easier to learn how to use our tooling than Ansible, provided you have seen neither of them before. * Some of the users are already using BIRD with Ansible. * We don't know much about pros and cons of Ansible. Proposal: We would retire our old testing setups and instead of that we would upstream a bunch of functionality tests implemented in Ansible. I have some questions for you: (1) Are you using Ansible with BIRD? (2) Do you have some testing setup implemented with Ansible? (3) Would you run the upstream functionality tests if they used Ansible? (4) If encountering a bug, would you consider creating an Ansible scenario to replicate that behavior? (5) If contributing, would it be OK for you to create also an appropriate functionality test in Ansible, or rather in our current tooling? Thank you all for your answers and discussion, please don't throw chairs nor eggs nor tomatoes on each other. Maria
Hi,
From my point of view it is somewhat orthogonal things. For me Ansible is more like a deploy tool, not testing. And probably for creating a testing setup with Ansible you need some additional tools to support creating of the environment.
(1) Are you using Ansible with BIRD?
We use ansible with bird, but there is nothing bird-specific in it. We just generate configs from templates and deploy them with Ansible.
(2) Do you have some testing setup implemented with Ansible?
For testing with Ansible, I think I would need to create a test envorinment by some means and Ansbile would only be used to deploy the configs to it. So no, we do not have testing implemented with Ansible.
(3) Would you run the upstream functionality tests if they used Ansible?
I do not actually understant what is it about. :)
(4) If encountering a bug, would you consider creating an Ansible scenario to replicate that behavior? (5) If contributing, would it be OK for you to create also an appropriate functionality test in Ansible, or rather in our current tooling?
I think if there are examples of other similar test nearby, there is not much difference. On Wed, Jun 1, 2022 at 11:44 AM Maria Matejka <maria.matejka@nic.cz> wrote:
TL;DR: Should we test BIRD with Ansible?
Hello!
Dear fellow users and anybody else reading this message, please take a deep breath and read the story of testing BIRD before you shout NO, YES, DEFINITELY MAYBE or whatever else.
Long time ago, when Linux implemented Network Namespaces feature, even before anybody knew anything about "ip netns", Santiago created a simple but quite powerful tool called "netlab" to test BIRD on one machine, using this exact feature. It has grown to quite a decent set of automatic functionality tests which you can see in the bird-tools repository[1] in the "netlab" directory.
After some time, when I got to BIRD development, we wanted to test also BIRD on different platforms. We went to virtualization, setup a decent testing setup with QEMU-KVM and OpenVSwitch and then I tried to run there a simple MPLS setup when the OpenVSwitch suddenly crashed on SegFault. I solved it by using just plain old bridges and veth's.
When I met some friends several days later, one of them being a Linux kernel developer working also on OpenVSwitch, I mentioned these problems, thinking that he may give me a pointer how to solve it. Instead, he interrupted me, saying "hey, shut up, this is an embargoed bug". I had missed that this segfault may be a security problem leading at least to DoS.
This was happening when the OpenStack and LibVirt and others were still in an early phase of development and we couldn't find any suitable solution for us so we just developed something for ourselves.
Anyway, now it is 2022 and when I reach out to almost any user of BIRD, I hear about Ansible. I'm thinking whether it is a good idea to leave our old tooling in a museum and to migrate to Ansible.
Facts: * Our team is familiar with the current tooling, not Ansible. * It's probably easier to learn how to use our tooling than Ansible, provided you have seen neither of them before. * Some of the users are already using BIRD with Ansible. * We don't know much about pros and cons of Ansible.
Proposal: We would retire our old testing setups and instead of that we would upstream a bunch of functionality tests implemented in Ansible.
I have some questions for you:
(1) Are you using Ansible with BIRD? (2) Do you have some testing setup implemented with Ansible? (3) Would you run the upstream functionality tests if they used Ansible? (4) If encountering a bug, would you consider creating an Ansible scenario to replicate that behavior? (5) If contributing, would it be OK for you to create also an appropriate functionality test in Ansible, or rather in our current tooling?
Thank you all for your answers and discussion, please don't throw chairs nor eggs nor tomatoes on each other.
Maria
On Wed, Jun 1, 2022 at 7:14 AM Alexander Zubkov <green@qrator.net> wrote:
Hi,
From my point of view it is somewhat orthogonal things. For me Ansible is more like a deploy tool, not testing. And probably for creating a testing setup with Ansible you need some additional tools to support creating of the environment.
(1) Are you using Ansible with BIRD?
We use ansible with bird, but there is nothing bird-specific in it. We just generate configs from templates and deploy them with Ansible.
Same here, 100%. I use Ansible to deploy and configure BIRD, but would not have thought about using it to test BIRD.
Hello!
From my point of view it is somewhat orthogonal things. For me Ansible is more like a deploy tool, not testing. And probably for creating a testing setup with Ansible you need some additional tools to support creating of the environment.
(1) Are you using Ansible with BIRD?
We use ansible with bird, but there is nothing bird-specific in it. We just generate configs from templates and deploy them with Ansible.
Same here, 100%. I use Ansible to deploy and configure BIRD, but would not have thought about using it to test BIRD.
Well, to setup a test case, we have to simulate a production environment somehow. We can either continue using our current tooling which we have to document and extend quite a lot. Or we can look somewhere else. This is why I'm asking -- we'll keep using qemu-kvm and netns to test different cases, as well as our Python scripts to check the results. I'm asking about the orchestration, how familiar are the users with using Ansible (or any other orchestration tool). Therefore the question may have also been – is Ansible a good and commonly used tool for BIRD deployment? BR. Maria
On Wednesday, June 1, 2022 8:32:43 AM ADT Maria Matejka wrote:
Well, to setup a test case, we have to simulate a production environment somehow. We can either continue using our current tooling which we have to document and extend quite a lot. Or we can look somewhere else.
This is why I'm asking -- we'll keep using qemu-kvm and netns to test different cases, as well as our Python scripts to check the results. I'm asking about the orchestration, how familiar are the users with using Ansible (or any other orchestration tool).
Therefore the question may have also been – is Ansible a good and commonly used tool for BIRD deployment?
I never thought about using Ansible for something like this. We used to use Linaro LAVA but ran into scaling issues. Now we do this with a thing we wrote mostly in bash. The idea is that we have a lot of complicated setups we need to test, some with even more than 10 roles. To do this we use prebuilt root filesystem images that get cloned for each role. The test script dumps the test code for the scenario on the image and sets up a systemd service to be booted when the role boots. Then we boot each one via systemd-nspawn, qemu-kvm, or even a physical device, netbooted from the modified image. To make it super fast to set up and boot we use Ceph for the image cloning but it's not really necessary. Btrfs or overlayfs could probably be used in its place while maintaining the speed without the external dependency. Since all the images are simply directories on a single box as the roles are running, you can have something outside collecting files with test results generated on the the running roles, even while they are running. We actually use MQTT to allow the roles to sync with each other and report results. Thanks to the way the filesystems are set up it would probably be easy to use a shared bind mount so they could sync via files as well. For networking, we just throw it all on a single bridge. If roles need to be isolated, we just set up VLANs on the roles when they start up. We also set it up so that the test scripts are identical on all roles. That way we can do stuff like this: if is_role gateway ; then # do something to set it up fi sync gateway-configured if is_role client ; then # Do client thing against gateway fi -- James Oakley james@ttgi.io
Ahoj Maria! For our production RS'es we do not use Ansible, yet we do for Restena route collector machine which runs on a Centos custom build and what we have done with Ansible is : . Building the bird config . Install bird2 . Ensure bird2 config folder exists . Remove old configs . Configure bird2 . Ensure bird2 is running and enabled Also we have used Ansible to deploy bird_exporter : . Installing the systemd service . Building its config and so on. I would be very glad to have soon bird officially being Ansible'd . Regards, See you soon Irene, On 01/06/2022 11:37, Maria Matejka wrote:
TL;DR: Should we test BIRD with Ansible?
Hello!
Dear fellow users and anybody else reading this message, please take a deep breath and read the story of testing BIRD before you shout NO, YES, DEFINITELY MAYBE or whatever else.
Long time ago, when Linux implemented Network Namespaces feature, even before anybody knew anything about "ip netns", Santiago created a simple but quite powerful tool called "netlab" to test BIRD on one machine, using this exact feature. It has grown to quite a decent set of automatic functionality tests which you can see in the bird-tools repository[1] in the "netlab" directory.
After some time, when I got to BIRD development, we wanted to test also BIRD on different platforms. We went to virtualization, setup a decent testing setup with QEMU-KVM and OpenVSwitch and then I tried to run there a simple MPLS setup when the OpenVSwitch suddenly crashed on SegFault. I solved it by using just plain old bridges and veth's.
When I met some friends several days later, one of them being a Linux kernel developer working also on OpenVSwitch, I mentioned these problems, thinking that he may give me a pointer how to solve it. Instead, he interrupted me, saying "hey, shut up, this is an embargoed bug". I had missed that this segfault may be a security problem leading at least to DoS.
This was happening when the OpenStack and LibVirt and others were still in an early phase of development and we couldn't find any suitable solution for us so we just developed something for ourselves.
Anyway, now it is 2022 and when I reach out to almost any user of BIRD, I hear about Ansible. I'm thinking whether it is a good idea to leave our old tooling in a museum and to migrate to Ansible.
Facts: * Our team is familiar with the current tooling, not Ansible. * It's probably easier to learn how to use our tooling than Ansible, provided you have seen neither of them before. * Some of the users are already using BIRD with Ansible. * We don't know much about pros and cons of Ansible.
Proposal: We would retire our old testing setups and instead of that we would upstream a bunch of functionality tests implemented in Ansible.
I have some questions for you:
(1) Are you using Ansible with BIRD? (2) Do you have some testing setup implemented with Ansible? (3) Would you run the upstream functionality tests if they used Ansible? (4) If encountering a bug, would you consider creating an Ansible scenario to replicate that behavior? (5) If contributing, would it be OK for you to create also an appropriate functionality test in Ansible, or rather in our current tooling?
Thank you all for your answers and discussion, please don't throw chairs nor eggs nor tomatoes on each other.
Maria
-- Irene Lalioti Network Engineer Fondation RESTENA 2, avenue de l'Université L-4365 Esch/Alzette Tel: +352 424409 1 Fax: +352 422473 This email may contain information for limited distribution only, please treat accordingly.
participants (5)
-
Alexander Zubkov -
Irene Lalioti -
James Oakley -
Kevin P. Fleming -
Maria Matejka