Calling Bird2 from Python script fails
Hi all, I am trying to automate Bird2 configuration with Python3 and I faced this strange issue: * If I generate a part of the config which is included in the main config file and then I call “birdc configure” via os.system() function, the new configuration is ignored. * If I generate a full config and I call “birdc configure” via os.system() function, I get the strange error “bird2_control.conf:1:1 No protocol is specified in the config file” Which is strange because the first line of my config is a big comment that can be ignored. * If myself execute ““birdc configure” without touching/changing anything in the config, it is accepted and executed immediately without issues !!! Things I tried so far to overcome the issue: * I used subprocess.run() but didn’t have any luck * Tried to use a 2s sleep between writing the new config and reloading bird2, didn’t work * I used python2 instead of python3 but still didn’t have any luck * I executed “birdc configure” via a BASH script, still didn’t work * User and file permissions checked, now problem. Anyone has experienced this weird problem before and has a suggestion to overcome it? It seems to be a birdc issue and doesn’t like to be called from scripts. FYI, I use Bird 2.0.10 on CentOS 7 and Python 3.6.8 Kind Regards Stavros
Hi Stavros, Could you share any of the Python3 scripts you used? This may help in debugging / reproducing your issue. A few thoughts in the meantime: - are there any other BIRD processes running on the system involved? - if so, are you connecting to the correct control socket? - are you sure all buffers (i.e. for the configuration file) have been flushed before calling birdc configure? - did you capture any output from birdc reconfigure? (Bird 2.0.10 on Ubuntu 20.04, though compiled from source, says "Reading configuration from /etc/bird.conf") - if so, did the correct configuration filename come up? Best regards, Gerdriaan Mulder On 24/11/2022 14:09, Stavros Konstantaras via Bird-users wrote:
Hi all,
I am trying to automate Bird2 configuration with Python3 and I faced this strange issue:
* If I generate a part of the config which is included in the main config file and then I call “birdc configure” via os.system() function, the new configuration is ignored.
* If I generate a full config and I call “birdc configure” via os.system() function, I get the strange error “bird2_control.conf:1:1 No protocol is specified in the config file” Which is strange because the first line of my config is a big comment that can be ignored.
* If myself execute ““birdc configure” without touching/changing anything in the config, it is accepted and executed immediately without issues !!!
Things I tried so far to overcome the issue:
* I used subprocess.run() but didn’t have any luck * Tried to use a 2s sleep between writing the new config and reloading bird2, didn’t work * I used python2 instead of python3 but still didn’t have any luck * I executed “birdc configure” via a BASH script, still didn’t work * User and file permissions checked, now problem.
Anyone has experienced this weird problem before and has a suggestion to overcome it? It seems to be a birdc issue and doesn’t like to be called from scripts.
FYI, I use Bird 2.0.10 on CentOS 7 and Python 3.6.8
Kind Regards
Stavros
Hi Gerdriaan, Thank you for your interest and asking those questions. Finally I found the issue and luckily was not a Bird2's fault but some weird python3 behavior. For the record: - The following does not work: my_file.write(template.render(nawas_routes).encode('ascii', 'ignore').decode('ascii')) - The following works: new_data = template.render(nawas_routes).encode('ascii', 'ignore').decode('ascii') my_file.write(new_data) Although the first expression is (syntactically and logically) correct, by the time the "birdc configure" was called/executed the config file was empty. Even if I put a 1-2-3 second time sleep the file was still empty and thus, Bird2 had nothing to read. Once, the Python3 script completes successfully and exits to the CLI, you could see however the new data in the "my_file. Thus, it was quite confusing to understand and find the issue. I need to reach some more experienced python developers to understand why, but I suspect that Python3 puts the first method in a separate thread and leaves it execute separately while the rest of the python3 interpreter moves to the rest of the script. When I switched the code to the second method it worked immediately. I hope this finding can help future Bird users with their automation scripts. Kind Regards Stavros Konstantaras | Sr. Network Engineer | AMS-IX Frederiksplein 42, 1017 XN Amsterdam, The Netherlands M +31 (0) 620 89 51 04 ams-ix.net <http://ams-ix.net> On 26/11/2022, 15:02, "Gerdriaan Mulder" <gmulder+birdcz@freedom.nl> wrote: Hi Stavros, Could you share any of the Python3 scripts you used? This may help in debugging / reproducing your issue. A few thoughts in the meantime: - are there any other BIRD processes running on the system involved? - if so, are you connecting to the correct control socket? - are you sure all buffers (i.e. for the configuration file) have been flushed before calling birdc configure? - did you capture any output from birdc reconfigure? (Bird 2.0.10 on Ubuntu 20.04, though compiled from source, says "Reading configuration from /etc/bird.conf") - if so, did the correct configuration filename come up? Best regards, Gerdriaan Mulder On 24/11/2022 14:09, Stavros Konstantaras via Bird-users wrote: > Hi all, > > I am trying to automate Bird2 configuration with Python3 and I faced > this strange issue: > > * If I generate a part of the config which is included in the main > config file and then I call “birdc configure” via os.system() > function, the new configuration is ignored. > > * If I generate a full config and I call “birdc configure” via > os.system() function, I get the strange error > “bird2_control.conf:1:1 No protocol is specified in the config file” > Which is strange because the first line of my config is a big > comment that can be ignored. > > * If myself execute ““birdc configure” without touching/changing > anything in the config, it is accepted and executed immediately > without issues !!! > > Things I tried so far to overcome the issue: > > * I used subprocess.run() but didn’t have any luck > * Tried to use a 2s sleep between writing the new config and reloading > bird2, didn’t work > * I used python2 instead of python3 but still didn’t have any luck > * I executed “birdc configure” via a BASH script, still didn’t work > * User and file permissions checked, now problem. > > Anyone has experienced this weird problem before and has a suggestion to > overcome it? It seems to be a birdc issue and doesn’t like to be called > from scripts. > > FYI, I use Bird 2.0.10 on CentOS 7 and Python 3.6.8 > > Kind Regards > > Stavros >
Stavros Konstantaras via Bird-users <bird-users@trubka.network.cz> writes:
Hi Gerdriaan,
Thank you for your interest and asking those questions. Finally I found the issue and luckily was not a Bird2's fault but some weird python3 behavior. For the record: - The following does not work: my_file.write(template.render(nawas_routes).encode('ascii', 'ignore').decode('ascii'))
- The following works: new_data = template.render(nawas_routes).encode('ascii', 'ignore').decode('ascii') my_file.write(new_data)
Although the first expression is (syntactically and logically) correct, by the time the "birdc configure" was called/executed the config file was empty. Even if I put a 1-2-3 second time sleep the file was still empty and thus, Bird2 had nothing to read. Once, the Python3 script completes successfully and exits to the CLI, you could see however the new data in the "my_file. Thus, it was quite confusing to understand and find the issue.
Are you calling my_file.close() anywhere? Otherwise, the data will stay in a buffer until the file *does* get closed, which will happen when the Python script exits... -Toke
Hi Toke, Yes I do, I didn't share the whole code snippet but my_file.close() was immediately below that line. __ And after closing the output file I proceed on calling subprocess with the "birdc configure" command. Best Regards Stavros On 28/11/2022, 12:54, "Toke Høiland-Jørgensen" <toke@toke.dk> wrote: Stavros Konstantaras via Bird-users <bird-users@trubka.network.cz> writes: > Hi Gerdriaan, > > Thank you for your interest and asking those questions. Finally I found the issue and luckily was not a Bird2's fault but some weird python3 behavior. For the record: > - The following does not work: > my_file.write(template.render(nawas_routes).encode('ascii', 'ignore').decode('ascii')) > > - The following works: > new_data = template.render(nawas_routes).encode('ascii', 'ignore').decode('ascii') > my_file.write(new_data) > > > Although the first expression is (syntactically and logically) > correct, by the time the "birdc configure" was called/executed the > config file was empty. Even if I put a 1-2-3 second time sleep the > file was still empty and thus, Bird2 had nothing to read. Once, the > Python3 script completes successfully and exits to the CLI, you could > see however the new data in the "my_file. Thus, it was quite confusing > to understand and find the issue. Are you calling my_file.close() anywhere? Otherwise, the data will stay in a buffer until the file *does* get closed, which will happen when the Python script exits... -Toke
Hi Stavros, On Mon, Nov 28, 2022 at 03:08:59PM +0000, Stavros Konstantaras via Bird-users wrote:
And after closing the output file I proceed on calling subprocess with the "birdc configure" command.
You might consider sending the command directly to the control socket, e.g. (no error handling!): ```python import socket with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: sock.connect('/run/bird/bird.ctl') sock.send(b'configure\n') # Assumption your config is under /etc/bird.conf assert sock.recv(4096) == b'0002-Reading configuration from /etc/bird.conf\n0003 Reconfigured\n' ``` Best Regards Inrin
Hello!
On Mon, Nov 28, 2022 at 03:08:59PM +0000, Stavros Konstantaras via Bird-users wrote:
And after closing the output file I proceed on calling subprocess with the "birdc configure" command.
You might consider sending the command directly to the control socket, e.g. (no error handling!):
```python import socket
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: sock.connect('/run/bird/bird.ctl') sock.send(b'configure\n') # Assumption your config is under /etc/bird.conf assert sock.recv(4096) == b'0002-Reading configuration from /etc/bird.conf\n0003 Reconfigured\n' ```
This is horrible, yet understandable. I hope we manage to include some machine-friendly control socket implementation in BIRD 3. It's gonna be based on CBOR format, hopefully distributed also with client Python libs, and if nobody in our team gets too angry with me for writing such a code, maybe also with an Ansible module allowing to write a nice-looking YAML definition. Take this as my personal hopes … yet if you wanna say ±1 or whatever else to this, feel free to respond. Maria
Anything more automation-friendly would be welcome! A better socket interface, Python lib, Ansible module, all great. On Sat, Dec 3, 2022 at 9:50 AM Maria Matejka via Bird-users < bird-users@trubka.network.cz> wrote:
Hello!
On Mon, Nov 28, 2022 at 03:08:59PM +0000, Stavros Konstantaras via Bird-users wrote:
And after closing the output file I proceed on calling subprocess with the "birdc configure" command.
You might consider sending the command directly to the control socket, e.g. (no error handling!):
```python import socket
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: sock.connect('/run/bird/bird.ctl') sock.send(b'configure\n') # Assumption your config is under /etc/bird.conf assert sock.recv(4096) == b'0002-Reading configuration from /etc/bird.conf\n0003 Reconfigured\n' ```
This is horrible, yet understandable.
I hope we manage to include some machine-friendly control socket implementation in BIRD 3. It's gonna be based on CBOR format, hopefully distributed also with client Python libs, and if nobody in our team gets too angry with me for writing such a code, maybe also with an Ansible module allowing to write a nice-looking YAML definition.
Take this as my personal hopes … yet if you wanna say ±1 or whatever else to this, feel free to respond.
Maria
participants (6)
-
Gerdriaan Mulder -
Inrin -
Maria Matejka -
Ross Tajvar -
Stavros Konstantaras -
Toke Høiland-Jørgensen