Hi i have following problem - the kernel protocol ignores routes with RTPROT_KERNEL source. Is there some particular reason why this is so? Why am i asking? I use BIRD in linux container orchestration context (Kubernetes) in conjunction with cilium project (https://cilium.io/). Lately they discovered, that systemd likes to mess with their routes and did a change to insert their routes with RTPROT_KERNEL source flag. This way, systemd doesn't try to mess with their routes. Side effect of this is that my installation got broken as BIRD ignores such routes. Now i don't know where to start - either they are wrong with using RTPROT_KERNEL flag in the first place, or BIRD is wrong by ignoring such routes. So i am trying here first. Also there was some discussion in the past https://bird.network.cz/pipermail/bird-users/2013-August/008519.html, which just died. Do anybody have some suggestions? Thank You Pavel Sorejs
Hello! On 8/23/23 14:12, Pavel Šorejs via Bird-users wrote:
i have following problem - the kernel protocol ignores routes with RTPROT_KERNEL source. Is there some particular reason why this is so? IIRC it simply was there in the beginning and nobody did any change. It should be possible to implement it cleanly in BIRD 3 (now in alpha stage), or kinda messily in BIRD 2 (but the current implementation is messy anyway so who cares).
Why am i asking? I use BIRD in linux container orchestration context (Kubernetes) in conjunction with cilium project (https://cilium.io/). Lately they discovered, that systemd likes to mess with their routes and did a change to insert their routes with RTPROT_KERNEL source flag. This way, systemd doesn't try to mess with their routes. Side effect of this is that my installation got broken as BIRD ignores such routes. Now i don't know where to start - either they are wrong with using RTPROT_KERNEL flag in the first place, or BIRD is wrong by ignoring such routes. So i am trying here first.
Probably everybody is wrong in some way. BIRD should have an option to load also RTPROT_KERNEL routes and mark them as flags appropriately. Cilium should use their own flag (and define it in /etc/iproute2/rt_protos). SystemD should not mess with routes unless configured so.
Do anybody have some suggestions?
You may, e.g.: * send us a messy patch for BIRD 2 (based on branch master) * send us a nicer patch for BIRD 3 (based on tag v3.0-alpha2) * update Cilium to insert their routes with a custom flag, like BIRD does – but not RTPROT_BIRD, please * fork Cilium to use BIRD instead of GoBGP * open a support request at bird-support@network.cz to pay us to do something of the above ;-) Best Regards, Maria Matejka -- Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o.
Hi I would like to implement a patch, now if to BIRD 2 or BIRD 3 - depends on how "alpha" is "alpha". I would like to get some guidance - you will probably want it configurable right? So the default behavior would be the current one (ignore) and optionally it can be turned on. Can you suggest how would would you like it to be done in .conf file? Single flag like `learn;`? Is so, can you suggest name of such flag? "and mark them as flags appropriately" - by this you mean to mark it in `pflags` parameter? Now the source for KRT is not stored anywhere. Sorry for so many questions Thank you Pavel Sorejs On 8/23/23 14:53, Maria Matejka via Bird-users wrote:
Hello!
On 8/23/23 14:12, Pavel Šorejs via Bird-users wrote:
i have following problem - the kernel protocol ignores routes with RTPROT_KERNEL source. Is there some particular reason why this is so? IIRC it simply was there in the beginning and nobody did any change. It should be possible to implement it cleanly in BIRD 3 (now in alpha stage), or kinda messily in BIRD 2 (but the current implementation is messy anyway so who cares).
Why am i asking? I use BIRD in linux container orchestration context (Kubernetes) in conjunction with cilium project (https://cilium.io/). Lately they discovered, that systemd likes to mess with their routes and did a change to insert their routes with RTPROT_KERNEL source flag. This way, systemd doesn't try to mess with their routes. Side effect of this is that my installation got broken as BIRD ignores such routes. Now i don't know where to start - either they are wrong with using RTPROT_KERNEL flag in the first place, or BIRD is wrong by ignoring such routes. So i am trying here first.
Probably everybody is wrong in some way.
BIRD should have an option to load also RTPROT_KERNEL routes and mark them as flags appropriately.
Cilium should use their own flag (and define it in /etc/iproute2/rt_protos).
SystemD should not mess with routes unless configured so.
Do anybody have some suggestions?
You may, e.g.:
* send us a messy patch for BIRD 2 (based on branch master) * send us a nicer patch for BIRD 3 (based on tag v3.0-alpha2) * update Cilium to insert their routes with a custom flag, like BIRD does – but not RTPROT_BIRD, please * fork Cilium to use BIRD instead of GoBGP * open a support request at bird-support@network.cz to pay us to do something of the above ;-)
Best Regards, Maria Matejka
-- Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o.
Here is first version - based on master Pavel --- doc/bird.sgml | 11 ++++++----- sysdep/linux/netlink.c | 2 +- sysdep/unix/krt.Y | 7 ++++++- sysdep/unix/krt.c | 15 +++++++++++---- sysdep/unix/krt.h | 4 ++++ 5 files changed, 28 insertions(+), 11 deletions(-) diff --git a/doc/bird.sgml b/doc/bird.sgml index 29e12b7a..af87d5dc 100644 --- a/doc/bird.sgml +++ b/doc/bird.sgml @@ -3454,9 +3454,8 @@ on the <cf/learn/ switch, such routes are either ignored or accepted to our table). <p>Note that routes created by OS kernel itself, namely direct routes -representing IP subnets of associated interfaces, are not imported even with -<cf/learn/ enabled. You can use <ref id="direct" name="Direct protocol"> to -generate these direct routes. +representing IP subnets of associated interfaces, are imported only with +<cf/learn all/ enabled. <p>If your OS supports only a single routing table, you can configure only one instance of the Kernel protocol. If it supports multiple tables (in order to @@ -3487,10 +3486,12 @@ channels. Time in seconds between two consecutive scans of the kernel routing table. - <tag><label id="krt-learn">learn <m/switch/</tag> + <tag><label id="krt-learn">learn [<m/all/]</tag> Enable learning of routes added to the kernel routing tables by other routing daemons or by the system administrator. This is possible only on - systems which support identification of route authorship. + systems which support identification of route authorship. By default, routes + created by kernel (marked as "proto kernel") are not imported. Use <m/all/ + option to import these routes. <tag><label id="krt-kernel-table">kernel table <m/number/</tag> Select which kernel table should this particular instance of the Kernel diff --git a/sysdep/linux/netlink.c b/sysdep/linux/netlink.c index 1af78766..29446cab 100644 --- a/sysdep/linux/netlink.c +++ b/sysdep/linux/netlink.c @@ -1598,7 +1598,7 @@ nl_parse_route(struct nl_parse_state *s, struct nlmsghdr *h) case RTPROT_KERNEL: krt_src = KRT_SRC_KERNEL; - return; + break; case RTPROT_BIRD: if (!s->scan) diff --git a/sysdep/unix/krt.Y b/sysdep/unix/krt.Y index 95b54d65..f3eb1393 100644 --- a/sysdep/unix/krt.Y +++ b/sysdep/unix/krt.Y @@ -32,6 +32,7 @@ CF_DECLS CF_KEYWORDS(KERNEL, PERSIST, SCAN, TIME, LEARN, DEVICE, ROUTES, GRACEFUL, RESTART, KRT_SOURCE, KRT_METRIC, MERGE, PATHS) CF_KEYWORDS(INTERFACE, PREFERRED) +%type <i> kern_learn %type <i> kern_mp_limit CF_GRAMMAR @@ -48,6 +49,10 @@ kern_proto_start: proto_start KERNEL { kern_proto: kern_proto_start proto_name '{' ; kern_proto: kern_proto kern_item ';' ; +kern_learn: + bool { $$ = $1 ? KRT_LEARN_SOME : KRT_LEARN_NONE; } + | ALL { $$ = KRT_LEARN_ALL; } + kern_mp_limit: /* empty */ { $$ = KRT_DEFAULT_ECMP_LIMIT; } | LIMIT expr { $$ = $2; if (($2 <= 0) || ($2 > 255)) cf_error("Merge paths limit must be in range 1-255"); } @@ -61,7 +66,7 @@ kern_item: /* Scan time of 0 means scan on startup only */ THIS_KRT->scan_time = $3 S_; } - | LEARN bool { + | LEARN kern_learn { THIS_KRT->learn = $2; #ifndef KRT_ALLOW_LEARN if ($2) diff --git a/sysdep/unix/krt.c b/sysdep/unix/krt.c index 9f95247f..3288fd0c 100644 --- a/sysdep/unix/krt.c +++ b/sysdep/unix/krt.c @@ -638,13 +638,14 @@ krt_got_route(struct krt_proto *p, rte *e, s8 src) #ifdef KRT_ALLOW_LEARN switch (src) - { - case KRT_SRC_KERNEL: - goto ignore; - + { case KRT_SRC_REDIRECT: goto delete; + case KRT_SRC_KERNEL: + if (KRT_CF->learn != KRT_LEARN_ALL) + goto ignore; + // fall through case KRT_SRC_ALIEN: if (KRT_CF->learn) krt_learn_scan(p, e); @@ -780,6 +781,12 @@ krt_got_route_async(struct krt_proto *p, rte *e, int new, s8 src) break; #ifdef KRT_ALLOW_LEARN + case KRT_SRC_KERNEL: + if (KRT_CF->learn == KRT_LEARN_ALL) + { + krt_learn_async(p, e, new); + } + break; case KRT_SRC_ALIEN: if (KRT_CF->learn) { diff --git a/sysdep/unix/krt.h b/sysdep/unix/krt.h index 18a206e6..694ebd34 100644 --- a/sysdep/unix/krt.h +++ b/sysdep/unix/krt.h @@ -27,6 +27,10 @@ struct kif_proto; #define KRT_REF_SEEN 0x1 /* Seen in table */ #define KRT_REF_BEST 0x2 /* Best in table */ +#define KRT_LEARN_NONE 0 /* Don't learn */ +#define KRT_LEARN_SOME 1 /* Learn some routes (excluding RTPROT_KERNEL) */ +#define KRT_LEARN_ALL 2 /* Learn all routes */ + /* Whenever we recognize our own routes, we allow learing of foreign routes */ #ifdef CONFIG_SELF_CONSCIOUS -- 2.41.0 On 8/23/23 18:07, Pavel Šorejs via Bird-users wrote:
Hi
I would like to implement a patch, now if to BIRD 2 or BIRD 3 - depends on how "alpha" is "alpha".
I would like to get some guidance - you will probably want it configurable right? So the default behavior would be the current one (ignore) and optionally it can be turned on. Can you suggest how would would you like it to be done in .conf file? Single flag like `learn;`? Is so, can you suggest name of such flag?
"and mark them as flags appropriately" - by this you mean to mark it in `pflags` parameter? Now the source for KRT is not stored anywhere.
Sorry for so many questions
Thank you
Pavel Sorejs
On 8/23/23 14:53, Maria Matejka via Bird-users wrote:
Hello!
On 8/23/23 14:12, Pavel Šorejs via Bird-users wrote:
i have following problem - the kernel protocol ignores routes with RTPROT_KERNEL source. Is there some particular reason why this is so? IIRC it simply was there in the beginning and nobody did any change. It should be possible to implement it cleanly in BIRD 3 (now in alpha stage), or kinda messily in BIRD 2 (but the current implementation is messy anyway so who cares).
Why am i asking? I use BIRD in linux container orchestration context (Kubernetes) in conjunction with cilium project (https://cilium.io/). Lately they discovered, that systemd likes to mess with their routes and did a change to insert their routes with RTPROT_KERNEL source flag. This way, systemd doesn't try to mess with their routes. Side effect of this is that my installation got broken as BIRD ignores such routes. Now i don't know where to start - either they are wrong with using RTPROT_KERNEL flag in the first place, or BIRD is wrong by ignoring such routes. So i am trying here first.
Probably everybody is wrong in some way.
BIRD should have an option to load also RTPROT_KERNEL routes and mark them as flags appropriately.
Cilium should use their own flag (and define it in /etc/iproute2/rt_protos).
SystemD should not mess with routes unless configured so.
Do anybody have some suggestions?
You may, e.g.:
* send us a messy patch for BIRD 2 (based on branch master) * send us a nicer patch for BIRD 3 (based on tag v3.0-alpha2) * update Cilium to insert their routes with a custom flag, like BIRD does – but not RTPROT_BIRD, please * fork Cilium to use BIRD instead of GoBGP * open a support request at bird-support@network.cz to pay us to do something of the above ;-)
Best Regards, Maria Matejka
-- Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o.
Hello! On first sight, this looks good. Gonna do some checks and tests and let you know whether anything more is needed from you. Thank you for your patch! Maria On 8/24/23 01:38, Pavel Šorejs via Bird-users wrote:
Here is first version - based on master
Pavel
--- doc/bird.sgml | 11 ++++++----- sysdep/linux/netlink.c | 2 +- sysdep/unix/krt.Y | 7 ++++++- sysdep/unix/krt.c | 15 +++++++++++---- sysdep/unix/krt.h | 4 ++++ 5 files changed, 28 insertions(+), 11 deletions(-)
diff --git a/doc/bird.sgml b/doc/bird.sgml index 29e12b7a..af87d5dc 100644 --- a/doc/bird.sgml +++ b/doc/bird.sgml @@ -3454,9 +3454,8 @@ on the <cf/learn/ switch, such routes are either ignored or accepted to our table).
<p>Note that routes created by OS kernel itself, namely direct routes -representing IP subnets of associated interfaces, are not imported even with -<cf/learn/ enabled. You can use <ref id="direct" name="Direct protocol"> to -generate these direct routes. +representing IP subnets of associated interfaces, are imported only with +<cf/learn all/ enabled.
<p>If your OS supports only a single routing table, you can configure only one instance of the Kernel protocol. If it supports multiple tables (in order to @@ -3487,10 +3486,12 @@ channels. Time in seconds between two consecutive scans of the kernel routing table.
- <tag><label id="krt-learn">learn <m/switch/</tag> + <tag><label id="krt-learn">learn [<m/all/]</tag> Enable learning of routes added to the kernel routing tables by other routing daemons or by the system administrator. This is possible only on - systems which support identification of route authorship. + systems which support identification of route authorship. By default, routes + created by kernel (marked as "proto kernel") are not imported. Use <m/all/ + option to import these routes.
<tag><label id="krt-kernel-table">kernel table <m/number/</tag> Select which kernel table should this particular instance of the Kernel diff --git a/sysdep/linux/netlink.c b/sysdep/linux/netlink.c index 1af78766..29446cab 100644 --- a/sysdep/linux/netlink.c +++ b/sysdep/linux/netlink.c @@ -1598,7 +1598,7 @@ nl_parse_route(struct nl_parse_state *s, struct nlmsghdr *h)
case RTPROT_KERNEL: krt_src = KRT_SRC_KERNEL; - return; + break;
case RTPROT_BIRD: if (!s->scan) diff --git a/sysdep/unix/krt.Y b/sysdep/unix/krt.Y index 95b54d65..f3eb1393 100644 --- a/sysdep/unix/krt.Y +++ b/sysdep/unix/krt.Y @@ -32,6 +32,7 @@ CF_DECLS CF_KEYWORDS(KERNEL, PERSIST, SCAN, TIME, LEARN, DEVICE, ROUTES, GRACEFUL, RESTART, KRT_SOURCE, KRT_METRIC, MERGE, PATHS) CF_KEYWORDS(INTERFACE, PREFERRED)
+%type <i> kern_learn %type <i> kern_mp_limit
CF_GRAMMAR @@ -48,6 +49,10 @@ kern_proto_start: proto_start KERNEL { kern_proto: kern_proto_start proto_name '{' ; kern_proto: kern_proto kern_item ';' ;
+kern_learn: + bool { $$ = $1 ? KRT_LEARN_SOME : KRT_LEARN_NONE; } + | ALL { $$ = KRT_LEARN_ALL; } + kern_mp_limit: /* empty */ { $$ = KRT_DEFAULT_ECMP_LIMIT; } | LIMIT expr { $$ = $2; if (($2 <= 0) || ($2 > 255)) cf_error("Merge paths limit must be in range 1-255"); } @@ -61,7 +66,7 @@ kern_item: /* Scan time of 0 means scan on startup only */ THIS_KRT->scan_time = $3 S_; } - | LEARN bool { + | LEARN kern_learn { THIS_KRT->learn = $2; #ifndef KRT_ALLOW_LEARN if ($2) diff --git a/sysdep/unix/krt.c b/sysdep/unix/krt.c index 9f95247f..3288fd0c 100644 --- a/sysdep/unix/krt.c +++ b/sysdep/unix/krt.c @@ -638,13 +638,14 @@ krt_got_route(struct krt_proto *p, rte *e, s8 src)
#ifdef KRT_ALLOW_LEARN switch (src) - { - case KRT_SRC_KERNEL: - goto ignore; - + { case KRT_SRC_REDIRECT: goto delete;
+ case KRT_SRC_KERNEL: + if (KRT_CF->learn != KRT_LEARN_ALL) + goto ignore; + // fall through case KRT_SRC_ALIEN: if (KRT_CF->learn) krt_learn_scan(p, e); @@ -780,6 +781,12 @@ krt_got_route_async(struct krt_proto *p, rte *e, int new, s8 src) break;
#ifdef KRT_ALLOW_LEARN + case KRT_SRC_KERNEL: + if (KRT_CF->learn == KRT_LEARN_ALL) + { + krt_learn_async(p, e, new); + } + break; case KRT_SRC_ALIEN: if (KRT_CF->learn) { diff --git a/sysdep/unix/krt.h b/sysdep/unix/krt.h index 18a206e6..694ebd34 100644 --- a/sysdep/unix/krt.h +++ b/sysdep/unix/krt.h @@ -27,6 +27,10 @@ struct kif_proto; #define KRT_REF_SEEN 0x1 /* Seen in table */ #define KRT_REF_BEST 0x2 /* Best in table */
+#define KRT_LEARN_NONE 0 /* Don't learn */ +#define KRT_LEARN_SOME 1 /* Learn some routes (excluding RTPROT_KERNEL) */ +#define KRT_LEARN_ALL 2 /* Learn all routes */ + /* Whenever we recognize our own routes, we allow learing of foreign routes */
#ifdef CONFIG_SELF_CONSCIOUS
-- Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o.
On Thu, Aug 24, 2023 at 01:38:12AM +0200, Pavel Šorejs via Bird-users wrote:
Here is first version - based on master
Hi Seems like your patch is mangled (likely lines wrapped by e-mail client). Could you send it as an attachment? -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
Here it is On 8/29/23 14:02, Ondrej Zajicek wrote:
On Thu, Aug 24, 2023 at 01:38:12AM +0200, Pavel Šorejs via Bird-users wrote:
Here is first version - based on master Hi
Seems like your patch is mangled (likely lines wrapped by e-mail client). Could you send it as an attachment?
On Tue, Aug 29, 2023 at 04:16:16PM +0200, Pavel Šorejs via Bird-users wrote:
Here it is
On 8/29/23 14:02, Ondrej Zajicek wrote:
On Thu, Aug 24, 2023 at 01:38:12AM +0200, Pavel Šorejs via Bird-users wrote:
Here is first version - based on master Hi
Seems like your patch is mangled (likely lines wrapped by e-mail client). Could you send it as an attachment?
Hi Merged and included in the v2.14 -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
participants (3)
-
Maria Matejka -
Ondrej Zajicek -
Pavel Šorejs