Hello Maria,

We hit this one too, on routers that hold a multihop upstream session,
so here is a reproducer for it, attached as
bird-repro-rte-is-valid.sh. It came out smaller than your sketch: one
upstream rather than ten, no `merge paths`, and no reconfiguration --
the trigger is `birdc disable`/`enable` of a static protocol. It uses
two network namespaces, needs only root, iproute2 and bird3, and cleans
up after itself.

Three conditions look each necessary and together sufficient:

1. the channel keeps filtered routes -- `import table on`, or `import keep filtered`
2. the next hop is resolved recursively -- i.e. a multihop session
3. the route resolving it changes -- appearing or disappearing, either direction

The script runs the same scenario five times, removing one ingredient
at a time. On Debian's bird3 3.3.2-1~bpo13+1:

BIRD version 3.3.2
  import table, multihop, remove       filtered=1  *** ABORTED ***
  keep filtered, multihop, remove       filtered=1  *** ABORTED ***
  import table, multihop, ADD            filtered=1  *** ABORTED ***
  import table, NOT multihop, rm       filtered=1  survived
  no keep-filtered, multihop, rm          filtered=0  survived

  the assertion, deduplicated over the aborted runs:
  Assertion '!new || rte_is_valid(new)' failed at nest/rt-table.c:1562

On your note about 3.1.x: I believe it will not abort there, although
it may well reach the same state. The assertion is in nest/rt-table.c
on v3.2.0, v3.2.3, v3.3.2 and thread-next, and absent on v3.1.7 and
v3.1.8, which carry `/* Ignore invalid routes */` and
`if (!rte_is_valid(new)) new = NULL;` at that point instead. It looks
introduced by 34a8a2749b1cab415c48b2cbdb05bc8faf345374 ("Table: Optimal
and Any Export refactoring"), which is not reachable from master, is
contained in no v2 tag, and whose earliest release tag is v3.2.0.

One more thing that may narrow it down. With `merge paths` configured,
sysdep/unix/krt.c:911 sets `cc->ra_mode = RA_MERGED`, while the channel
default is RA_OPTIMAL (nest/proto.c:1322) and the assertion is in the
RA_OPTIMAL consumer -- so in Christoph's configuration the kernel
channel should not have been the subscriber that died. His follow-up
message on the list mentions that two of his three upstreams are
multihop, which fits condition 2 above.

The script is standalone and needs nothing beyond iproute2, so it
should adapt into a netlab case without much work.

Thanks,