The queries need to be yielding results based on live data, so dumping the RIB into a file will not work. Regarding using multiple tables and PIPE, can you elaborate? I'm fairly new to BIRD, and I'm not sure I understand how this would work. The objective of this is to provide a scalable way to yield real-time route lookups using BIRD. Imagine a looking glass that needs to support tens (or hundreds) of thousands of queries per second. The expected number of routes fed into BIRD currently sits at 7-10 million, which yields 5,000 QPS per core (with PyBird). The above implementation can be achieved using multiple BIRD processes, with a parent process reflecting its master table to N children/worker processes (using the add-path branch). I'd like to avoid this, as it replicates the memory footprint to each child/worker processes -- not to mention the add-path branch has been segfaulting on me with 'rr client;' enabled. -- Eric Cables On Fri, Aug 23, 2013 at 4:01 PM, Matthew Walster <matthew@walster.org>wrote:
Why not read into multiple tables and then scale out using something like PIPE? Or just dump the RIB into a file and process there? Why do you need so many queries?
Perhaps if we knew what you wanted to achieve we might be able to advise better.
M On 23 Aug 2013 21:19, "Eric Cables" <ecables@gmail.com> wrote:
Yes, this can work, and will achieve my stated goal. My stated goal, however, does not tell the whole story...
I suppose my follow-up to this would be -- Can BIRD run in a shadow/read-only mode, allowing a primary BIRD process to read/write data into memory, but multiple BIRD processes to read the primary's memory space, each with a unique control socket for direct queries?
The above is meant to serve as an extremely scalable system for querying BIRD directly, while distributing queries across multiple read-only workers; at the same time not duplicating the memory space of the parent BIRD process for each worker. As it stands, the BIRD memory space I am trying to query consumes about 1.1GB, and yields 5,000 queries per second before a core reaches 99% CPU. I would like to scale this out, and achieve (5,000 * n) QPS without also consuming (1.1GB * n) memory.
Thoughts?
-- Eric Cables
On Fri, Aug 23, 2013 at 7:11 AM, Ondrej Zajicek <santiago@crfreenet.org>wrote:
On Thu, Aug 22, 2013 at 06:05:01PM -0700, Eric Cables wrote:
Is it possible to run BIRD in a speaker-only only mode, such that it does not bind to a local TCP socket, for the BGP protocol? While i can bind BIRD to different TCP ports at start-up, many vendors do not support specifying a destination port other than tcp/179.
There is no option for this, but you could either bind BIRD instances to a different port or to a different IP (listen option). If you bind BIRD to a different port, you don't need to configure neighbors - outgoing connections will be still to tcp/179.
-- Elen sila lumenn' omentielvo
Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAlIXbY0ACgkQw1GB2RHercPhPgCdGqQeerR0uZxkUmW7p2MANkx/ V58AniOkTc2pbosmnokAQBVvJi/d/KmJ =tp7d -----END PGP SIGNATURE-----