Memory Usage of BIRD
Hi Folks, I just made a little experiment: * let BIRD peer with a router that has Full Feed * letting BIRD eat Full Feed (and tweet) - memory footprint ~ 130 MB BTW (64 Bit Linux) * shutting the session down on the router-side * BIRD's memory consumption decreased to 128 MB, but was never freed again. So now I don't know if this is 'by design' (routes are marked dirty and overwritten on next usage) or a bug... Thanks for any replies; Wolfgang -- www.vix.at | www.aco.net wh@univie.ac.at | WH844-RIPE Vienna University Computer Center
On Fri, Sep 11, 2009 at 10:14:04AM +0200, Wolfgang Hennerbichler wrote:
Hi Folks,
I just made a little experiment: * let BIRD peer with a router that has Full Feed * letting BIRD eat Full Feed (and tweet) - memory footprint ~ 130 MB BTW (64 Bit Linux) * shutting the session down on the router-side * BIRD's memory consumption decreased to 128 MB, but was never freed again.
This is probably caused by memory fragmentation and is otherwise harmless. You can try that if you shut down the session and after a while shut up the session, the memory footprint shouldn't be much larger than initially (memory that was freed by BIRD but not released to OS is reused). -- Elen sila lumenn' omentielvo Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
On Sep 11, 2009, at 11:56 , Ondrej Zajicek wrote:
On Fri, Sep 11, 2009 at 10:14:04AM +0200, Wolfgang Hennerbichler wrote:
Hi Folks,
I just made a little experiment: * let BIRD peer with a router that has Full Feed * letting BIRD eat Full Feed (and tweet) - memory footprint ~ 130 MB BTW (64 Bit Linux) * shutting the session down on the router-side * BIRD's memory consumption decreased to 128 MB, but was never freed again.
This is probably caused by memory fragmentation and is otherwise harmless. You can try that if you shut down the session and after a while shut up the session, the memory footprint shouldn't be much larger than initially (memory that was freed by BIRD but not released to OS is reused).
Hey, diggin up this old thread. I just did a test on that behaviour on Linux and FreeBSD. If sessions go down, the routing tables go empty, and even the protocols are disabled on Linux (64 Bit) BIRD doesn't free the memory to the OS. Or Linux doesn't reclaim the free memory. It is re-used if new routes are sent in again, but bascially it never gets freed to the os. I wrote a c program that would malloc the rest of the memory (swap disabled), so that linux would be forced to give back the unused memory from bird. it didn't. I let sit bird there with disabled and re-enabled protocols over a night and no routes in the routing table, all peerings down, memory usage wouldn't go down. I am in the lucky position to reproduce that problem any time you want me to, so I'm up for tests any time. I can even give you access to the box if you want to. On FreeBSD the behaviour is different, FreeBSD gets back most of BIRD's unused memory. Maybe we can look at this together. Wolfgang
-- Elen sila lumenn' omentielvo
Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
-- www.vix.at | www.aco.net wh@univie.ac.at | WH844-RIPE Vienna University Computer Center
On Thu, Dec 03, 2009 at 12:33:22PM +0100, Wolfgang Hennerbichler wrote:
Hi Folks,
I just made a little experiment: * let BIRD peer with a router that has Full Feed * letting BIRD eat Full Feed (and tweet) - memory footprint ~ 130 MB BTW (64 Bit Linux) * shutting the session down on the router-side * BIRD's memory consumption decreased to 128 MB, but was never freed again.
This is probably caused by memory fragmentation and is otherwise harmless. You can try that if you shut down the session and after a while shut up the session, the memory footprint shouldn't be much larger than initially (memory that was freed by BIRD but not released to OS is reused).
diggin up this old thread. I just did a test on that behaviour on Linux and FreeBSD. If sessions go down, the routing tables go empty, and even the protocols are disabled on Linux (64 Bit) BIRD doesn't free the memory to the OS. Or Linux doesn't reclaim the free memory. ... On FreeBSD the behaviour is different, FreeBSD gets back most of BIRD's unused memory. Maybe we can look at this together.
I would expect that this is not a bug in BIRD, but a behavior of glibc memory management code (cause by memory fragmentation of heap and reluctancy of glibc to return memory to the kernel). There are three levels in memory management - program, libc and kernel. If program frees some memory to libc and libc does not free it to kernel, it counts as a memory consumption of that process (from the external point of view) and kernel cannot reclaim that memory. This problem is general and well-known and might be mitigated by some precautions, but i don' think it is worth the work now. -- Elen sila lumenn' omentielvo Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
On Dec 4, 2009, at 16:56 , Ondrej Zajicek wrote:
On Thu, Dec 03, 2009 at 12:33:22PM +0100, Wolfgang Hennerbichler wrote:
Hi Folks,
I just made a little experiment: * let BIRD peer with a router that has Full Feed * letting BIRD eat Full Feed (and tweet) - memory footprint ~ 130 MB BTW (64 Bit Linux) * shutting the session down on the router-side * BIRD's memory consumption decreased to 128 MB, but was never freed again.
This is probably caused by memory fragmentation and is otherwise harmless. You can try that if you shut down the session and after a while shut up the session, the memory footprint shouldn't be much larger than initially (memory that was freed by BIRD but not released to OS is reused).
diggin up this old thread. I just did a test on that behaviour on Linux and FreeBSD. If sessions go down, the routing tables go empty, and even the protocols are disabled on Linux (64 Bit) BIRD doesn't free the memory to the OS. Or Linux doesn't reclaim the free memory. ... On FreeBSD the behaviour is different, FreeBSD gets back most of BIRD's unused memory. Maybe we can look at this together.
I would expect that this is not a bug in BIRD, but a behavior of glibc memory management code (cause by memory fragmentation of heap and reluctancy of glibc to return memory to the kernel). There are three levels in memory management - program, libc and kernel. If program frees some memory to libc and libc does not free it to kernel, it counts as a memory consumption of that process (from the external point of view) and kernel cannot reclaim that memory. This problem is general and well-known and might be mitigated by some precautions, but i don' think it is worth the work now.
Thinking about this a little more this really isn't work the worth - we have dedicated boxes and memory for bird, so actually if it ever consumes too much memory we're screwed anyway. Thanks for your reply; Wolfgang
-- Elen sila lumenn' omentielvo
Ondrej 'SanTiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
-- www.vix.at | www.aco.net wh@univie.ac.at | WH844-RIPE Vienna University Computer Center
participants (2)
-
Ondrej Zajicek -
Wolfgang Hennerbichler