I must admit, after deploying 1.6.2 to production, I am still seeing more memory usage than expected. I was going to take some time to gather some more stats - but seeing this made me jump early :) Memory usage is still better than it was originally, but I see 2G in some cases, which seems pretty excessive for 2500 routes. The other strange this is, I see a lot of variance across hosts with the same route count, config, and similar process run times. Some node processes are using 12Mg - 16Mg, while some are using up to 2228Mg after just one day. The bird.conf has changed a little, so I'll attach it again here. Madhu - perhaps you should provide your bird.conf as well ? Also - some basic info: # birdc BIRD 1.6.2 ready. bird> bird> show route count 2484 of 2484 routes for 1245 networks bird> show mem BIRD memory usage Routing tables: 2235 MB Route attributes: 87 kB ROA tables: 192 B Protocols: 41 kB Total: 2235 MB If you need any further info - please let me know. Cheers, Just On 11 October 2016 at 02:21, Madhu <informmadhu@gmail.com> wrote:

I have 3000 routes with 64 path ecmp. It is crashing . I don't see the coredump . Is there anyway to fix using trace ?

Madhu

Sent from my iPhone

On Oct 10, 2016, at 6:08 PM, Jonathan Stewart <jonathan.stewart@gmail.com> wrote:

It is not a general issue, many here run hundreds of thousands of routes.

It must be something specific to your environment.

Jonathan

On 10 Oct 2016 4:40 p.m., "Madhu" <informmadhu@gmail.com> wrote:

...

Looks like the problem is still there. I used 1.6.0 and also 1.6.2.

On Mon, Oct 10, 2016 at 10:56 AM, james machado <hvgeekwtrvl@gmail.com> wrote:

...

Madhu check out the following thread. http://bird.network.cz/piperm ail/bird-users/2016-September/010578.html

On Mon, Oct 10, 2016 at 10:02 AM, Madhu <informmadhu@gmail.com> wrote:

...

Hi,

I am seeing a out of memory crash with 3000 routes in BGP. Is this a known issue? I m seeing this issue in 1.6.0 and 1.6.2 .

[ 1098.912753] Call Trace: [ 1098.915481] [<ffffffff81b96521>] dump_stack+0x64/0x82 [ 1098.921225] [<ffffffff81b91a46>] dump_header+0x7f/0x1f1 [ 1098.927163] [<ffffffff810b3e76>] ? put_online_cpus+0x56/0x80 [ 1098.933587] [<ffffffff811110ac>] ? rcu_oom_notify+0xcc/0xf0 [ 1098.939914] [<ffffffff811a2735>] oom_kill_process+0x205/0x360 [ 1098.946435] [<ffffffff814595b5>] ? security_capable_noaudit+0x15/ 0x20 [ 1098.953733] [<ffffffff811a2ee2>] out_of_memory+0x492/0x4d0 [ 1098.959962] [<ffffffff811a8f30>] __alloc_pages_nodemask+0xa00/0xb60 [ 1098.967067] [<ffffffff811eb1fa>] alloc_pages_vma+0x9a/0x160 [ 1098.973393] [<ffffffff811cd5c8>] handle_mm_fault+0xd38/0x1080 [ 1098.979914] [<ffffffff811d4b74>] ? change_protection+0x594/0x850 [ 1098.986726] [<ffffffff8109f5ae>] __do_page_fault+0x19e/0x560 [ 1098.993149] [<ffffffff811d4f81>] ? mprotect_fixup+0x151/0x290 [ 1098.999669] [<ffffffff8109f9a1>] do_page_fault+0x31/0x70 [ 1099.005704] [<ffffffff81ba40e8>] page_fault+0x28/0x30

Madhu

-- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group. References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

It looks like oom-kill from the call trace I think. Cheers, Just On 11 October 2016 at 13:07, Ondrej Zajicek <santiago@crfreenet.org> wrote:

On Mon, Oct 10, 2016 at 06:21:44PM -0700, Madhu wrote:

...

I have 3000 routes with 64 path ecmp. It is crashing . I don't see the coredump . Is there anyway to fix using trace ?

Hi

I would like to confirm, it is BIRD that was killed by out of memory killer and could be just restarted? Or Linux kernel crashed? (You wrote 'the kernel crashed with out of memory issue' in one prior mail.)

In the first case, how much memory BIRD takes after start and before it crashes? How long it takes to crash to crash due to OOM?

...

On Oct 10, 2016, at 6:08 PM, Jonathan Stewart < jonathan.stewart@gmail.com> wrote: It must be something specific to your environment.

It is likely related to 64 path BGP multipath exported to kernel.

-- Elen sila lumenn' omentielvo

Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."

-- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group. References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

Are there any thoughts as to why I still see quite large memory usage, and only on some [ seemingly random ] hosts ? Cheers, Just On 11 October 2016 at 19:05, Madhu <informmadhu@gmail.com> wrote:

Hi everyone,

Now i tested with 16 path and 32 path, I don't see the crash. Below are the memory usage for each.

*16 path*

bird> show memory BIRD memory usage Routing tables: 5109 kB Route attributes: 13 kB ROA tables: 192 B Protocols: 163 kB Total: 5384 kB

*32 path*

bird> show memory BIRD memory usage Routing tables: 10 MB Route attributes: 16 kB ROA tables: 192 B Protocols: 239 kB Total: 10 MB

Now I increased to* 62 path*, I see that* CPU is going high to 100%*, and the memory usage is 0.1%. birdc socket got struck. I was not able to access the socket.

bird> show memory BIRD memory usage Routing tables: 5127 kB Route attributes: 20 kB ROA tables: 192 B Protocols: 382 kB Total: 5641 kB bird> show memory >>. After this it got struck

# top top - 17:47:50 up 1:02, 2 users, load average: 1.35, 1.42, 1.10 Tasks: 163 total, 2 running, 161 sleeping, 0 stopped, 0 zombie %Cpu(s): 29.7 us, 0.5 sy, 0.2 ni, 69.4 id, 0.0 wa, 0.0 hi, 0.2 si, 0.0 st KiB Mem: 8127384 total, 1279304 used, 6848080 free, 36040 buffers KiB Swap: 812736 total, 0 used, 812736 free. 456220 cached Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 8768 bird 20 0 21728 8808 2352 R* 100.0* 0.1 8:12.97 bird

Attached the bird config file.

Madhu

On Tue, Oct 11, 2016 at 5:56 AM, Justin Cattle <j@ocado.com> wrote:

...

It looks like oom-kill from the call trace I think.

Cheers, Just

On 11 October 2016 at 13:07, Ondrej Zajicek <santiago@crfreenet.org> wrote:

...

On Mon, Oct 10, 2016 at 06:21:44PM -0700, Madhu wrote:

...

I have 3000 routes with 64 path ecmp. It is crashing . I don't see the coredump . Is there anyway to fix using trace ?

Hi

I would like to confirm, it is BIRD that was killed by out of memory killer and could be just restarted? Or Linux kernel crashed? (You wrote 'the kernel crashed with out of memory issue' in one prior mail.)

In the first case, how much memory BIRD takes after start and before it crashes? How long it takes to crash to crash due to OOM?

...

On Oct 10, 2016, at 6:08 PM, Jonathan Stewart < jonathan.stewart@gmail.com> wrote: It must be something specific to your environment.

It is likely related to 64 path BGP multipath exported to kernel.

-- Elen sila lumenn' omentielvo

Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."

Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group.

If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses.

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group.

References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

-- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group. References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

Good stuff - once again, please let me know if you want to me test any patching at this end :) Cheers, Just On 12 October 2016 at 13:02, Ondrej Zajicek <santiago@crfreenet.org> wrote:

On Wed, Oct 12, 2016 at 12:09:17PM +0100, Justin Cattle wrote:

...

Are there any thoughts as to why I still see quite large memory usage, and only on some [ seemingly random ] hosts ?

Hi

We are doing some testing and trying to identify the cause of the problem. We found some problems and strange behavior. One of them is excessive memory consumption when kernel export filters are defined.

-- Elen sila lumenn' omentielvo

Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."

-- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group. References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

The patch looks very promising in the lab. All hosts are at around 153kB for 195 routes each. As the patch is quite small, if it remains stable over the next day or so I may consider rolling this one out sooner rather than later. Thanks :) Cheers, Just On 12 October 2016 at 13:27, Jan Matejka <jan.matejka@nic.cz> wrote:

Hi!

Please try the last commit in GIT, branch master. Commit ID 2e7fb11a6e31324151c6db98df2fe26d2d6cffab. Attaching the patch as well.

Thank you both for reporting this issue.

Jan

On 10/12/2016 02:10 PM, Justin Cattle wrote:

...

Good stuff - once again, please let me know if you want to me test any patching at this end :)

Cheers, Just

On 12 October 2016 at 13:02, Ondrej Zajicek <santiago@crfreenet.org <mailto:santiago@crfreenet.org>> wrote:

On Wed, Oct 12, 2016 at 12:09:17PM +0100, Justin Cattle wrote: > Are there any thoughts as to why I still see quite large memory usage, and > only on some [ seemingly random ] hosts ?

Hi

We are doing some testing and trying to identify the cause of the problem. We found some problems and strange behavior. One of them is excessive memory consumption when kernel export filters are defined.

-- Elen sila lumenn' omentielvo

Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org <mailto: santiago@crfreenet.org>) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net <http://wwwkeys.pgp.net>) "To err is human -- to blame it on a computer is even more so."

Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group.

If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses.

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group.

References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

-- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group. References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

Personally - I get the debian source package, then apply the patch to that using quilt, bump the version with my own local identifier, and build the new package. Cheers, Just On 12 October 2016 at 16:19, Madhu <informmadhu@gmail.com> wrote:

How to install this patch?

Madhu

Sent from my iPhone

...

On Oct 12, 2016, at 5:27 AM, Jan Matejka <jan.matejka@nic.cz> wrote:

Hi!

Please try the last commit in GIT, branch master. Commit ID 2e7fb11a6e31324151c6db98df2fe26d2d6cffab. Attaching the patch as well.

Thank you both for reporting this issue.

Jan

...

On 10/12/2016 02:10 PM, Justin Cattle wrote: Good stuff - once again, please let me know if you want to me test any patching at this end :)

Cheers, Just

On 12 October 2016 at 13:02, Ondrej Zajicek <santiago@crfreenet.org <mailto:santiago@crfreenet.org>> wrote:

On Wed, Oct 12, 2016 at 12:09:17PM +0100, Justin Cattle wrote:

...

Are there any thoughts as to why I still see quite large memory usage, and only on some [ seemingly random ] hosts ?

Hi

We are doing some testing and trying to identify the cause of the problem. We found some problems and strange behavior. One of them is excessive memory consumption when kernel export filters are defined.

-- Elen sila lumenn' omentielvo

Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org <mailto: santiago@crfreenet.org>) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net <http://wwwkeys.pgp.net>) "To err is human -- to blame it on a computer is even more so."

Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group.

If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses.

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group.

References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

<0001-Fixed-memory-bloating-on-kernel-merge-paths-together.patch>

-- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group. References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

I'd like to, but sorry I can't really. I can give you a lot of pointers though - and it's good to learn something about the package management :) First, get yourself a host that is the same debian or ubuntu version that you want to deploy the package on, and use it to build the package. Then create a directory where you want to do the build and cd in, preferably not as root. # setup # sudo apt-get install devscripts build-essential dpkg-dev quilt cat << END > ~/.quiltrc

export QUILT_PATCHES=debian/patches export QUILT_NO_DIFF_INDEX=1 export QUILT_NO_DIFF_TIMESTAMPS=1 export QUILT_REFRESH_ARGS="-p ab" export QUILT_DIFF_ARGS="--color=auto" END

sudo apt-get build-dep bird # get source # apt-get source bird # Apply patch # cd bird-1.6.2 quilt new 1000-fixed-memory-bloating-on-kernel-merge-paths-together.patch quilt add nest/rt-table.c quilt refresh quilt header -e # enter a description of the patch quilt push quilt refresh # Bump version with local name # dch -l companyname # or some other name - enter into as appropriate and save the file # Build the package # dpkg-buildpackage -tc # That's it! # cd .. ls -al # you should see some packages if it worked. Hope that helps.. Cheers, Just On 12 October 2016 at 18:21, Madhu <informmadhu@gmail.com> wrote:

Hi just,

Is there any way you can give me your built debian package, So that I can install and check :) ?

Madhu

On Wed, Oct 12, 2016 at 8:37 AM, Justin Cattle <j@ocado.com> wrote:

...

Personally - I get the debian source package, then apply the patch to that using quilt, bump the version with my own local identifier, and build the new package.

Cheers, Just

On 12 October 2016 at 16:19, Madhu <informmadhu@gmail.com> wrote:

...

How to install this patch?

Madhu

Sent from my iPhone

...

On Oct 12, 2016, at 5:27 AM, Jan Matejka <jan.matejka@nic.cz> wrote:

Hi!

Please try the last commit in GIT, branch master. Commit ID 2e7fb11a6e31324151c6db98df2fe26d2d6cffab. Attaching the patch as well.

Thank you both for reporting this issue.

Jan

...

On 10/12/2016 02:10 PM, Justin Cattle wrote: Good stuff - once again, please let me know if you want to me test any patching at this end :)

Cheers, Just

On 12 October 2016 at 13:02, Ondrej Zajicek <santiago@crfreenet.org <mailto:santiago@crfreenet.org>> wrote:

On Wed, Oct 12, 2016 at 12:09:17PM +0100, Justin Cattle wrote:

...

Are there any thoughts as to why I still see quite large memory usage, and only on some [ seemingly random ] hosts ?

Hi

We are doing some testing and trying to identify the cause of the problem. We found some problems and strange behavior. One of them is excessive memory consumption when kernel export filters are defined.

-- Elen sila lumenn' omentielvo

Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org <mailto: santiago@crfreenet.org>) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net <http://wwwkeys.pgp.net>) "To err is human -- to blame it on a computer is even more so."

Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group.

If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses.

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group.

References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

<0001-Fixed-memory-bloating-on-kernel-merge-paths-together.patch>

Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group.

If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses.

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group.

References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

-- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group. References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

I would be: bird_1.6.2-1~bpo8+1madhupatch1_amd64.deb Apologies, but I just realised I forgot the step to actually apply the patch to the file! :) You may have realised that, and done it - but if not, just repeat the steps again, but change the "apply patch" section to this: # Apply patch # cd bird-1.6.2 quilt new 1000-fixed-memory-bloating-on-kernel-merge-paths-together.patch quilt add nest/rt-table.c *patch -p1 < ../0001-Fixed-memory-bloating-on-kernel-merge-paths-together.patc* quilt refresh quilt header -e # enter a description of the patch quilt push quilt refresh If you do have to do this process again, it's best to delete the directory you did the work in, and create it again, or just do it again in another directory starting from "# get source #" That should prevent any associated build issues. Cheers, Just On 13 October 2016 at 01:14, Madhu <informmadhu@gmail.com> wrote:

Thanks Just. I followed your steps. I got these files. Which one I have to install?

ls bird-1.6.2 bird-bgp_1.6.2-1~bpo8+1madhupatch1_all.deb bird-doc_1.6.2-1~bpo8+1madhupatch1_all.deb bird_1.6.2-1~bpo8+1.debian.tar.xz bird_1.6.2-1~bpo8+1.dsc bird_1.6.2-1~bpo8+1madhupatch1.debian.tar.xz bird_1.6.2-1~bpo8+1madhupatch1.dsc bird_1.6.2-1~bpo8+1madhupatch1_amd64.changes bird_1.6.2-1~bpo8+1madhupatch1_amd64.deb bird_1.6.2.orig.tar.gz

Please let me know.

Thanks, Madhu

On Wed, Oct 12, 2016 at 12:16 PM, Justin Cattle <j@ocado.com> wrote:

...

I'd like to, but sorry I can't really.

I can give you a lot of pointers though - and it's good to learn something about the package management :)

First, get yourself a host that is the same debian or ubuntu version that you want to deploy the package on, and use it to build the package. Then create a directory where you want to do the build and cd in, preferably not as root.

# setup #

sudo apt-get install devscripts build-essential dpkg-dev quilt

cat << END > ~/.quiltrc

...

export QUILT_PATCHES=debian/patches export QUILT_NO_DIFF_INDEX=1 export QUILT_NO_DIFF_TIMESTAMPS=1 export QUILT_REFRESH_ARGS="-p ab" export QUILT_DIFF_ARGS="--color=auto" END

sudo apt-get build-dep bird

# get source #

apt-get source bird

# Apply patch #

cd bird-1.6.2

quilt new 1000-fixed-memory-bloating-on-kernel-merge-paths-together.patch quilt add nest/rt-table.c quilt refresh quilt header -e # enter a description of the patch quilt push quilt refresh

# Bump version with local name #

dch -l companyname # or some other name - enter into as appropriate and save the file

# Build the package #

dpkg-buildpackage -tc

# That's it! # cd ..

ls -al # you should see some packages if it worked.

Hope that helps..

Cheers, Just

On 12 October 2016 at 18:21, Madhu <informmadhu@gmail.com> wrote:

...

Hi just,

Is there any way you can give me your built debian package, So that I can install and check :) ?

Madhu

On Wed, Oct 12, 2016 at 8:37 AM, Justin Cattle <j@ocado.com> wrote:

...

Personally - I get the debian source package, then apply the patch to that using quilt, bump the version with my own local identifier, and build the new package.

Cheers, Just

On 12 October 2016 at 16:19, Madhu <informmadhu@gmail.com> wrote:

...

How to install this patch?

Madhu

Sent from my iPhone

...

On Oct 12, 2016, at 5:27 AM, Jan Matejka <jan.matejka@nic.cz> wrote:

Hi!

Please try the last commit in GIT, branch master. Commit ID 2e7fb11a6e31324151c6db98df2fe26d2d6cffab. Attaching the patch as well.

Thank you both for reporting this issue.

Jan

> On 10/12/2016 02:10 PM, Justin Cattle wrote: > Good stuff - once again, please let me know if you want to me test any patching at this end :) > > > > Cheers, > Just > > On 12 October 2016 at 13:02, Ondrej Zajicek <santiago@crfreenet.org <mailto:santiago@crfreenet.org>> wrote: > > On Wed, Oct 12, 2016 at 12:09:17PM +0100, Justin Cattle wrote: >> Are there any thoughts as to why I still see quite large memory usage, and >> only on some [ seemingly random ] hosts ? > > Hi > > We are doing some testing and trying to identify the cause of the problem. > We found some problems and strange behavior. One of them is excessive > memory consumption when kernel export filters are defined. > > -- > Elen sila lumenn' omentielvo > > Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org <mailto:santiago@crfreenet.org>) > OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net <http://wwwkeys.pgp.net>) > "To err is human -- to blame it on a computer is even more so." > > > > Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. > > > > If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. > > > > Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group. > > > > References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE. >

<0001-Fixed-memory-bloating-on-kernel-merge-paths-together.patch>

Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group.

If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses.

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group.

References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group.

If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses.

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group.

References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

-- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group. References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

Any update on this ? Madhu Sent from my iPhone

On Oct 14, 2016, at 11:40 AM, Madhu <informmadhu@gmail.com> wrote:

I still see the crash with the changes. Below is the trace.

[ 1037.412668] Call Trace: [ 1037.415402] [<ffffffff81b96521>] dump_stack+0x64/0x82 [ 1037.421146] [<ffffffff81b91a46>] dump_header+0x7f/0x1f1 [ 1037.427083] [<ffffffff810b3e76>] ? put_online_cpus+0x56/0x80 [ 1037.433507] [<ffffffff811110ac>] ? rcu_oom_notify+0xcc/0xf0 [ 1037.439834] [<ffffffff811a2735>] oom_kill_process+0x205/0x360 [ 1037.446356] [<ffffffff814595b5>] ? security_capable_noaudit+0x15/0x20 [ 1037.453655] [<ffffffff811a2ee2>] out_of_memory+0x492/0x4d0 [ 1037.459883] [<ffffffff811a8f30>] __alloc_pages_nodemask+0xa00/0xb60 [ 1037.466988] [<ffffffff811eb1fa>] alloc_pages_vma+0x9a/0x160 [ 1037.473314] [<ffffffff811cd5c8>] handle_mm_fault+0xd38/0x1080 [ 1037.479835] [<ffffffff811d4b74>] ? change_protection+0x594/0x850 [ 1037.486647] [<ffffffff8109f5ae>] __do_page_fault+0x19e/0x560 [ 1037.493070] [<ffffffff811d4f81>] ? mprotect_fixup+0x151/0x290 [ 1037.499588] [<ffffffff8109f9a1>] do_page_fault+0x31/0x70 [ 1037.505623] [<ffffffff81ba40e8>] page_fault+0x28/0x30

...

On Thu, Oct 13, 2016 at 1:52 AM, Justin Cattle <j@ocado.com> wrote:

I would be:

bird_1.6.2-1~bpo8+1madhupatch1_amd64.deb

Apologies, but I just realised I forgot the step to actually apply the patch to the file! :) You may have realised that, and done it - but if not, just repeat the steps again, but change the "apply patch" section to this:

# Apply patch #

cd bird-1.6.2

quilt new 1000-fixed-memory-bloating-on-kernel-merge-paths-together.patch quilt add nest/rt-table.c patch -p1 < ../0001-Fixed-memory-bloating-on-kernel-merge-paths-together.patc quilt refresh quilt header -e # enter a description of the patch quilt push quilt refresh

If you do have to do this process again, it's best to delete the directory you did the work in, and create it again, or just do it again in another directory starting from "# get source #" That should prevent any associated build issues.

Cheers, Just

...

On 13 October 2016 at 01:14, Madhu <informmadhu@gmail.com> wrote: Thanks Just. I followed your steps. I got these files. Which one I have to install?

ls bird-1.6.2 bird-bgp_1.6.2-1~bpo8+1madhupatch1_all.deb bird-doc_1.6.2-1~bpo8+1madhupatch1_all.deb bird_1.6.2-1~bpo8+1.debian.tar.xz bird_1.6.2-1~bpo8+1.dsc bird_1.6.2-1~bpo8+1madhupatch1.debian.tar.xz bird_1.6.2-1~bpo8+1madhupatch1.dsc bird_1.6.2-1~bpo8+1madhupatch1_amd64.changes bird_1.6.2-1~bpo8+1madhupatch1_amd64.deb bird_1.6.2.orig.tar.gz

Please let me know.

Thanks, Madhu

...

On Wed, Oct 12, 2016 at 12:16 PM, Justin Cattle <j@ocado.com> wrote: I'd like to, but sorry I can't really.

I can give you a lot of pointers though - and it's good to learn something about the package management :)

First, get yourself a host that is the same debian or ubuntu version that you want to deploy the package on, and use it to build the package. Then create a directory where you want to do the build and cd in, preferably not as root.

# setup #

sudo apt-get install devscripts build-essential dpkg-dev quilt

cat << END > ~/.quiltrc

...

export QUILT_PATCHES=debian/patches export QUILT_NO_DIFF_INDEX=1 export QUILT_NO_DIFF_TIMESTAMPS=1 export QUILT_REFRESH_ARGS="-p ab" export QUILT_DIFF_ARGS="--color=auto" END

sudo apt-get build-dep bird

# get source #

apt-get source bird

# Apply patch #

cd bird-1.6.2

quilt new 1000-fixed-memory-bloating-on-kernel-merge-paths-together.patch quilt add nest/rt-table.c quilt refresh quilt header -e # enter a description of the patch quilt push quilt refresh

# Bump version with local name #

dch -l companyname # or some other name - enter into as appropriate and save the file

# Build the package #

dpkg-buildpackage -tc

# That's it! # cd ..

ls -al # you should see some packages if it worked.

Hope that helps..

Cheers, Just

...

On 12 October 2016 at 18:21, Madhu <informmadhu@gmail.com> wrote: Hi just,

Is there any way you can give me your built debian package, So that I can install and check :) ?

Madhu

...

On Wed, Oct 12, 2016 at 8:37 AM, Justin Cattle <j@ocado.com> wrote: Personally - I get the debian source package, then apply the patch to that using quilt, bump the version with my own local identifier, and build the new package.

Cheers, Just

> On 12 October 2016 at 16:19, Madhu <informmadhu@gmail.com> wrote: > How to install this patch? > > Madhu > > Sent from my iPhone > > > On Oct 12, 2016, at 5:27 AM, Jan Matejka <jan.matejka@nic.cz> wrote: > > > > Hi! > > > > Please try the last commit in GIT, branch master. Commit ID 2e7fb11a6e31324151c6db98df2fe26d2d6cffab. > > Attaching the patch as well. > > > > Thank you both for reporting this issue. > > > > Jan > > > >> On 10/12/2016 02:10 PM, Justin Cattle wrote: > >> Good stuff - once again, please let me know if you want to me test any patching at this end :) > >> > >> > >> > >> Cheers, > >> Just > >> > >> On 12 October 2016 at 13:02, Ondrej Zajicek <santiago@crfreenet.org <mailto:santiago@crfreenet.org>> wrote: > >> > >> On Wed, Oct 12, 2016 at 12:09:17PM +0100, Justin Cattle wrote: > >>> Are there any thoughts as to why I still see quite large memory usage, and > >>> only on some [ seemingly random ] hosts ? > >> > >> Hi > >> > >> We are doing some testing and trying to identify the cause of the problem. > >> We found some problems and strange behavior. One of them is excessive > >> memory consumption when kernel export filters are defined. > >> > >> -- > >> Elen sila lumenn' omentielvo > >> > >> Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org <mailto:santiago@crfreenet.org>) > >> OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net <http://wwwkeys.pgp.net>) > >> "To err is human -- to blame it on a computer is even more so." > >> > >> > >> > >> Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. > >> > >> > >> > >> If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. > >> > >> > >> > >> Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group. > >> > >> > >> > >> References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE. > >> > > > > <0001-Fixed-memory-bloating-on-kernel-merge-paths-together.patch> >

Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group.

If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses.

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group.

References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group.

If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses.

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group.

References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.

Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group.

If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses.

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled is a trading name of Marie Claire Beauty Limited, both members of the Ocado Group.

References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE.