[Mongrel] Mongrel_rails memory usage ballooning rapidly

mcr at simtone.net mcr at simtone.net
Wed Jul 22 10:50:12 EDT 2009

>>>>> "Piyush" == Piyush Ranjan <piyush.pr at gmail.com> writes:
    Piyush> 8 mongrels are way too much capacity. Do you need that many ? Mongrels
    Piyush> taking 300MB is not unheard of as James said. Are you using loads of
    Piyush> libraries or cache in memory ? Moreover counting per mongrel memory is not
    Piyush> so easy. If they use shared libraries the memory per process shoots up in
    Piyush> the "ps auwx" command but actual memory usage is not that
    Piyush> high.

ps auwx output:

xdsott003-[~] mcr 1256 %ps auwx
root         1  0.0  0.2   1704   540 ?        Ss   Jun18   0:00 /sbin/init
root         2  0.0  0.0      0     0 ?        S    Jun18   0:00 [migration/0]

The VSZ gives the total virtual size.
The RSS gives the resident set size.

If RSS is staying small-ish, and VSZ is climbing, then you have a memory
leak (inside the process) of some kind. The leaked memory will be pushed
into swap.

If you run "lsof -p XXX" on the PIDs of the mongrels:

mustang-[~] mcr 1093 %lsof -p 15305
mongrel_r 15305 xas2  cwd   unknown                       /proc/15305/cwd (readlink: Permission denied)
mongrel_r 15305 xas2  rtd   unknown                       /proc/15305/root (readlink: Permission denied)
mongrel_r 15305 xas2  txt   unknown                       /proc/15305/exe (readlink: Permission denied)
mongrel_r 15305 xas2  mem       REG    8,1    3336 266044 /usr/bin/ruby1.8
mongrel_r 15305 xas2  mem       REG    0,0              0 [heap] (stat: No such file or directory)
mongrel_r 15305 xas2  mem       REG    8,1   38372 170534 /lib/tls/i686/cmov/libnss_files-2.3.6.so
mongrel_r 15305 xas2  mem       REG    8,1   76548 170528 /lib/tls/i686/cmov/libnsl-2.3.6.so

You can see the size of each shared object.  Mostly, they will all be
loaded by the time that the mongrel has served a few requests.  (It is
certainly possible that you only load some other things later on, if so,
there is an easy out)

I have not seen my mongrels grow to 300M each.  Typically, they are 90
to 100M, but I do not do a lot of caching, etc.

I think it is worth writing some perl scripts to try to characterize if
your mongrel grows to 300M quickly, or slowly.  You might do this
externally (in perl/shell/awk/), or you could do it inside of mongrel by
using /proc/self to self-examine and lot sizes on each request.

I agree that having 8 mongrels may not be the best idea --- depends upon
how much CPU and RAM you have.  I planned to have have 4 per single-CPU
virtual machine, and scale horizontally with additional application

(Someone has a patch to have capistrano turn off one mongrel at a time
during a deployment, but I do not recall where)

Michael Richardson <mcr at simtone.net>
Director -- Consumer Desktop Development, Simtone Corporation, Ottawa, Canada
Personal: http://www.sandelman.ca/mcr/ 

SIMtone Corporation fundamentally transforms computing into simple,
secure, and very low-cost network-provisioned services pervasively
accessible by everyone.  Learn more at www.simtone.net and www.SIMtoneVDU.com 

More information about the Mongrel-users mailing list