[Mongrel] mongrel memory usage ballooning and process stomping

Surendra Singhi singhi.surendra at gmail.com
Thu Jan 18 02:06:42 EST 2007


Hi,

First of all thanks everyone for their responses.

On 1/18/07, Ezra Zygmuntowicz <ezmobius at gmail.com> wrote:
>
>
>         Some linux distros have weird reporting of processes. I have seen
> it
> where top or ps will report 3 mongrels for each one that is really
> running. And looking at your output you can see that each set of
> three mongrels reported have the same port number. This means that
> you really only have 3 mongrels running because only one can be
> running per port at a time.


The output of 'ps aefux' is below. Though I don't understand ps completely
it seems that the one mongrel process spawns another child process, which in
turns spawns a third one.


ssinghi  22988  0.7 11.3 142900 102572 ?     S    21:53   0:24
/usr/bin/ruby1.8 /usr/bin/mongrel_rails start -d -e production -p 8000 -a
127.0.0.1 -P log/mon
ssinghi  22998  0.0 11.3 142900 102572 ?     S    21:54   0:00  \_
/usr/bin/ruby1.8 /usr/bin/mongrel_rails start -d -e production -p 8000 -a
127.0.0.1 -P log
ssinghi  23000  0.0 11.3 142900 102572 ?     S    21:54   0:00      \_
/usr/bin/ruby1.8 /usr/bin/mongrel_rails start -d -e production -p 8000 -a
127.0.0.1 -P
ssinghi  22991  0.9 12.0 131192 108844 ?     S    21:53   0:33
/usr/bin/ruby1.8 /usr/bin/mongrel_rails start -d -e production -p 8001 -a
127.0.0.1 -P log/mon
ssinghi  22996  0.0 12.0 131192 108844 ?     S    21:54   0:00  \_
/usr/bin/ruby1.8 /usr/bin/mongrel_rails start -d -e production -p 8001 -a
127.0.0.1 -P log
ssinghi  22997  0.0 12.0 131192 108844 ?     S    21:54   0:00      \_
/usr/bin/ruby1.8 /usr/bin/mongrel_rails start -d -e production -p 8001 -a
127.0.0.1 -P
ssinghi  22994  0.5 10.1 127980 92160 ?      S    21:53   0:18
/usr/bin/ruby1.8 /usr/bin/mongrel_rails start -d -e production -p 8002 -a
127.0.0.1 -P log/mon
ssinghi  22999  0.0 10.1 127980 92160 ?      S    21:54   0:00  \_
/usr/bin/ruby1.8 /usr/bin/mongrel_rails start -d -e production -p 8002 -a
127.0.0.1 -P log
ssinghi  23001  0.0 10.1 127980 92160 ?      S    21:54   0:00      \_
/usr/bin/ruby1.8 /usr/bin/mongrel_rails start -d -e production -p 8002 -a
127.0.0.1 -P



        As far as the memory usage goes, that sounds like a classic memory
> leak in your rails app. I have seen mongrel balloon like that when
> people add an extra :include directive to a find that ends up loading
> a ton of extra records into memory. If you are loading more then a
> few hundred ActiveRecords into memory on any one page then that will
> surely cause memory to ballon like this.


Yes, we are doing eager loading at many places, and there is a sitemap part
where about 10,000 records are loaded. I will try to optimize those parts,
and see
if they make a difference.

        A little more info about your app and what it does woudl help debug.
> Are you using Rmagcik?


Yes we are using that. I will look into mini-magick. Joey thanks for that.

Are you useing send_file or send_data to
> stream out large content?


We are streaming data for images.

We are also using ferret and mediacloth.

I guess, I need to investigate more on the above mentioned things.
Thanks a lot.

Surendra Singhi



On Jan 17, 2007, at 10:36 AM, Michael Biven wrote:
>
> > How are the mongrel started (cron, launchd, etc)? I had something
> > similar happen to me, but I think it was from the cluster being
> > started by root instead of the user I wanted to start it. It gave me
> > an extra set of mongrel processes running that mongrel_rails
> > cluster::stop wouldn't stop.
> >
> > Michael B.
> >
> > On 1/17/07, Surendra Singhi <singhi.surendra at gmail.com> wrote:
> >> Hi,
> >>   I have mongrel 0.3.14, with ruby 1.8.5, rails 1.1.6 and mongrel
> >> cluster
> >> 0.2.1, on debian sage 3.1 with apache 2.0, and fastthread 0.6.1.
> >> I am load balancing 3 mongrel processes using the random port trick.
> >>
> >>  When I start mongrel the processes have about 60MB, but after
> >> some hour of
> >> usage the memory ballons upto more than 180MB and the site becomes
> >> terribly
> >> slow. Forcing me to restart mongrel cluster.
> >>
> >> Also, it reports me 9 mongrel processes instead of three. I am not
> >> able to
> >> understand why that's happening. Are really 9 mongrel processes
> >> started
> >> instead of three?
> >>
> >> Please help. Thanks.
> >
> > --
> > Michael Biven
> > michael.biven at gmail.com
> > http://biven.org
> > _______________________________________________
> > Mongrel-users mailing list
> > Mongrel-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/mongrel-users
>
> -- Ezra Zygmuntowicz
> -- Lead Rails Evangelist
> -- ez at engineyard.com
> -- Engine Yard, Serious Rails Hosting
> -- (866) 518-YARD (9273)
>
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>



-- 
Surendra Singhi
http://ssinghi.kreeti.com, http://www.kreeti.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070118/9b203e83/attachment-0001.html 


More information about the Mongrel-users mailing list