[Mongrel] Memory leaks in my site

Ezra Zygmuntowicz ezmobius at gmail.com
Tue Mar 6 16:17:45 EST 2007


On Mar 6, 2007, at 12:34 PM, Joey Geiger wrote:

>> flex_image uses rmagick... which folks have lots of issues with...
>
> I re-wrote the bits of flex_image that I use to use mini-magick and
> it's working quite nicely.
> I did that when the whole discussion of "RMagick bad" came up a couple
> of months ago.
>
> On 3/6/07, Alexey Verkhovsky <alexey.verkhovsky at gmail.com> wrote:
>> So, memory leak is the problem, and monit is just highlighting it.  
>> Is that
>> correct?
>
> For me, yes. I'm not exactly sure what is happening, but on my current
> application, the only controller that was hit overnight was the monit
> checker, which was enough to cause the leak.
>
> Here's a copy of the controller, which I've tried to strip down as
> much as possible. (On my current production app, it's doing 10k
> requests per second :)
>
> class MonitController < ActionController::Base
>   session :off
> ##   this is used by the monitoring scripts to see if the mongrel is
> up and running
>   def index
>   end
> end


	I have seen weird behavior from having monit check with a http  
request. It woudl cause weird issues and mem leaks. I have since  
removed any http checks from all my monit recipes and used an  
external siteuptime monitoring for http status check. Monit just  
watches memory and cpu usage and restarts dead or bloated mongrels.

	It's just a fact of life with large rails applications, some of them  
just like to keep using memory as much as possible. I have found that  
restarting mongrels that go over 110Mb for a few cycles will keep  
things in check pretty well. This is for 64bit systems where memory  
usage is higher then on 32 bit systems.

	Here is a monit recipe I am using on a large number of servers that  
seems to work very well.

set httpd port 9111
     allow localhost

set daemon 60
set logfile /data/username/shared/log/monit.log
set mail-format {from:info at engineyard.com}
set mailserver smtp.engineyard.com
set alert eymonit at gmail.com

check process mongrel_username_0
   with pidfile /data/username/shared/log/mongrel.5000.pid
   start program = "/data/username/shared/bin/start_mongrel.sh 5000  
username"
   stop program = "/data/username/shared/bin/stop_mongrel.sh 5000  
username"
   if totalmem is greater than 110.0 MB for 4 cycles then  
restart       # eating up memory?
   if cpu is greater than 50% for 2 cycles then  
alert                  # send an email to admin
   if cpu is greater than 80% for 3 cycles then  
restart                # hung process?
   if loadavg(5min) greater than 10 for 8 cycles then  
restart          # bad, bad, bad
   if 10 restarts within 10 cycles then  
timeout                         # something is wrong, call the sys-admin
   group mongrel

check process mongrel_username_1
   with pidfile /data/username/shared/log/mongrel.5001.pid
   start program = "/data/username/shared/bin/start_mongrel.sh 5001  
username"
   stop program = "/data/username/shared/bin/stop_mongrel.sh 5001  
username"
   if totalmem is greater than 110.0 MB for 4 cycles then  
restart       # eating up memory?
   if cpu is greater than 50% for 2 cycles then  
alert                  # send an email to admin
   if cpu is greater than 80% for 3 cycles then  
restart                # hung process?
   if loadavg(5min) greater than 10 for 8 cycles then  
restart          # bad, bad, bad
   if 10 restarts within 10 cycles then  
timeout                         # something is wrong, call the sys-admin
   group mongrel

check process mongrel_username_2
   with pidfile /data/username/shared/log/mongrel.5002.pid
   start program = "/data/username/shared/bin/start_mongrel.sh 5002  
username"
   stop program = "/data/username/shared/bin/stop_mongrel.sh 5002  
username"
   if totalmem is greater than 110.0 MB for 4 cycles then  
restart       # eating up memory?
   if cpu is greater than 50% for 2 cycles then  
alert                  # send an email to admin
   if cpu is greater than 80% for 3 cycles then  
restart                # hung process?
   if loadavg(5min) greater than 10 for 8 cycles then  
restart          # bad, bad, bad
   if 10 restarts within 10 cycles then  
timeout                         # something is wrong, call the sys-admin
   group mongrel


Cheers-
-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)




More information about the Mongrel-users mailing list