[Mongrel] Mongrel 0.3.13.3 processes occasionally lock up - critical issue
zedshaw at zedshaw.com
Wed Aug 30 20:42:20 EDT 2006
On Wed, 2006-08-30 at 15:10 -0700, kigsteronline at mac.com wrote:
> Hello Mongrel gurus!
> We are about to deploy our Rails application to production, and even
> though the last couple of weeks of testing went really well, today we
> are seeing a new critical issue - the browser just hangs sometimes
> trying to load a page. When this happens, it seems that one or more of
> the mongrel_rails processes is hanging and not responding, while
> others are (so depending on which process the request is routed to,
> the page loads or not). The only remedy from this point onward
> appears to be brutal kill and restart of mongrel processes.
Ah, you might be hitting the CLOSE_WAIT bug. This makes me think more
and more that Apache is doing something funky. This bug and a 99% CPU
bug is driving down right insane since I can't replicate it to save my
life. I've got mongrel serving ISO images to an RFuzz script that
randomly closes sockets and does nasty requests while thrashing mongrel
with httperf with a short timeout.
Nothing. Not even one CLOSE_WAIT socket, no memory leaks (anymore), and
no CPU thrashing. It's really driving me nuts.
What I've done is created a special "debug version" of the mongrel
pre-release that I'd like you to run. Here's what you do:
1) download http://www.zedshaw.com/mongrel-0.3.13.4.gem
2) install: gem install mongrel-0.3.13.4.gem
3) start your app just like it is.
4) Hit mongrels with USR1: killall -USR1 mongrel_rails
5) Let mongrel run until it dies horribly.
6) Just before you kill -9 mongrel, run: lsof -i -P | grep CLOSE_WAIT >
7) Stop mongrel and send me log/mongrel.log and close_wait.log to
zedshaw at zedshaw.com.
Hopefully I can nail this for the next release. The CLOSE_WAIT used to
be really bad but I cut it down quite a bit, so it should take longer to
Also make sure to tell me if the CPU usage for mongrel is above 90%
Thanks for the help.
Zed A. Shaw
http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.
More information about the Mongrel-users