[Mongrel] Mongrel .pids disappearing?

Christopher Opena counterveil at gmail.com
Wed Sep 3 19:17:50 EDT 2008


Hello everyone, new to the list :)

Strange problem with Mongrels that we're having in a few cases...the
Mongrels start up fine, but as part of a long Capistrano deploy + database
we will normally cycle them - normally no issues there.  In those edge
cases, we've seen that the mongrel .pids have just up and disappeared while
running, so when shutting them down we get the error message that the .pids
do not exist.  Normally not a problem, except that when restarting them (and
thereby needing to create a new pid), the log comes back with the error
message:

** Daemonized, any open files are closed.  Look at log/mongrel.4102.pid
 and log/mongrel.4102.log for info.
 ** Starting Mongrel listening at 127.0.0.1:4102 <http://127.0.0.1:9002/>
 /usr/lib64/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/tcphack.rb:12:in
 `initialize_without_backlog': Address already in use - bind(2)
 (Errno::EADDRINUSE)

Logging into the server over ssh and executing 'sudo /usr/sbin/lsof -i:4102'
reveals that indeed a mongrel process is running without a .pid.  From what
I can see there is no log rotation, cron job, or similar activity on the box
that would be deleting the pids.  In fact, some of the 6 .pids are there
while others are not.  The pattern also changes with regard to which ones
are missing, which is mildly aggravating.

Box is CentOS 5.1 if that helps.  I also noted that someone was having this
issue before and was cured by running the MySQL gem after seeing MySQL
connections going stale / dying on their server, but we're already running
the MySQL gem and are not seeing those errors in the Mongrel log.

In fact, looking at the last successful start just before the error shown
above, there are no other errors.  In other words, Mongrel starts up
daemonized without an issue or any errors in the log, and then some time
later (in this case, 2 weeks), trying to stop them generates a nonexistent
.pid error.  The subsequent start attempt generates the 'Address already in
use' message, which forces us to log in and actually kill the processes
before we can start up again.

Processes without .pids don't seem to be using more or less memory than
usual.

We are stopping and starting the mongrels with the following bash script:

mongrel_rails cluster::start -C $MONGREL_CONF
mongrel_rails cluster::stop -C $MONGREL_CONF

Mongrel version is 1.0.1 (as described above), and MySQL gem is 2.7.

Has anyone seen a similar issue, and if so did you figure out the root
cause?  I'm a bit at a loss with the mongrel logs not giving any information
nor the system giving any inidication that it deleted the .pid files.  Any
help is greatly appreciated.

-Chris.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mongrel-users/attachments/20080903/069492f8/attachment.html>


More information about the Mongrel-users mailing list