[Mongrel] Mongrel spewing backtraces and nanosleeping

Jacob Atzen jacob at jacobatzen.dk
Tue Sep 19 12:06:24 EDT 2006


On Fri, Sep 15, 2006 at 03:02:11PM +0200, Jacob Atzen wrote:
> A little heads-up before the weekend, I've been running with the above
> modifications and a few more to make the mutex calls run. So far I've
> been processing ~28000 rails requests on a mongrel with a limit of 64
> processors. I will perform further testing and supply a full diff of my
> modifications early next week.

I have now performed some more extensive testing based on the
suggestions by Kirk and have found Mongrel crashing when I raise the
number of processors. I have attached a diff of my altarations to
rails.rb in case anyone is interested. So alas, this did not cure the problems.
I will try to upgrade mongrel to the new release and see how that holds up.

Here is a snippet from mongrel.log up to the point where Mongrel crashed:

Server overloaded with 512 processors (512 max). Dropping connection.
Fri Sep 15 21:45:46 CEST 2006: Reaping 512 threads for slow workers because of 'max processors'
Thread #<Thread:0x8dc971c sleep> is too old, killing.
Fri Sep 15 21:45:46 CEST 2006: Error calling Dispatcher.dispatch #<Mongrel::TimeoutError: Mongrel timed out this thread: max processors>
/usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:637:in `lock'
/usr/local/lib/ruby/1.8/thread.rb:133:in `synchronize'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel/rails.rb:88:in `process'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:580:in `process_client'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:579:in `process_client'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:686:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:686:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:673:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel/configurator.rb:267:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel/configurator.rb:266:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/bin/mongrel_rails:127:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel/command.rb:203:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/bin/mongrel_rails:231
/usr/local/bin/mongrel_rails:18
Thread #<Thread:0x8fcf14c run> is too old, killing.
Fri Sep 15 21:45:46 CEST 2006: ERROR: Mongrel timed out this thread: max processors
Thread #<Thread:0x8cc6b6c dead> is too old, killing.
Thread #<Thread:0x8d15050 dead> is too old, killing.
Thread #<Thread:0x8f294e0 dead> is too old, killing.
Thread #<Thread:0x8f6c394 dead> is too old, killing.
Thread #<Thread:0x8fac7c8 dead> is too old, killing.
Thread #<Thread:0x8fc41c0 dead> is too old, killing.
Thread #<Thread:0x8fc8590 dead> is too old, killing.
Thread #<Thread:0x8fd0830 run> is too old, killing.
/usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:637:in `run': Mongrel timed out this thread: max processors (Mongrel::T
imeoutError)
        from /usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:686:in `run'
        from /usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:673:in `run'
        from /usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel/configurator.rb:267:in `run'
        from /usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel/configurator.rb:266:in `run'
        from /usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/bin/mongrel_rails:127:in `run'
        from /usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel/command.rb:203:in `run'
        from /usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/bin/mongrel_rails:231
        from /usr/local/bin/mongrel_rails:18

-- 
Cheers,
- Jacob Atzen
-------------- next part --------------
9c9
< require 'sync'
---
> require 'thread'
45c45
<         @guard = Sync.new
---
>         @guard = Mutex.new
83c83
<             @guard.synchronize(:EX) {
---
>             @guard.synchronize {
100c100
<           STDERR.puts "#{Time.now}: #{@guard.sync_waiting.length} threads sync_waiting for #{event}, #{self.listener.workers.list.length} still active in mongrel."
---
>           #STDERR.puts "#{Time.now}: #{@guard.sync_waiting.length} threads sync_waiting for #{event}, #{self.listener.workers.list.length} still active in mongrel."


More information about the Mongrel-users mailing list