[Backgroundrb-devel] Server dying with perpetual "Connection reset by peer"

Daniel Azuma dazuma at alumni.caltech.edu
Sun Jul 29 14:29:39 EDT 2007


I'm at a loss to explain a very strange error I'm getting.

This seems to happen on our production system where we have a  
backgroundrb server sharing a host with a Rails app running on  
Mongrel. The first one or two, occasionally three, calls to a worker  
(direct MiddleMan calls from Rails, not scheduled) succeed, but  
afterwards, the server fails to initialize new workers. The following  
exception appears in the server logs, and I have to restart the  
server to get it to work again (for one or two requests before it  
dies again).

This appears to be the same issue that Peer Allan observed back in  
May-- in that case the workers were triggered on a schedule, but in  
my case it is triggered via a remote MiddleMan.

Does anyone have any idea what could cause this? Sample stack trace  
below:

Connection reset by peer - (DRb::DRbConnError)
/usr/local/lib/ruby/1.8/drb/drb.rb:566:in `read'
/usr/local/lib/ruby/1.8/drb/drb.rb:566:in `load'
/usr/local/lib/ruby/1.8/drb/drb.rb:632:in `recv_reply'
/usr/local/lib/ruby/1.8/drb/drb.rb:921:in `recv_reply'
/usr/local/lib/ruby/1.8/drb/drb.rb:1195:in `send_message'
/usr/local/lib/ruby/1.8/drb/drb.rb:1086:in `method_missing'
/usr/local/lib/ruby/1.8/drb/drb.rb:1170:in `open'
/usr/local/lib/ruby/1.8/drb/drb.rb:1085:in `method_missing'
/usr/local/lib/ruby/1.8/drb/drb.rb:1103:in `with_friend'
/usr/local/lib/ruby/1.8/drb/drb.rb:1084:in `method_missing'
/usr/local/lib/ruby/1.8/drb/drb.rb:1072:in `respond_to?'
/usr/local/lib/ruby/gems/1.8/gems/slave-1.2.1/lib/slave.rb:454:in  
`initialize'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
middleman.rb:210:in `new'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
middleman.rb:210:in `new_worker'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
thread_pool.rb:36:in `dispatch'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
thread_pool.rb:22:in `initialize'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
thread_pool.rb:22:in `new'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
thread_pool.rb:22:in `dispatch'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
middleman.rb:199:in `new_worker'
/usr/local/lib/ruby/1.8/drb/drb.rb:1555:in `__send__'
/usr/local/lib/ruby/1.8/drb/drb.rb:1555:in `perform_without_block'
/usr/local/lib/ruby/1.8/drb/drb.rb:1515:in `perform'
/usr/local/lib/ruby/1.8/drb/drb.rb:1589:in `main_loop'
/usr/local/lib/ruby/1.8/drb/drb.rb:1585:in `loop'
/usr/local/lib/ruby/1.8/drb/drb.rb:1585:in `main_loop'
/usr/local/lib/ruby/1.8/drb/drb.rb:1581:in `start'
/usr/local/lib/ruby/1.8/drb/drb.rb:1581:in `main_loop'
/usr/local/lib/ruby/1.8/drb/drb.rb:1430:in `run'
/usr/local/lib/ruby/1.8/drb/drb.rb:1427:in `start'
/usr/local/lib/ruby/1.8/drb/drb.rb:1427:in `run'
/usr/local/lib/ruby/1.8/drb/drb.rb:1347:in `initialize'
/usr/local/lib/ruby/1.8/drb/drb.rb:1627:in `new'
/usr/local/lib/ruby/1.8/drb/drb.rb:1627:in `start_service'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb_server.rb:315:in `run'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
application.rb:194:in `call'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
application.rb:194:in `start_proc'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
daemonize.rb:192:in `call'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
daemonize.rb:192:in `call_as_daemon'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
application.rb:198:in `start_proc'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
application.rb:234:in `start'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
controller.rb:69:in `run'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons.rb:185:in  
`run_proc'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
cmdline.rb:105:in `call'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
cmdline.rb:105:in `catch_exceptions'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons.rb:184:in  
`run_proc'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb_server.rb:301:in `run'

Thanks,
Daniel Azuma



More information about the Backgroundrb-devel mailing list