[Backgroundrb-devel] Drb Connection error on multiple dispatch.fcgi 's

Ezra Zygmuntowicz ezmobius at gmail.com
Mon Apr 9 15:19:26 EDT 2007


	I think what is happening is that when apache spawns a new  
dispatch.fcgi it forks the first one to do so. And the drb connection  
cannot survive across a fork. The quikc fix is to use a set number of  
dispatchers so they each start with their own drb connection.

	I'm sorry to say that I haven't used bdrb with fastcgi as I gave up  
on fcgi a while ago. But it does appear that apache-fcgid forking a  
new process from the first is what causes the second forked process  
to not have a good drb connection.


-Ezra

On Apr 9, 2007, at 11:01 AM, Pascal Rettig wrote:

> Hi All,
>
>
> I'm using Backgroundrb as a general purpose long-running-task back-end
> (upload processing, email sending, etc), and it's been a great  
> solution.
> However I've recently run into some some intermittent connection  
> issues
> that have me baffled. I'm running on apache2/fcgid and the problem
> occurs in both devlopment and production mode. The problem seems to  
> only
> trigger when there's enough load on the server to create a second
> dispatch.fcgi instance. When this happens, all of a sudden 1/2 of  
> my new
> workers fail. I haven't tried a mongrel setup but I found a message  
> from
> January 30th from jacobatzen.dk that had the same backtrace error,  
> so I
> don't think it's uniquely a fcgid issue.
>
> Anyone come across this before or have any ideas where to look to fix?
> The worker I'm using for testing right now just spits out a log  
> message
> (code at the bottom).
>
> Thanks,
>
> -Pascal
>
>
>> From log/background_server.log :
>
> Connection reset by peer - (DRb::DRbConnError)
> /usr/lib/ruby/1.8/drb/drb.rb:563:in `read'
> /usr/lib/ruby/1.8/drb/drb.rb:563:in `load'
> /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply'
> /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply'
> /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message'
> /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing'
> /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'
> /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing'
> /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend'
> /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing'
> /usr/lib/ruby/1.8/drb/drb.rb:1069:in `respond_to?'
> /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/slave.rb:454:in  
> `initialize'
> /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ 
> server/lib/backgroundrb/middleman.rb:210:in `new_worker'
> /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ 
> server/lib/backgroundrb/thread_pool.rb:36:in `dispatch'
> /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ 
> server/lib/backgroundrb/thread_pool.rb:22:in `dispatch'
> /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ 
> server/lib/backgroundrb/middleman.rb:199:in `new_worker'
> /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block'
> /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform'
> /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop'
> /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop'
> /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop'
> /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run'
> /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run'
> /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize'
> /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service'
> /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ 
> server/lib/backgroundrb_server.rb:315:in `run'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ 
> application.rb:187:in `start_proc'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ 
> application.rb:197:in `start_proc'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ 
> application.rb:227:in `start'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/controller.rb: 
> 72:in `run'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:182:in
> `run_proc'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb: 
> 105:in
> `catch_exceptions'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:181:in
> `run_proc'
> /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ 
> server/lib/backgroundrb_server.rb:301:in `run'
> ./script/backgroundrb:29
>
> The output from ./script/backgroundrb run:
>
> undefined method `wait' for nil:NilClass - (NoMethodError)
> /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ 
> server/lib/backgroundrb/middleman.rb:217:in `new_worker'
> /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ 
> server/lib/backgroundrb/thread_pool.rb:36:in `dispatch'
> /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ 
> server/lib/backgroundrb/thread_pool.rb:22:in `dispatch'
> /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ 
> server/lib/backgroundrb/middleman.rb:199:in `new_worker'
> /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block'
> /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform'
> /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop'
> /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop'
> /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop'
> /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run'
> /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run'
> /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize'
> /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service'
> /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ 
> server/lib/backgroundrb_server.rb:315:in `run'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ 
> application.rb:187:in `start_proc'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ 
> application.rb:197:in `start_proc'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ 
> application.rb:227:in `start'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/controller.rb: 
> 72:in `run'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:182:in
> `run_proc'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb: 
> 105:in
> `catch_exceptions'
> /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:181:in
> `run_proc'
> /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ 
> server/lib/backgroundrb_server.rb:301:in `run'
> ./script/backgroundrb:29
>
>
> In lib/workers/other_worker.rb:
>
> class OtherWorker <  BackgrounDRb::Rails
>
>   def do_work(args)
>
>     logger.warn("In Do Work:")
>     @processed = true
>   end
>
>   def finished?
>     @processed == true
>   end
>
>
> end
> OtherWorker.register
>
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>

-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)




More information about the Backgroundrb-devel mailing list