with the command's ouput. Is this possible? [mp] Quoting Ezra Zygmuntowicz : > OK I know whats happening. Your while loop completes and calls kill on > the worker before your task_progress controller method ever gets > called> So the worker is deleted and when you try to access it from > rails you get an error because there is no longer a worker at that job > key. The kill method is meant to be used within a worker that you fire > and forget. If you want to get the results then you should not call > kill right in your worker. You can call kill or delete_worker from > rails in your done method. > > So if you want to see the progress bar working I suggest you put a > sleep in the while loop. A while loop with no sleep can count to 1200 > before you could blink and then the kill method gets called and your > screwed. So add sleep 0.3 or somethign within the while loop so you can > see it work. > > And if you want to see the results of your worker in rails you should > call the kill method from rails through the middlwman. If you want to > just fire and forget a worker then you want to call kill within the > worker when it is done working. > > Other then that your code is fine. Give it another shot with some > sleep in the loop. > > Cheers- > -Ezra > > > > On Aug 16, 2006, at 10:25 AM, mattp at digimonkey.com wrote: > >> Weird is, I had it working a second ago and then reloaded the >> webrick server and now it's erroring out with: >> >> NoMethodError (You have a nil object when you didn't expect it! >> The error occured while evaluating nil.progress): >> /app/controllers/operations_controller.rb:14:in `task_progress' >> >> Here's the code -- >> >> [ /lib/workers/ping_worker.rb ] >> >> class PingWorker < BackgrounDRb::Rails >> >> attr_accessor :progress >> >> def do_work(args) >> @progress = 0 >> calculate_the_meaning_of_life(args) >> kill >> end >> >> def calculate_the_meaning_of_life(args) >> while @progress < 100 >> @progress += 1 >> end >> end >> >> end >> >> [ operations_controller.rb ] >> >> class OperationsController < ApplicationController >> sidebar :general >> sidebar :operations >> >> # start new worker and put the job_key into the session so you can >> # get the status of your job later. >> def background_task >> session[:job_key] = MiddleMan.new_worker(:class => :ping_worker, >> :args => {:baz => >> 'hello!', :qux => 'another arg!'}) >> end >> >> def task_progress >> if request.xhr? >> progress_percent = MiddleMan.get_worker(session[:job_key]).progress >> render :update do |page| >> page.call('progressPercent', 'progressbar', progress_percent) >> page.redirect_to( :action => 'done') if progress_percent >= 100 >> end >> else >> redirect_to :action => 'index' >> end >> end >> >> def results >> @results = MiddleMan.get_worker(session[:job_key]).results >> MiddleMan.delete_worker(session[:job_key]) >> end >> >> def done >> end >> >> end >> >> Quoting Ezra Zygmuntowicz : >> >>> >>> On Aug 16, 2006, at 10:08 AM, backgroundrb-devel- >>> bounces at rubyforge.org wrote: >>> >>>> >>>> From: mattp at digimonkey.com >>>> Date: August 16, 2006 10:01:26 AM PDT >>>> To: backgroundrb-devel at rubyforge.org >>>> Subject: newbie needs help >>>> >>>> >>>> I'm sure this is the billionth newbie help request on here, but I >>>> sure am hitting my head against the wall on this. >>>> >>>> I've tried numberous of Ezra's examples that I was able to find >>>> online. The best I'm able to get is nothing happens and >>>> development log says: >>>> >>>> NoMethodError (undefined method `progress' for #): >>>> /app/controllers/operations_controller.rb:13:in `get_progress' >>>> >>>> Any ideas? I'm sure there's a basic thing I'm not doing right... >>>> but I couldn't figure out what that might be, based on the >>>> documentation. >>>> >>>> [mp] >>>> >>>> >>> >>> >>> Hi Matt- >>> >>> It looks like you are missing something. Can you post your worker >>> class? It looks like you either need a progress method or you need to >>> set attr_accessor :progress in your worker. >>> >>> >>> -Ezra >> >> From mattp at digimonkey.com Wed Aug 16 16:14:44 2006 From: mattp at digimonkey.com (mattp at digimonkey.com) Date: Wed, 16 Aug 2006 15:14:44 -0500 Subject: [Backgroundrb-devel] "tailing" a running system command? In-Reply-To: <20060816124803.jnxb7v6jb48wck0w@digimonkey.com> References: <4F966DB6-838C-4E83-9B9B-AF4C63299963@gmail.com> <20060816122506.1w2qkrus7k7ocgck@digimonkey.com> <28FB7167-3BE4-4E8D-8E27-F7B50FBAB7B0@gmail.com> <20060816124803.jnxb7v6jb48wck0w@digimonkey.com> Message-ID: <20060816151444.yc5xuiao9yysggso@digimonkey.com> Dunno if anyone saw the short question at the end of my last thread, so i'm reposting in a thread with the appropriate subject: Here's what I need to do -- Run a system-level command like 'ping -t www.google.com' and have it periodically update a

with the command's ouput. Is this possible? (the trick here is that I want *periodic* updates of the div... i want to see incremental output as 'ping' is running, just like if I were looking at console...) From gladwig at gmx.de Wed Aug 16 16:31:47 2006 From: gladwig at gmx.de (=?ISO-8859-1?Q?G=FCnter_Ladwig?=) Date: Wed, 16 Aug 2006 22:31:47 +0200 Subject: [Backgroundrb-devel] "tailing" a running system command? In-Reply-To: <20060816151444.yc5xuiao9yysggso@digimonkey.com> References: <4F966DB6-838C-4E83-9B9B-AF4C63299963@gmail.com> <20060816122506.1w2qkrus7k7ocgck@digimonkey.com> <28FB7167-3BE4-4E8D-8E27-F7B50FBAB7B0@gmail.com> <20060816124803.jnxb7v6jb48wck0w@digimonkey.com> <20060816151444.yc5xuiao9yysggso@digimonkey.com> Message-ID: <1D79F299-D1E4-4C3B-B0FB-88EE42BC602F@gmx.de> On 16.08.2006, at 22:14, mattp at digimonkey.com wrote: > Run a system-level command like 'ping -t www.google.com' and have it > periodically update a

with the command's ouput. Is this > possible? > > (the trick here is that I want *periodic* updates of the div... i want > to see incremental output as 'ping' is running, just like if I were > looking at console...) Yes, it should be possible. Have a look at the Rails function periodically_call_remote. In the controller method you could then get the output from your worker and replace the HTML in a particular

-tag. G?nter From me at seebq.com Wed Aug 16 17:19:49 2006 From: me at seebq.com (Charles Brian Quinn) Date: Wed, 16 Aug 2006 17:19:49 -0400 Subject: [Backgroundrb-devel] "tailing" a running system command? In-Reply-To: <20060816151444.yc5xuiao9yysggso@digimonkey.com> References: <4F966DB6-838C-4E83-9B9B-AF4C63299963@gmail.com> <20060816122506.1w2qkrus7k7ocgck@digimonkey.com> <28FB7167-3BE4-4E8D-8E27-F7B50FBAB7B0@gmail.com> <20060816124803.jnxb7v6jb48wck0w@digimonkey.com> <20060816151444.yc5xuiao9yysggso@digimonkey.com> Message-ID: <3a2de0cd0608161419j64b1b321j1360157740b2e0bf@mail.gmail.com> We're working on a super cool new tool to help automate capistrano and rake deployment task running on remote servers, and we do something similiar to this using backgrounDRb (thanks Ezra!). It is based off our RailsDay 2006 entry: http://heartbeat.highgroove.com/ -- enough with the bragging, here is how we do it: your worker now needs to have an @output class variable (instead of progress). in your worker, you can run the task one of two ways which might or might not work: # you can use Kernel.system to make the call which I don't know how you get output if not Kernel.system("ping google) errors.add_to_base(" your error here ") # this is technically in a model hence the errors obj raise end # or: cmd_to_run = "ping google" stdin, stdout, stderr = Open3.popen3(cmd_to_run) s_stderr = stderr.read.to_s s_stdout = stdout.read.to_s # if you'd like to see it logger.info "stdout: #{s_stdout}" logger.info "stderr: #{s_stderr}" if s_stderr.empty? && s_stdout.starts_with?("error stuff here") return true else errors.add_to_base("error msg here") return false end But, as for setting your @output, well, you'd need to run inside a loop or perhaps even in another thread within the bg worker thread. We don't need to do this because we're executing our task using ssh and we have an open channel and just do @output << in the loop as it runs. You may not be able to use the popen3 calls above, to be honest -- the read pretty much kills that idea. Our worker method that appends output looks like: Net::SSH.start( host, :username => username, :password => password, :port => port ) do |session| session.open_channel do |channel| channel.on_data do |ch, data| @output << "#{data}" end channel.on_extended_data do |ch, type, data| @output << "#{data}" end channel.exec( "ping google\n" ) end session.loop end Maybe you can find a simliar asynchronous call for executing methods, that updates output as it runs. Now, in your controller: the method, let's say execute: def execute session[:job_key] = Middleman.... # call your worker end and for the view we used rjs to update a page (a lightbox actually):

Connecting to Server...

execute.rjs: page << "showBG();" page << "showIbox('#{url_for(:action => 'output', :id => @task)}','',parseQuery('width=760&height=400'))" page << "output = new Ajax.PeriodicalUpdater('task_output','#{url_for(:action => 'update_output', :id=> @task.id)}',2);" back to your controller, it can be as simple as: def update_output @job = MiddleMan.get_worker(session[:job_key]) render :inline => '<%= simple_format(@job.output) %>' end The hard part will be appending @output as it's running. I know I didn't answer the @output side, but hopefully helped with the backgroundrb worker side. cheers and good luck, let us know how it goes. On 8/16/06, mattp at digimonkey.com wrote: > Dunno if anyone saw the short question at the end of my last thread, > so i'm reposting in a thread with the appropriate subject: > > Here's what I need to do -- > > Run a system-level command like 'ping -t www.google.com' and have it > periodically update a

with the command's ouput. Is this possible? > > (the trick here is that I want *periodic* updates of the div... i want > to see incremental output as 'ping' is running, just like if I were > looking at console...) > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > -- Charles Brian Quinn self-promotion: www.seebq.com highgroove studios: www.highgroove.com slingshot hosting: www.slingshothosting.com From mattp at digimonkey.com Wed Aug 16 17:47:23 2006 From: mattp at digimonkey.com (mattp at digimonkey.com) Date: Wed, 16 Aug 2006 16:47:23 -0500 Subject: [Backgroundrb-devel] "tailing" a running system command? In-Reply-To: <3a2de0cd0608161419j64b1b321j1360157740b2e0bf@mail.gmail.com> References: <4F966DB6-838C-4E83-9B9B-AF4C63299963@gmail.com> <20060816122506.1w2qkrus7k7ocgck@digimonkey.com> <28FB7167-3BE4-4E8D-8E27-F7B50FBAB7B0@gmail.com> <20060816124803.jnxb7v6jb48wck0w@digimonkey.com> <20060816151444.yc5xuiao9yysggso@digimonkey.com> <3a2de0cd0608161419j64b1b321j1360157740b2e0bf@mail.gmail.com> Message-ID: <20060816164723.eqpn58tzqa04sggk@digimonkey.com> Check it out... I got it working: [ worker ] class PingWorker < BackgrounDRb::Rails attr_accessor :result def do_work(args) @result = IO.popen("ping -n 10 www.google.com") end end [ controller ] def ping_task session[:job_key] = MiddleMan.new_worker(:class => :ping_worker) end def ping_response if request.xhr? msg = MiddleMan.get_worker(session[:job_key]).result.gets render :update do |page| if msg.nil? MiddleMan.delete_worker(session[:job_key]) # ??? need some way to stop the PeriodicalExecuter. else page.insert_html :bottom, 'ping_result', msg+'
' end end else redirect_to :action => 'index' end end [ view ]

<%= periodically_call_remote(:url => {:action => :ping_response}, :frequency => 1) -%> Now if I can just figure out a way to stop the PeriodicalExecuter... [mp] Quoting Charles Brian Quinn : > We're working on a super cool new tool to help automate capistrano and > rake deployment task running on remote servers, and we do something > similiar to this using backgrounDRb (thanks Ezra!). > > It is based off our RailsDay 2006 entry: > http://heartbeat.highgroove.com/ -- enough with the bragging, here is > how we do it: > > your worker now needs to have an @output class variable (instead of > progress). > > in your worker, you can run the task one of two ways which might or > might not work: > > # you can use Kernel.system to make the call which I don't > know how you get output > if not Kernel.system("ping google) > errors.add_to_base(" your error here ") # this is > technically in a model hence the errors obj > raise > end > > # or: > cmd_to_run = "ping google" > > stdin, stdout, stderr = Open3.popen3(cmd_to_run) > > s_stderr = stderr.read.to_s > s_stdout = stdout.read.to_s > > # if you'd like to see it > logger.info "stdout: #{s_stdout}" > logger.info "stderr: #{s_stderr}" > > if s_stderr.empty? && s_stdout.starts_with?("error stuff here") > return true > else > errors.add_to_base("error msg here") > return false > end > > But, as for setting your @output, well, you'd need to run inside a > loop or perhaps even in another thread within the bg worker thread. > We don't need to do this because we're executing our task using ssh > and we have an open channel and just do @output << in the loop as it > runs. You may not be able to use the popen3 calls above, to be honest > -- the read pretty much kills that idea. Our worker method that > appends output looks like: > > Net::SSH.start( host, > :username => username, > :password => password, > :port => port ) do |session| > session.open_channel do |channel| > > channel.on_data do |ch, data| > @output << "#{data}" > end > > channel.on_extended_data do |ch, type, data| > @output << "#{data}" > end > > channel.exec( "ping google\n" ) > end > session.loop > end > > Maybe you can find a simliar asynchronous call for executing methods, > that updates output as it runs. > > Now, in your controller: > > the method, let's say execute: > > def execute > session[:job_key] = Middleman.... # call your worker > end > > and for the view we used rjs to update a page (a lightbox actually): > >

> Connecting to Server... >

> > execute.rjs: > > page << "showBG();" > page << "showIbox('#{url_for(:action => 'output', :id => > @task)}','',parseQuery('width=760&height=400'))" > page << "output = new > Ajax.PeriodicalUpdater('task_output','#{url_for(:action => > 'update_output', :id=> @task.id)}',2);" > > back to your controller, it can be as simple as: > > def update_output > @job = MiddleMan.get_worker(session[:job_key]) > render :inline => '<%= simple_format(@job.output) %>' > end > > The hard part will be appending @output as it's running. I know I > didn't answer the @output side, but hopefully helped with the > backgroundrb worker side. > > cheers and good luck, let us know how it goes. > > On 8/16/06, mattp at digimonkey.com wrote: >> Dunno if anyone saw the short question at the end of my last thread, >> so i'm reposting in a thread with the appropriate subject: >> >> Here's what I need to do -- >> >> Run a system-level command like 'ping -t www.google.com' and have it >> periodically update a

with the command's ouput. Is this possible? >> >> (the trick here is that I want *periodic* updates of the div... i want >> to see incremental output as 'ping' is running, just like if I were >> looking at console...) >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >> > > > -- > Charles Brian Quinn > self-promotion: www.seebq.com > highgroove studios: www.highgroove.com > slingshot hosting: www.slingshothosting.com From ezmobius at gmail.com Wed Aug 16 19:02:28 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Wed, 16 Aug 2006 16:02:28 -0700 Subject: [Backgroundrb-devel] "tailing" a running system command? In-Reply-To: <20060816164723.eqpn58tzqa04sggk@digimonkey.com> References: <4F966DB6-838C-4E83-9B9B-AF4C63299963@gmail.com> <20060816122506.1w2qkrus7k7ocgck@digimonkey.com> <28FB7167-3BE4-4E8D-8E27-F7B50FBAB7B0@gmail.com> <20060816124803.jnxb7v6jb48wck0w@digimonkey.com> <20060816151444.yc5xuiao9yysggso@digimonkey.com> <3a2de0cd0608161419j64b1b321j1360157740b2e0bf@mail.gmail.com> <20060816164723.eqpn58tzqa04sggk@digimonkey.com> Message-ID: <5679F401-7AA7-4D55-B133-820EA0FA3069@brainspl.at> Very cool! I finally figured out how to stop a periodically_call_remote via rjs or js. Do it like this, in the view where you render your periodic call: <%= periodically_call_remote(:url => {:action => 'update_results'}, :frequency => 5, :condition => "stop_polling == false") %> Stop Polling #Or if you need to tell it to stop from the controller via rjs: def stop_polling render :update { |p| p.assign 'stop_polling', true } end And thats it, the periodic call will stop polling once the js variable stop_polling is set to true. Of course you could also just set it to true in the view but it depends on where you get the info that tells you to stop polling. Cheers- -Ezra On Aug 16, 2006, at 2:47 PM, mattp at digimonkey.com wrote: > Check it out... I got it working: > > [ worker ] > class PingWorker < BackgrounDRb::Rails > attr_accessor :result > > def do_work(args) > @result = IO.popen("ping -n 10 www.google.com") > end > end > > [ controller ] > def ping_task > session[:job_key] = MiddleMan.new_worker(:class => :ping_worker) > end > > def ping_response > if request.xhr? > msg = MiddleMan.get_worker(session[:job_key]).result.gets > render :update do |page| > if msg.nil? > MiddleMan.delete_worker(session[:job_key]) > # ??? need some way to stop the PeriodicalExecuter. > else > page.insert_html :bottom, 'ping_result', msg+'
' > end > end > else > redirect_to :action => 'index' > end > end > > [ view ] >

> <%= periodically_call_remote(:url => {:action => :ping_response}, > :frequency => 1) -%> > > > Now if I can just figure out a way to stop the PeriodicalExecuter... > > [mp] > > Quoting Charles Brian Quinn : > >> We're working on a super cool new tool to help automate capistrano >> and >> rake deployment task running on remote servers, and we do something >> similiar to this using backgrounDRb (thanks Ezra!). >> >> It is based off our RailsDay 2006 entry: >> http://heartbeat.highgroove.com/ -- enough with the bragging, here is >> how we do it: >> >> your worker now needs to have an @output class variable (instead of >> progress). >> >> in your worker, you can run the task one of two ways which might or >> might not work: >> >> # you can use Kernel.system to make the call which I don't >> know how you get output >> if not Kernel.system("ping google) >> errors.add_to_base(" your error here ") # this is >> technically in a model hence the errors obj >> raise >> end >> >> # or: >> cmd_to_run = "ping google" >> >> stdin, stdout, stderr = Open3.popen3(cmd_to_run) >> >> s_stderr = stderr.read.to_s >> s_stdout = stdout.read.to_s >> >> # if you'd like to see it >> logger.info "stdout: #{s_stdout}" >> logger.info "stderr: #{s_stderr}" >> >> if s_stderr.empty? && s_stdout.starts_with?("error stuff >> here") >> return true >> else >> errors.add_to_base("error msg here") >> return false >> end >> >> But, as for setting your @output, well, you'd need to run inside a >> loop or perhaps even in another thread within the bg worker thread. >> We don't need to do this because we're executing our task using ssh >> and we have an open channel and just do @output << in the loop as it >> runs. You may not be able to use the popen3 calls above, to be >> honest >> -- the read pretty much kills that idea. Our worker method that >> appends output looks like: >> >> Net::SSH.start( host, >> :username => username, >> :password => password, >> :port => port ) do |session| >> session.open_channel do |channel| >> >> channel.on_data do |ch, data| >> @output << "#{data}" >> end >> >> channel.on_extended_data do |ch, type, data| >> @output << "#{data}" >> end >> >> channel.exec( "ping google\n" ) >> end >> session.loop >> end >> >> Maybe you can find a simliar asynchronous call for executing methods, >> that updates output as it runs. >> >> Now, in your controller: >> >> the method, let's say execute: >> >> def execute >> session[:job_key] = Middleman.... # call your worker >> end >> >> and for the view we used rjs to update a page (a lightbox actually): >> >>

>> Connecting to Server... >>

>> >> execute.rjs: >> >> page << "showBG();" >> page << "showIbox('#{url_for(:action => 'output', :id => >> @task)}','',parseQuery('width=760&height=400'))" >> page << "output = new >> Ajax.PeriodicalUpdater('task_output','#{url_for(:action => >> 'update_output', :id=> @task.id)}',2);" >> >> back to your controller, it can be as simple as: >> >> def update_output >> @job = MiddleMan.get_worker(session[:job_key]) >> render :inline => '<%= simple_format(@job.output) %>' >> end >> >> The hard part will be appending @output as it's running. I know I >> didn't answer the @output side, but hopefully helped with the >> backgroundrb worker side. >> >> cheers and good luck, let us know how it goes. >> >> On 8/16/06, mattp at digimonkey.com wrote: >>> Dunno if anyone saw the short question at the end of my last thread, >>> so i'm reposting in a thread with the appropriate subject: >>> >>> Here's what I need to do -- >>> >>> Run a system-level command like 'ping -t www.google.com' and have it >>> periodically update a

with the command's ouput. Is this >>> possible? >>> >>> (the trick here is that I want *periodic* updates of the div... i >>> want >>> to see incremental output as 'ping' is running, just like if I were >>> looking at console...) >>> _______________________________________________ >>> Backgroundrb-devel mailing list >>> Backgroundrb-devel at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >>> >> >> >> -- >> Charles Brian Quinn >> self-promotion: www.seebq.com >> highgroove studios: www.highgroove.com >> slingshot hosting: www.slingshothosting.com > > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > From mattp at digimonkey.com Wed Aug 16 19:15:34 2006 From: mattp at digimonkey.com (mattp at digimonkey.com) Date: Wed, 16 Aug 2006 18:15:34 -0500 Subject: [Backgroundrb-devel] "tailing" a running system command? In-Reply-To: <5679F401-7AA7-4D55-B133-820EA0FA3069@brainspl.at> References: <4F966DB6-838C-4E83-9B9B-AF4C63299963@gmail.com> <20060816122506.1w2qkrus7k7ocgck@digimonkey.com> <28FB7167-3BE4-4E8D-8E27-F7B50FBAB7B0@gmail.com> <20060816124803.jnxb7v6jb48wck0w@digimonkey.com> <20060816151444.yc5xuiao9yysggso@digimonkey.com> <3a2de0cd0608161419j64b1b321j1360157740b2e0bf@mail.gmail.com> <20060816164723.eqpn58tzqa04sggk@digimonkey.com> <5679F401-7AA7-4D55-B133-820EA0FA3069@brainspl.at> Message-ID: <20060816181534.02z5o6bo7s0kw08w@digimonkey.com> Nice. Based on your suggestion, I also found a simpler way (for me at least). [controller] def ping_response if request.xhr? msg = MiddleMan.get_worker(session[:job_key]).result.gets render :update do |page| if msg.nil? MiddleMan.delete_worker(session[:job_key]) page.remove('spinner') page.call('stop_xhr') # <=== LOOK HERE! THIS IS THE FANCY BIT!! else page.insert_html :bottom, 'ping_result', msg+'
' end end else redirect_to :action => 'index' end end [view] <%= periodically_call_remote(:url => {:action => :ping_response}, :frequency => 0.5, :condition => "stop_polling == false" ) -%> Quoting Ezra Zygmuntowicz : > Very cool! I finally figured out how to stop a > periodically_call_remote via rjs or js. Do it like this, in the view > where you render your periodic call: > > > > <%= periodically_call_remote(:url => {:action => 'update_results'}, > :frequency => 5, > :condition => "stop_polling == false") %> > > Stop Polling > > > #Or if you need to tell it to stop from the controller via rjs: > > def stop_polling > render :update { |p| p.assign 'stop_polling', true } > end > > > > And thats it, the periodic call will stop polling once the js variable > stop_polling is set to true. Of course you could also just set it to > true in the view but it depends on where you get the info that tells > you to stop polling. > > > Cheers- > -Ezra > > > On Aug 16, 2006, at 2:47 PM, mattp at digimonkey.com wrote: > >> Check it out... I got it working: >> >> [ worker ] >> class PingWorker < BackgrounDRb::Rails >> attr_accessor :result >> >> def do_work(args) >> @result = IO.popen("ping -n 10 www.google.com") >> end >> end >> >> [ controller ] >> def ping_task >> session[:job_key] = MiddleMan.new_worker(:class => :ping_worker) >> end >> >> def ping_response >> if request.xhr? >> msg = MiddleMan.get_worker(session[:job_key]).result.gets >> render :update do |page| >> if msg.nil? >> MiddleMan.delete_worker(session[:job_key]) >> # ??? need some way to stop the PeriodicalExecuter. >> else >> page.insert_html :bottom, 'ping_result', msg+'
' >> end >> end >> else >> redirect_to :action => 'index' >> end >> end >> >> [ view ] >>

>> <%= periodically_call_remote(:url => {:action => :ping_response}, >> :frequency => 1) -%> >> >> >> Now if I can just figure out a way to stop the PeriodicalExecuter... >> >> [mp] >> >> Quoting Charles Brian Quinn : >> >>> We're working on a super cool new tool to help automate capistrano and >>> rake deployment task running on remote servers, and we do something >>> similiar to this using backgrounDRb (thanks Ezra!). >>> >>> It is based off our RailsDay 2006 entry: >>> http://heartbeat.highgroove.com/ -- enough with the bragging, here is >>> how we do it: >>> >>> your worker now needs to have an @output class variable (instead of >>> progress). >>> >>> in your worker, you can run the task one of two ways which might or >>> might not work: >>> >>> # you can use Kernel.system to make the call which I don't >>> know how you get output >>> if not Kernel.system("ping google) >>> errors.add_to_base(" your error here ") # this is >>> technically in a model hence the errors obj >>> raise >>> end >>> >>> # or: >>> cmd_to_run = "ping google" >>> >>> stdin, stdout, stderr = Open3.popen3(cmd_to_run) >>> >>> s_stderr = stderr.read.to_s >>> s_stdout = stdout.read.to_s >>> >>> # if you'd like to see it >>> logger.info "stdout: #{s_stdout}" >>> logger.info "stderr: #{s_stderr}" >>> >>> if s_stderr.empty? && s_stdout.starts_with?("error stuff here") >>> return true >>> else >>> errors.add_to_base("error msg here") >>> return false >>> end >>> >>> But, as for setting your @output, well, you'd need to run inside a >>> loop or perhaps even in another thread within the bg worker thread. >>> We don't need to do this because we're executing our task using ssh >>> and we have an open channel and just do @output << in the loop as it >>> runs. You may not be able to use the popen3 calls above, to be honest >>> -- the read pretty much kills that idea. Our worker method that >>> appends output looks like: >>> >>> Net::SSH.start( host, >>> :username => username, >>> :password => password, >>> :port => port ) do |session| >>> session.open_channel do |channel| >>> >>> channel.on_data do |ch, data| >>> @output << "#{data}" >>> end >>> >>> channel.on_extended_data do |ch, type, data| >>> @output << "#{data}" >>> end >>> >>> channel.exec( "ping google\n" ) >>> end >>> session.loop >>> end >>> >>> Maybe you can find a simliar asynchronous call for executing methods, >>> that updates output as it runs. >>> >>> Now, in your controller: >>> >>> the method, let's say execute: >>> >>> def execute >>> session[:job_key] = Middleman.... # call your worker >>> end >>> >>> and for the view we used rjs to update a page (a lightbox actually): >>> >>>

>>> Connecting to Server... >>>

>>> >>> execute.rjs: >>> >>> page << "showBG();" >>> page << "showIbox('#{url_for(:action => 'output', :id => >>> @task)}','',parseQuery('width=760&height=400'))" >>> page << "output = new >>> Ajax.PeriodicalUpdater('task_output','#{url_for(:action => >>> 'update_output', :id=> @task.id)}',2);" >>> >>> back to your controller, it can be as simple as: >>> >>> def update_output >>> @job = MiddleMan.get_worker(session[:job_key]) >>> render :inline => '<%= simple_format(@job.output) %>' >>> end >>> >>> The hard part will be appending @output as it's running. I know I >>> didn't answer the @output side, but hopefully helped with the >>> backgroundrb worker side. >>> >>> cheers and good luck, let us know how it goes. >>> >>> On 8/16/06, mattp at digimonkey.com wrote: >>>> Dunno if anyone saw the short question at the end of my last thread, >>>> so i'm reposting in a thread with the appropriate subject: >>>> >>>> Here's what I need to do -- >>>> >>>> Run a system-level command like 'ping -t www.google.com' and have it >>>> periodically update a

with the command's ouput. Is this possible? >>>> >>>> (the trick here is that I want *periodic* updates of the div... i want >>>> to see incremental output as 'ping' is running, just like if I were >>>> looking at console...) >>>> _______________________________________________ >>>> Backgroundrb-devel mailing list >>>> Backgroundrb-devel at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >>>> >>> >>> >>> -- >>> Charles Brian Quinn >>> self-promotion: www.seebq.com >>> highgroove studios: www.highgroove.com >>> slingshot hosting: www.slingshothosting.com >> >> >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >> From mark at martianrover.com Thu Aug 17 17:15:54 2006 From: mark at martianrover.com (P. Mark Anderson) Date: Thu, 17 Aug 2006 14:15:54 -0700 Subject: [Backgroundrb-devel] Worker pool worker for page scanners: lost connection to mysql Message-ID: <66578419-55AF-46D1-B99D-A04DDDA649B8@martianrover.com> Hello, I want to scan many URLs simultaneously with a new scanner worker for each URL, but I keep getting Mysql::Error: Lost connection to MySQL. Yes, allow_concurrency=false. Each scanner thread needs access to my ActiveRecord models to store discovered feeds and whatnot. There is a DB table called page_scans that acts as a big queue holding URLs waiting to be scanned. It would be nice to run a worker that spawns up to 10 scanner workers at a time to chip away at the queue, but first I need to figure out what kinds of problems kill the mysql connection in a worker. Any thoughts are much appreciated! Mark From ezmobius at gmail.com Thu Aug 17 18:24:06 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Thu, 17 Aug 2006 15:24:06 -0700 Subject: [Backgroundrb-devel] Worker pool worker for page scanners: lost connection to mysql In-Reply-To: <66578419-55AF-46D1-B99D-A04DDDA649B8@martianrover.com> References: <66578419-55AF-46D1-B99D-A04DDDA649B8@martianrover.com> Message-ID: On Aug 17, 2006, at 2:15 PM, P. Mark Anderson wrote: > Hello, > > I want to scan many URLs simultaneously with a new scanner worker for > each URL, but I keep getting Mysql::Error: Lost connection to MySQL. > Yes, allow_concurrency=false. > > Each scanner thread needs access to my ActiveRecord models to store > discovered feeds and whatnot. There is a DB table called page_scans > that acts as a big queue holding URLs waiting to be scanned. > > It would be nice to run a worker that spawns up to 10 scanner workers > at a time to chip away at the queue, but first I need to figure out > what kinds of problems kill the mysql connection in a worker. > > Any thoughts are much appreciated! > > Mark > > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel Mark-- Have you tried it with allow_concurency = true? Stuff like this is hard because even though Active Record pretends like it is thread safe I don't really think it is. If it doesn't work with allow_concurrency = true then you might need to look at just using the mysql-ruby bindings directly and *gasp* writing some sql just for this worker. Cheers- -Ezra From bjohnson at contuitive.com Fri Aug 18 18:14:08 2006 From: bjohnson at contuitive.com (Ben Johnson) Date: Fri, 18 Aug 2006 17:14:08 -0500 Subject: [Backgroundrb-devel] Ezra....allow_concurrency = false? Message-ID: <135E4B51-50E0-433E-8706-22EB617ED845@contuitive.com> In a previous email you wrote this: Glad you're getting some use out of the plugin ;) ActiveRecord does have some issues with threading and concurrency that can sometimes cause that error you are seeing. Open up your script/backgroundrb/ start file and comment out the following line: ActiveRecord::Base.allow_concurrency = true Make sure you do that up in your main app script folder and not in the plugin. Well you can do it in the plugin as well to keep it consistent. Then you need to restart the drb server. Give this a shot and see if it gets rid of your problem. Also I am curious about using this plugin on textdrive. Did they give you an extra port number to run it on? If you run into problems with running the drb server on a port then let me know and I can show you how to use a unix domain socket file instead so you don't take up an extra port. You wrote that in response to getting rid of the MySQL server has gone away error. Why would you comment it out in the app and not in the start script? Wouldn't you want to comment it out in the plugin start script? Thank You, Ben Johnson E: bjohnson at contuitive.com O: 800-341-6826 M: 817-229-4258 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060818/aeeedef5/attachment.html From ezmobius at gmail.com Fri Aug 18 18:51:46 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Fri, 18 Aug 2006 15:51:46 -0700 Subject: [Backgroundrb-devel] Ezra....allow_concurrency = false? In-Reply-To: <135E4B51-50E0-433E-8706-22EB617ED845@contuitive.com> References: <135E4B51-50E0-433E-8706-22EB617ED845@contuitive.com> Message-ID: On Aug 18, 2006, at 3:14 PM, Ben Johnson wrote: > In a previous email you wrote this: > > Glad you're getting some use out of the plugin ;) ActiveRecord does > have some issues with threading and concurrency that can sometimes > cause that error you are seeing. Open up your script/backgroundrb/ > start file and comment out the following line: > > ActiveRecord::Base.allow_concurrency = true > > Make sure you do that up in your main app script folder and not in > the plugin. Well you can do it in the plugin as well to keep it > consistent. Then you need to restart the drb server. Give this a shot > and see if it gets rid of your problem. Also I am curious about using > this plugin on textdrive. Did they give you an extra port number to > run it on? If you run into problems with running the drb server on a > port then let me know and I can show you how to use a unix domain > socket file instead so you don't take up an extra port. > > You wrote that in response to getting rid of the MySQL server has > gone away error. Why would you comment it out in the app and not in > the start script? Wouldn't you want to comment it out in the plugin > start script? > > Thank You, > Ben Johnson > E: bjohnson at contuitive.com > O: 800-341-6826 > M: 817-229-4258 When you run the rake backgroundrb:setup task it copies the start script up into your apps script directory. So you need to make the change up there. Or you can make the change in the plugin and then rerun the setup task if you like. Either way its the same thing. You need to make the changes to the start script that is in the script/ backgroundrb directory of your rails app in order for them to be used. Does that answer your question? -Ezra -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060818/4988b2ed/attachment.html From bjohnson at contuitive.com Fri Aug 18 21:20:11 2006 From: bjohnson at contuitive.com (Ben Johnson) Date: Fri, 18 Aug 2006 20:20:11 -0500 Subject: [Backgroundrb-devel] Impossible to get stable? Message-ID: <284E7D6B-A5E6-48D1-B912-D7CFFB23DD32@contuitive.com> I have concluded that it is impossible to get this background process stable. I am out of ideas and could really use some help. Here is what I have: 50 workers, just running an infinite loop that constantly calls save! on a model. Keep in mind this is just for testing purposes. I catch all exceptions and put them in the log. I am tried setting allow_concurrency to true and false. Neither of them is stable. For both true and false I have got the following errors: # Is it possible to make the process priority higher for backgroundrb? Thank You, Ben Johnson E: bjohnson at contuitive.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060819/7a2e6d1b/attachment.html From ezmobius at gmail.com Sun Aug 20 00:39:25 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Sat, 19 Aug 2006 21:39:25 -0700 Subject: [Backgroundrb-devel] Impossible to get stable? In-Reply-To: <284E7D6B-A5E6-48D1-B912-D7CFFB23DD32@contuitive.com> References: <284E7D6B-A5E6-48D1-B912-D7CFFB23DD32@contuitive.com> Message-ID: <1FD6BA47-DD3A-4A1C-98FA-0E17BA8B391D@gmail.com> On Aug 18, 2006, at 6:20 PM, Ben Johnson wrote: > I have concluded that it is impossible to get this background > process stable. I am out of ideas and could really use some help. > > Here is what I have: > > 50 workers, just running an infinite loop that constantly calls > save! on a model. Keep in mind this is just for testing purposes. I > catch all exceptions and put them in the log. > > I am tried setting allow_concurrency to true and false. Neither of > them is stable. For both true and false I have got the following > errors: > > # exceeded; try restarting transaction: > > # gone away: > > What have you found to be most stable? Setting allow concurrency to > false and just putting a Mutex.synchronize object around all > database calls? Or setting it true? > > Thank You, > Ben Johnson > E: bjohnson at contuitive.com > O: 800-341-6826 > M: 817-229-4258 Hey Ben- I feel your pain. I have come to the conclusion that ActiveRecord is just not thread safe no matter what you do. My best suggestion is to use either dbi or mysql bindings directly in your workers. These are thread safe and while you would have to write some sql yourself, it would solve your issue. I get the feeling that railscore is not very interested in making rails thread safe at this point and pushing as many db connections in as many threads as you are doing is more the ActiveRecord can handle. I know its not what you want to hear but its the situation we have to deal with. -Ezra -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060819/d73ba4a7/attachment-0001.html From ezmobius at gmail.com Sun Aug 20 00:45:07 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Sat, 19 Aug 2006 21:45:07 -0700 Subject: [Backgroundrb-devel] Setting the process priority? In-Reply-To: <8A7776BA-71E7-4A0F-9D79-23F07429083E@contuitive.com> References: <8A7776BA-71E7-4A0F-9D79-23F07429083E@contuitive.com> Message-ID: <7F5DF5EB-940D-4BE2-90CC-F99BC2C7E8F5@brainspl.at> On Aug 19, 2006, at 3:45 PM, Ben Johnson wrote: > Is it possible to make the process priority higher for backgroundrb? > > > Thank You, > Ben Johnson > E: bjohnson at contuitive.com > > > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel Ben- You mean like the OS level? I think that would be totally dependent on what OS you are running. And I don't think it would be something you could do from within ruby. I think you would have to do it from within your OS. Well you could make some system calls to set the nice level i guess. Also in your case maybe you might want to consider running more then one bgdrb daemon. You could easily seperate the drb servers and run a certain amount of workers in one and more workers in another. This would require some hacking on your part but not too much. You could duplicate the code block in the plugin/lib/backgrondrb.rb file and rename the second one MiddleMan2 or whatever and change the port number. I may add this as an option in a future release. I am also playing with the idea of using Rinda TupleSpace to distribute the workers across multiple processes. This would allow for bigger jobs to be easily processed by multiple workers in their own processes. Let me know what you end up doing. -Ezra -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060819/b6604674/attachment.html From bjohnson at contuitive.com Sun Aug 20 03:27:09 2006 From: bjohnson at contuitive.com (Ben Johnson) Date: Sun, 20 Aug 2006 02:27:09 -0500 Subject: [Backgroundrb-devel] Impossible to get stable? In-Reply-To: <1FD6BA47-DD3A-4A1C-98FA-0E17BA8B391D@gmail.com> References: <284E7D6B-A5E6-48D1-B912-D7CFFB23DD32@contuitive.com> <1FD6BA47-DD3A-4A1C-98FA-0E17BA8B391D@gmail.com> Message-ID: <6E59ED5A-4BC2-409D-97E3-A4A2DE50A522@contuitive.com> Hi Ezra, Thanks a lot for your help. Can you clarify what dbi is or mysql bindings? I am not sure what you are talking about. Thanks again for your help. Thank You, Ben Johnson E: bjohnson at contuitive.com O: 800-341-6826 M: 817-229-4258 On Aug 19, 2006, at 11:39 PM, Ezra Zygmuntowicz wrote: > > On Aug 18, 2006, at 6:20 PM, Ben Johnson wrote: > >> I have concluded that it is impossible to get this background >> process stable. I am out of ideas and could really use some help. >> >> Here is what I have: >> >> 50 workers, just running an infinite loop that constantly calls >> save! on a model. Keep in mind this is just for testing purposes. >> I catch all exceptions and put them in the log. >> >> I am tried setting allow_concurrency to true and false. Neither of >> them is stable. For both true and false I have got the following >> errors: >> >> #> exceeded; try restarting transaction: >> >> #> gone away: >> >> What have you found to be most stable? Setting allow concurrency >> to false and just putting a Mutex.synchronize object around all >> database calls? Or setting it true? >> >> Thank You, >> Ben Johnson >> E: bjohnson at contuitive.com >> O: 800-341-6826 >> M: 817-229-4258 > > > Hey Ben- > > I feel your pain. I have come to the conclusion that ActiveRecord > is just not thread safe no matter what you do. My best suggestion > is to use either dbi or mysql bindings directly in your workers. > These are thread safe and while you would have to write some sql > yourself, it would solve your issue. I get the feeling that > railscore is not very interested in making rails thread safe at > this point and pushing as many db connections in as many threads as > you are doing is more the ActiveRecord can handle. > > I know its not what you want to hear but its the situation we have > to deal with. > > -Ezra -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060820/9251d721/attachment.html From info at siebert-wd.de Sun Aug 20 03:36:07 2006 From: info at siebert-wd.de (Michael Siebert) Date: Sun, 20 Aug 2006 09:36:07 +0200 Subject: [Backgroundrb-devel] Impossible to get stable? In-Reply-To: <6E59ED5A-4BC2-409D-97E3-A4A2DE50A522@contuitive.com> References: <284E7D6B-A5E6-48D1-B912-D7CFFB23DD32@contuitive.com> <1FD6BA47-DD3A-4A1C-98FA-0E17BA8B391D@gmail.com> <6E59ED5A-4BC2-409D-97E3-A4A2DE50A522@contuitive.com> Message-ID: <32b567370608200036x661aa9b1kcdfd58a53ab5d375@mail.gmail.com> dbi and mysql is the basic layer of database access. its just a way to connect to a db server and send a query. no modelous abstractions, no stuff around, just tailor your sql like back in (good) old php days. thats surely not the rails way and uncomfortable but it should give no errors (except that your db only can handle a certain amount of connections) 2006/8/20, Ben Johnson : > > Hi Ezra, > Thanks a lot for your help. Can you clarify what dbi is or mysql bindings? > I am not sure what you are talking about. > > Thanks again for your help. > > Thank You, > Ben Johnson > E: bjohnson at contuitive.com > O: 800-341-6826 > M: 817-229-4258 > > > > On Aug 19, 2006, at 11:39 PM, Ezra Zygmuntowicz wrote: > > > On Aug 18, 2006, at 6:20 PM, Ben Johnson wrote: > > I have concluded that it is impossible to get this background process > stable. I am out of ideas and could really use some help. > Here is what I have: > > 50 workers, just running an infinite loop that constantly calls save! on a > model. Keep in mind this is just for testing purposes. I catch all > exceptions and put them in the log. > > I am tried setting allow_concurrency to true and false. Neither of them is > stable. For both true and false I have got the following errors: > > # exceeded; try restarting transaction: > > # away: > > What have you found to be most stable? Setting allow concurrency to false > and just putting a Mutex.synchronize object around all database calls? Or > setting it true? > > Thank You, > Ben Johnson > E: bjohnson at contuitive.com > O: 800-341-6826 > M: 817-229-4258 > > > > Hey Ben- > > I feel your pain. I have come to the conclusion that ActiveRecord is just > not thread safe no matter what you do. My best suggestion is to use either > dbi or mysql bindings directly in your workers. These are thread safe and > while you would have to write some sql yourself, it would solve your issue. > I get the feeling that railscore is not very interested in making rails > thread safe at this point and pushing as many db connections in as many > threads as you are doing is more the ActiveRecord can handle. > > I know its not what you want to hear but its the situation we have to deal > with. > > -Ezra > > > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > -- Michael Siebert www.siebert-wd.de - Gedanken lesen www.stellar-legends.de - Weltraum-Browsergame im Alpha-Stadium -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060820/ed5cf82a/attachment.html From eimorton at gmail.com Sun Aug 20 08:20:45 2006 From: eimorton at gmail.com (Erik Morton) Date: Sun, 20 Aug 2006 08:20:45 -0400 Subject: [Backgroundrb-devel] Impossible to get stable? In-Reply-To: <1FD6BA47-DD3A-4A1C-98FA-0E17BA8B391D@gmail.com> References: <284E7D6B-A5E6-48D1-B912-D7CFFB23DD32@contuitive.com> <1FD6BA47-DD3A-4A1C-98FA-0E17BA8B391D@gmail.com> Message-ID: <2D2D37F4-5FE5-4C96-9A3C-A69C64F8D3AA@gmail.com> I'm planning on using Backgroundrb to run background processes for a fairly large all-Rails application that is launching in two weeks. I've been getting a bit paranoid about how ActiveRecord would react with possibly as many as twenty workers chugging away at the same time on the same tables. I put together a dead-simple "test" and the results were, I think, very positive. (The OS is OSX 10.4.7) Setup: One worker class that queries a MySQL database (4.1.18) n times and then calls ActiveRecord::Base.connection.disconnect! and kill. The worker executes the "SHOW PROCESSLIST" query, which returns a list of active MySQL processes, and logs the number of active connections along with the to_s of the connection and the thread. The test script uses MiddleMan to instantiate 100 workers passing in the worker's number as the number (10, 11, 12...110) of times the worker should loop and query the database. I took this approach so that the workers would be closing connections while other threads had connections open. Goals: 1) verify that each Thread has his own connection. 2) verify that each worker can call disconnect! to close their own connection to free it up for other workers. Here's a snippet of the log file. worker number, ActiveRecord::Base.object_id, ActiveRecord::Base.connection.to_s, Thread.current.to_s, # of active MySQL connections 25, 3705192, #, #, 12 21, 3705192, #, #, 12 18, 3705192, #, #, 12 17, 3705192, #, #, 12 16, 3705192, #, #, 12 15, 3705192, #, #, 12 23, 3705192, #, #, 11 In this rather unscientific way I have positively confirmed (I think) that on Mac OSX 10.4.7, MySQL 4.1.18 and AR 1.14.4, each worker gets his own thread with his own connection, and can freely close his connection without adversely affecting any other thread. Any thoughts on this approach? I made one small change to swap out the SHOW PROCESS list query with MyModel.find :first and the results were the same -- each worker had his own thread with his own connection that he can close without affecting any other threads. Finally, I altered the start script to set allow concurrency to false, ran the test and not surprisingly all heck broke loose: 39, 3705192, #, #, 1 25, 3705192, #, #, 1 86, 3705192,#,#, 54, 3705192, #, #, 1 87, 3705192,#,#, Mysql::Error: MySQL server has gone away: SHOW PROCESSLIST; - (ActiveRecord::StatementInvalid) /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/ active_record/connection_adapters/abstract_adapter.rb:120:in `log' /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/ active_record/connection_adapters/mysql_adapter.rb:184:in `execute' Different threads but the same connection and the first worker that closes his connection ruins the whole party. Are results like the above consistent with the intention of Backgroundrb? Is my approach was at least somewhat valid? Erik On Aug 20, 2006, at 12:39 AM, Ezra Zygmuntowicz wrote: > > On Aug 18, 2006, at 6:20 PM, Ben Johnson wrote: > >> I have concluded that it is impossible to get this background >> process stable. I am out of ideas and could really use some help. >> >> Here is what I have: >> >> 50 workers, just running an infinite loop that constantly calls >> save! on a model. Keep in mind this is just for testing purposes. >> I catch all exceptions and put them in the log. >> >> I am tried setting allow_concurrency to true and false. Neither of >> them is stable. For both true and false I have got the following >> errors: >> >> #> exceeded; try restarting transaction: >> >> #> gone away: >> >> What have you found to be most stable? Setting allow concurrency >> to false and just putting a Mutex.synchronize object around all >> database calls? Or setting it true? >> >> Thank You, >> Ben Johnson >> E: bjohnson at contuitive.com >> O: 800-341-6826 >> M: 817-229-4258 > > > Hey Ben- > > I feel your pain. I have come to the conclusion that ActiveRecord > is just not thread safe no matter what you do. My best suggestion > is to use either dbi or mysql bindings directly in your workers. > These are thread safe and while you would have to write some sql > yourself, it would solve your issue. I get the feeling that > railscore is not very interested in making rails thread safe at > this point and pushing as many db connections in as many threads as > you are doing is more the ActiveRecord can handle. > > I know its not what you want to hear but its the situation we have > to deal with. > > -Ezra > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel From ezmobius at gmail.com Sun Aug 20 14:41:49 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Sun, 20 Aug 2006 11:41:49 -0700 Subject: [Backgroundrb-devel] Impossible to get stable? In-Reply-To: <2D2D37F4-5FE5-4C96-9A3C-A69C64F8D3AA@gmail.com> References: <284E7D6B-A5E6-48D1-B912-D7CFFB23DD32@contuitive.com> <1FD6BA47-DD3A-4A1C-98FA-0E17BA8B391D@gmail.com> <2D2D37F4-5FE5-4C96-9A3C-A69C64F8D3AA@gmail.com> Message-ID: <8715E247-7DF9-46FE-8AC3-43BCA608A291@gmail.com> Hey Eric- Yeah I think your approach is valid. Would you mind sending me the code you used for this experiment? I would like to run some tests myself. I will add some unit tests to try and see what the upper limits of threaded AR usage is. Thanks- -Ezra On Aug 20, 2006, at 5:20 AM, Erik Morton wrote: > I'm planning on using Backgroundrb to run background processes for a > fairly large all-Rails application that is launching in two weeks. > I've been getting a bit paranoid about how ActiveRecord would react > with possibly as many as twenty workers chugging away at the same > time on the same tables. I put together a dead-simple "test" and the > results were, I think, very positive. (The OS is OSX 10.4.7) > > Setup: One worker class that queries a MySQL database (4.1.18) n > times and then calls ActiveRecord::Base.connection.disconnect! and > kill. The worker executes the "SHOW PROCESSLIST" query, which returns > a list of active MySQL processes, and logs the number of active > connections along with the to_s of the connection and the thread. > > The test script uses MiddleMan to instantiate 100 workers passing in > the worker's number as the number (10, 11, 12...110) of times the > worker should loop and query the database. I took this approach so > that the workers would be closing connections while other threads had > connections open. > > Goals: 1) verify that each Thread has his own connection. 2) verify > that each worker can call disconnect! to close their own connection > to free it up for other workers. > > Here's a snippet of the log file. > > worker number, ActiveRecord::Base.object_id, > ActiveRecord::Base.connection.to_s, Thread.current.to_s, # of active > MySQL connections > 25, 3705192, # 0x277ba04>, #, 12 > 21, 3705192, # 0x277a730>, #, 12 > 18, 3705192, # 0x26c86e8>, #, 12 > 17, 3705192, # 0x26d61d0>, #, 12 > 16, 3705192, # 0x26e06d0>, #, 12 > 15, 3705192, # 0x26e7098>, #, 12 > 23, 3705192, # 0x2784f8c>, #, 11 > > In this rather unscientific way I have positively confirmed (I think) > that on Mac OSX 10.4.7, MySQL 4.1.18 and AR 1.14.4, each worker gets > his own thread with his own connection, and can freely close his > connection without adversely affecting any other thread. Any thoughts > on this approach? > > I made one small change to swap out the SHOW PROCESS list query with > MyModel.find :first and the results were the same -- each worker had > his own thread with his own connection that he can close without > affecting any other threads. > > Finally, I altered the start script to set allow concurrency to > false, ran the test and not surprisingly all heck broke loose: > > 39, 3705192, # 0x26fdca8>, #, 1 > 25, 3705192, # 0x26fdca8>, #, 1 > 86, 3705192,# 0x26fdca8>,#, > 54, 3705192, # 0x26fdca8>, #, 1 > 87, 3705192,# 0x26fdca8>,#, > Mysql::Error: MySQL server has gone away: SHOW PROCESSLIST; - > (ActiveRecord::StatementInvalid) > /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/ > active_record/connection_adapters/abstract_adapter.rb:120:in `log' > /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/ > active_record/connection_adapters/mysql_adapter.rb:184:in `execute' > > Different threads but the same connection and the first worker that > closes his connection ruins the whole party. > > Are results like the above consistent with the intention of > Backgroundrb? Is my approach was at least somewhat valid? > > Erik > > On Aug 20, 2006, at 12:39 AM, Ezra Zygmuntowicz wrote: > >> >> On Aug 18, 2006, at 6:20 PM, Ben Johnson wrote: >> >>> I have concluded that it is impossible to get this background >>> process stable. I am out of ideas and could really use some help. >>> >>> Here is what I have: >>> >>> 50 workers, just running an infinite loop that constantly calls >>> save! on a model. Keep in mind this is just for testing purposes. >>> I catch all exceptions and put them in the log. >>> >>> I am tried setting allow_concurrency to true and false. Neither of >>> them is stable. For both true and false I have got the following >>> errors: >>> >>> #>> exceeded; try restarting transaction: >>> >>> #>> gone away: >>> >>> What have you found to be most stable? Setting allow concurrency >>> to false and just putting a Mutex.synchronize object around all >>> database calls? Or setting it true? >>> >>> Thank You, >>> Ben Johnson >>> E: bjohnson at contuitive.com >>> O: 800-341-6826 >>> M: 817-229-4258 >> >> >> Hey Ben- >> >> I feel your pain. I have come to the conclusion that ActiveRecord >> is just not thread safe no matter what you do. My best suggestion >> is to use either dbi or mysql bindings directly in your workers. >> These are thread safe and while you would have to write some sql >> yourself, it would solve your issue. I get the feeling that >> railscore is not very interested in making rails thread safe at >> this point and pushing as many db connections in as many threads as >> you are doing is more the ActiveRecord can handle. >> >> I know its not what you want to hear but its the situation we have >> to deal with. >> >> -Ezra >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > From lapomme00 at gmail.com Sun Aug 20 22:37:46 2006 From: lapomme00 at gmail.com (Brett Walker) Date: Mon, 21 Aug 2006 10:37:46 +0800 Subject: [Backgroundrb-devel] Recommended start order when deploying with Capistrano... Message-ID: When deploying with Capistrano, what's the recommended start ordering for BackgrounDRb? mongrel_stop backgroundrb_restart mongrel_start - or - mongrel_restart backgroundrb_restart - or - backgroundrb_restart mongrel_restart Or does it matter? Cheers, Brett From ezmobius at gmail.com Sun Aug 20 23:02:15 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Sun, 20 Aug 2006 20:02:15 -0700 Subject: [Backgroundrb-devel] Recommended start order when deploying with Capistrano... In-Reply-To: References: Message-ID: <83A84CD6-9009-4725-8AB1-6AD30C9CC7FF@brainspl.at> On Aug 20, 2006, at 7:37 PM, Brett Walker wrote: > When deploying with Capistrano, what's the recommended start ordering > for BackgrounDRb? > > mongrel_stop > backgroundrb_restart > mongrel_start > > - or - > > mongrel_restart > backgroundrb_restart > > - or - > > backgroundrb_restart > mongrel_restart > > > Or does it matter? > > Cheers, > Brett Brett- You need to start the drb server before the mongrels start or restart. The drb server needs to be running when rails boots up so the MiddleMan can conect to the drb server -Ezra From gethemant at gmail.com Mon Aug 21 01:24:47 2006 From: gethemant at gmail.com (hemant) Date: Mon, 21 Aug 2006 10:54:47 +0530 Subject: [Backgroundrb-devel] strange problem with system call Message-ID: Hi, I am using backgroundrb to create a automated system of typo blogs.So, users can create new typo applications using web interface and all that jazz. Now..its working perfectly except one minor glitch. Since latest version of typo has a command called "typo", i simply use that command to install new typo app.And it works. def do_work(args) DIR = "/home/foobar/public_html" username = args blog_name = username + "_blog" # creation part works flawlessly so lets skip that. # there is a problem while stopping it. if (Dir.chdir(DIR)) == 0 current_dir = `pwd` @logger.debug("Current directory is #{current_dir}") stop_command = "typo stop "+ blog_name debug_stuff = `#{stop_command}` #line no 33 @logger.debug("### #{debug_stuff}") else @logger.debug("### Error changing directory") end end Now let me assure that, args has correct username, because starting of blog,migrating from sqlite to mysql works perfectly using this worker thread.The only glitch is, in line # 33, "typo stop " doesn't work.I have tried system also.system returns true, so typo should be stopped.But when i do "ps aux|grep foobar" , it isn't ,process is still there. When i tried `command`. All i got in debug_stuff variable is a empty string.However, when i try same stuff from irb, it works as expected and typo stopped.Now..again let me assure that, the worker thread IS getting CALLED. I have tried various stuff.But since, I am simply not able to debug, where the heck problem is, I am lost. I was told that, system or `command` both wait for the command to finish execution to resume normal execution.But for me it SEEMS that the call to "typo stop #{blog_name}" returns immediately. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060821/84d21e30/attachment.html From me at seebq.com Mon Aug 21 11:29:21 2006 From: me at seebq.com (Charles Brian Quinn) Date: Mon, 21 Aug 2006 11:29:21 -0400 Subject: [Backgroundrb-devel] strange problem with system call In-Reply-To: References: Message-ID: <3a2de0cd0608210829l317170f3j4e71ca44fd18ec6f@mail.gmail.com> Many times those calls to start/stop scripts get backgrounded, so it could actually be returning instantly. You could try wrapping it, or go check what typo stop is actually doing and make sure there's no "blah blah &" in the script. You could also try using popen and popen3 processes instead of the `command` backticks. I find those hard to debug as you don't know what's going on when they run. This will allow you to see stdin, stderr, and stdout on the process. Sample: cmd_to_run = "typo stop" stdin, stdout, stderr = Open3.popen3(cmd_to_run) s_stderr = stderr.read.to_s s_stdout = stdout.read.to_s # if you'd like to see it logger.info "stdout: #{s_stdout}" logger.info "stderr: #{s_stderr}" if s_stderr.empty? && s_stdout.starts_with?("error stuff here") return true else errors.add_to_base("error msg here") return false end hope that helps! On 8/21/06, hemant wrote: > Hi, > > I am using backgroundrb to create a automated system of typo blogs.So, users > can create new typo applications using web interface and all that jazz. > > Now..its working perfectly except one minor glitch. Since latest version of > typo has a command called "typo", i simply use that command to install new > typo app.And it works. > > def do_work(args) > DIR = "/home/foobar/public_html" > username = args > blog_name = username + "_blog" > # creation part works flawlessly so lets skip that. > # there is a problem while stopping it. > if (Dir.chdir(DIR)) == 0 > current_dir = `pwd` > @logger.debug("Current directory is #{current_dir}") > stop_command = "typo stop "+ blog_name > debug_stuff = `#{stop_command}` > #line no 33 > @logger.debug("### #{debug_stuff}") > else > @logger.debug("### Error changing directory") > end > end > Now let me assure that, args has correct username, because starting of > blog,migrating from sqlite to mysql works perfectly using this worker > thread.The only glitch is, in line # 33, "typo stop " doesn't work.I have > tried system also.system returns true, so typo should be stopped.But when i > do "ps aux|grep foobar" , it isn't ,process is still there. > > When i tried `command`. All i got in debug_stuff variable is a empty > string.However, when i try same stuff from irb, it works as expected and > typo stopped.Now..again let me assure that, the worker thread IS getting > CALLED. > > I have tried various stuff.But since, I am simply not able to debug, where > the heck problem is, I am lost. I was told that, system or `command` both > wait for the command to finish execution to resume normal execution.But for > me it SEEMS that the call to "typo stop #{blog_name}" returns immediately. > > > > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > -- Charles Brian Quinn self-promotion: www.seebq.com highgroove studios: www.highgroove.com slingshot hosting: www.slingshothosting.com From ezmobius at gmail.com Mon Aug 21 11:38:34 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Mon, 21 Aug 2006 08:38:34 -0700 Subject: [Backgroundrb-devel] strange problem with system call In-Reply-To: <3a2de0cd0608210829l317170f3j4e71ca44fd18ec6f@mail.gmail.com> References: <3a2de0cd0608210829l317170f3j4e71ca44fd18ec6f@mail.gmail.com> Message-ID: <7AD41E49-92B6-4615-85B7-593615BC26FB@gmail.com> Yeah Charles has the right idea. One other way to do what you want would be to kill the process directly by running pid = `ps axww | grep -v grep | grep typo` # or whatever the process is called? then get the pid out of that text and do a kill -9 on it. But I think you are probably better off with a popen solution. I am going to be redoing the start scripts to use the daemons gem which might fix this problem. -Ezra On Aug 21, 2006, at 8:29 AM, Charles Brian Quinn wrote: > Many times those calls to start/stop scripts get backgrounded, so it > could actually be returning instantly. You could try wrapping it, or > go check what typo stop is actually doing and make sure there's no > "blah blah &" in the script. > > You could also try using popen and popen3 processes instead of the > `command` backticks. I find those hard to debug as you don't know > what's going on when they run. > > This will allow you to see stdin, stderr, and stdout on the > process. Sample: > > cmd_to_run = "typo stop" > > stdin, stdout, stderr = Open3.popen3(cmd_to_run) > > s_stderr = stderr.read.to_s > s_stdout = stdout.read.to_s > > # if you'd like to see it > logger.info "stdout: #{s_stdout}" > logger.info "stderr: #{s_stderr}" > > if s_stderr.empty? && s_stdout.starts_with?("error stuff here") > return true > else > errors.add_to_base("error msg here") > return false > end > > hope that helps! > > On 8/21/06, hemant wrote: >> Hi, >> >> I am using backgroundrb to create a automated system of typo >> blogs.So, users >> can create new typo applications using web interface and all that >> jazz. >> >> Now..its working perfectly except one minor glitch. Since latest >> version of >> typo has a command called "typo", i simply use that command to >> install new >> typo app.And it works. >> >> def do_work(args) >> DIR = "/home/foobar/public_html" >> username = args >> blog_name = username + "_blog" >> # creation part works flawlessly so lets skip that. >> # there is a problem while stopping it. >> if (Dir.chdir(DIR)) == 0 >> current_dir = `pwd` >> @logger.debug("Current directory is #{current_dir}") >> stop_command = "typo stop "+ blog_name >> debug_stuff = `#{stop_command}` >> #line no 33 >> @logger.debug("### #{debug_stuff}") >> else >> @logger.debug("### Error changing directory") >> end >> end >> Now let me assure that, args has correct username, because >> starting of >> blog,migrating from sqlite to mysql works perfectly using this worker >> thread.The only glitch is, in line # 33, "typo stop " doesn't >> work.I have >> tried system also.system returns true, so typo should be >> stopped.But when i >> do "ps aux|grep foobar" , it isn't ,process is still there. >> >> When i tried `command`. All i got in debug_stuff variable is a empty >> string.However, when i try same stuff from irb, it works as >> expected and >> typo stopped.Now..again let me assure that, the worker thread IS >> getting >> CALLED. >> >> I have tried various stuff.But since, I am simply not able to >> debug, where >> the heck problem is, I am lost. I was told that, system or >> `command` both >> wait for the command to finish execution to resume normal >> execution.But for >> me it SEEMS that the call to "typo stop #{blog_name}" returns >> immediately. >> >> >> >> >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >> >> > > > -- > Charles Brian Quinn > self-promotion: www.seebq.com > highgroove studios: www.highgroove.com > slingshot hosting: www.slingshothosting.com > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel From gethemant at gmail.com Mon Aug 21 17:00:06 2006 From: gethemant at gmail.com (hemant) Date: Tue, 22 Aug 2006 02:30:06 +0530 Subject: [Backgroundrb-devel] strange problem with system call In-Reply-To: <7AD41E49-92B6-4615-85B7-593615BC26FB@gmail.com> References: <3a2de0cd0608210829l317170f3j4e71ca44fd18ec6f@mail.gmail.com> <7AD41E49-92B6-4615-85B7-593615BC26FB@gmail.com> Message-ID: Thanks guys, I dont have any "&" in my code.But yes, popen solution sounds better. I will give it a spin. On 8/21/06, Ezra Zygmuntowicz wrote: > > > Yeah Charles has the right idea. One other way to do what you want > would be to kill the process directly by running > > pid = `ps axww | grep -v grep | grep typo` # or whatever the process > is called? > > then get the pid out of that text and do a kill -9 on it. But I think > you are probably better off with a popen solution. I am going to be > redoing the start scripts to use the daemons gem which might fix this > problem. > > -Ezra > > On Aug 21, 2006, at 8:29 AM, Charles Brian Quinn wrote: > > > Many times those calls to start/stop scripts get backgrounded, so it > > could actually be returning instantly. You could try wrapping it, or > > go check what typo stop is actually doing and make sure there's no > > "blah blah &" in the script. > > > > You could also try using popen and popen3 processes instead of the > > `command` backticks. I find those hard to debug as you don't know > > what's going on when they run. > > > > This will allow you to see stdin, stderr, and stdout on the > > process. Sample: > > > > cmd_to_run = "typo stop" > > > > stdin, stdout, stderr = Open3.popen3(cmd_to_run) > > > > s_stderr = stderr.read.to_s > > s_stdout = stdout.read.to_s > > > > # if you'd like to see it > > logger.info "stdout: #{s_stdout}" > > logger.info "stderr: #{s_stderr}" > > > > if s_stderr.empty? && s_stdout.starts_with?("error stuff here") > > return true > > else > > errors.add_to_base("error msg here") > > return false > > end > > > > hope that helps! > > > > On 8/21/06, hemant wrote: > >> Hi, > >> > >> I am using backgroundrb to create a automated system of typo > >> blogs.So, users > >> can create new typo applications using web interface and all that > >> jazz. > >> > >> Now..its working perfectly except one minor glitch. Since latest > >> version of > >> typo has a command called "typo", i simply use that command to > >> install new > >> typo app.And it works. > >> > >> def do_work(args) > >> DIR = "/home/foobar/public_html" > >> username = args > >> blog_name = username + "_blog" > >> # creation part works flawlessly so lets skip that. > >> # there is a problem while stopping it. > >> if (Dir.chdir(DIR)) == 0 > >> current_dir = `pwd` > >> @logger.debug("Current directory is #{current_dir}") > >> stop_command = "typo stop "+ blog_name > >> debug_stuff = `#{stop_command}` > >> #line no 33 > >> @logger.debug("### #{debug_stuff}") > >> else > >> @logger.debug("### Error changing directory") > >> end > >> end > >> Now let me assure that, args has correct username, because > >> starting of > >> blog,migrating from sqlite to mysql works perfectly using this worker > >> thread.The only glitch is, in line # 33, "typo stop " doesn't > >> work.I have > >> tried system also.system returns true, so typo should be > >> stopped.But when i > >> do "ps aux|grep foobar" , it isn't ,process is still there. > >> > >> When i tried `command`. All i got in debug_stuff variable is a empty > >> string.However, when i try same stuff from irb, it works as > >> expected and > >> typo stopped.Now..again let me assure that, the worker thread IS > >> getting > >> CALLED. > >> > >> I have tried various stuff.But since, I am simply not able to > >> debug, where > >> the heck problem is, I am lost. I was told that, system or > >> `command` both > >> wait for the command to finish execution to resume normal > >> execution.But for > >> me it SEEMS that the call to "typo stop #{blog_name}" returns > >> immediately. > >> > >> > >> > >> > >> _______________________________________________ > >> Backgroundrb-devel mailing list > >> Backgroundrb-devel at rubyforge.org > >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel > >> > >> > > > > > > -- > > Charles Brian Quinn > > self-promotion: www.seebq.com > > highgroove studios: www.highgroove.com > > slingshot hosting: www.slingshothosting.com > > _______________________________________________ > > Backgroundrb-devel mailing list > > Backgroundrb-devel at rubyforge.org > > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > -- nothing much to talk -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060822/bff59dbb/attachment.html From hammed at gmail.com Mon Aug 21 19:44:00 2006 From: hammed at gmail.com (Hammed Malik) Date: Tue, 22 Aug 2006 09:44:00 +1000 Subject: [Backgroundrb-devel] subscribe Message-ID: <5440c2a90608211644r5a218d00g32bf7e83b587722d@mail.gmail.com> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060822/ba4ce794/attachment.html From fernand.galiana at gmail.com Wed Aug 23 02:50:53 2006 From: fernand.galiana at gmail.com (Fernand Galiana) Date: Wed, 23 Aug 2006 00:50:53 -0600 Subject: [Backgroundrb-devel] Auto-Reload... Message-ID: <30ab479c0608222350n48957e9ak4a587b3573f3c0a5@mail.gmail.com> Hi, Not sure if this was posted before.... Is there a way to tell the backgroundrb server to reload workers a la rails in dev mode ? It seems that I have to cycle the server everytime I change my workers... Also has anyone cooked up a script for monitoring the background drb process on a nix OS ? ie rc init for server startup and making sure the process is happy thereafter... Thank you all, -Fernand -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060823/b8a1e8b0/attachment.html From info at siebert-wd.de Wed Aug 23 14:33:53 2006 From: info at siebert-wd.de (Michael Siebert) Date: Wed, 23 Aug 2006 20:33:53 +0200 Subject: [Backgroundrb-devel] Singleton-like Worker Message-ID: <32b567370608231133x74c0076el5a85826e5c3ff5ab@mail.gmail.com> Hello there, I need a Worker Class to be Singleton-like, that means there must be at maximum one Instance at a time. When a new Request is made to create another worker of this type, it should return and wait for the other instance to be destroyed. what i need to do: i have a worker who needs to do sth with java and openoffice, and it seems when i have more tha one worker simultaneously, everything goes terribly wrong... is there already sth in bgdrb or how would i do that? -- Michael Siebert www.siebert-wd.de - Gedanken lesen www.stellar-legends.de - Weltraum-Browsergame im Alpha-Stadium -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060823/556e6625/attachment.html From ezmobius at gmail.com Wed Aug 23 14:39:30 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Wed, 23 Aug 2006 11:39:30 -0700 Subject: [Backgroundrb-devel] Auto-Reload... In-Reply-To: <30ab479c0608222350n48957e9ak4a587b3573f3c0a5@mail.gmail.com> References: <30ab479c0608222350n48957e9ak4a587b3573f3c0a5@mail.gmail.com> Message-ID: <30784BB9-6A92-44B8-A935-D95EF8B4ABC9@gmail.com> On Aug 22, 2006, at 11:50 PM, Fernand Galiana wrote: > Hi, > > Not sure if this was posted before.... > > Is there a way to tell the backgroundrb server to reload > workers a la rails in dev mode ? > It seems that I have to cycle the server everytime I change my > workers... > > Also has anyone cooked up a script for monitoring the > background drb process on a nix OS ? > ie rc init for server startup and making sure the process is > happy thereafter... > > Thank you all, > > -Fernand > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel Hey Fernand- Yeah that will be in the next release. I have most of the code for it done so it should be soon. It will also include breakpoint support. -Ezra From ezmobius at gmail.com Wed Aug 23 14:42:17 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Wed, 23 Aug 2006 11:42:17 -0700 Subject: [Backgroundrb-devel] Singleton-like Worker In-Reply-To: <32b567370608231133x74c0076el5a85826e5c3ff5ab@mail.gmail.com> References: <32b567370608231133x74c0076el5a85826e5c3ff5ab@mail.gmail.com> Message-ID: <05FA633B-C3C4-4CE2-B6BE-6CDCBA959BDA@gmail.com> On Aug 23, 2006, at 11:33 AM, Michael Siebert wrote: > Hello there, > I need a Worker Class to be Singleton-like, that means there must > be at maximum one Instance at a time. When a new Request is made to > create another worker of this type, it should return and wait for > the other instance to be destroyed. > > what i need to do: i have a worker who needs to do sth with java > and openoffice, and it seems when i have more tha one worker > simultaneously, everything goes terribly wrong... > > is there already sth in bgdrb or how would i do that? > > -- > Michael Siebert > Hey Michael- Yeah there is already singleton type workers in the plugin. It doesn't work exactly as you want it to though. When you create a singleton worker it will start doing its thing and any call to new_worker for the same singleton will just return the key for the already existing singleton worker. So you are always garaunteed to get the same singleton. There is not currently any support for returning and then starting up after the singleton dies. But check it out anyway as it might do what you want. There is also now an option to have a singleton worker so there is always only one of a certain worker. Even if you call new_worker twice with this option enabled you will get back the same instance of the class. Do not create a worker class and use it as a singleton and a non singleton worker at the same time. This is not supported. MiddleMan.new_worker(:class => :foo_worker, :args => "Bar", :job_key => :singleton_worker, :singleton => true) -Ezra -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060823/c18dc769/attachment-0001.html From info at siebert-wd.de Wed Aug 23 15:30:40 2006 From: info at siebert-wd.de (Michael Siebert) Date: Wed, 23 Aug 2006 21:30:40 +0200 Subject: [Backgroundrb-devel] Singleton-like Worker In-Reply-To: <05FA633B-C3C4-4CE2-B6BE-6CDCBA959BDA@gmail.com> References: <32b567370608231133x74c0076el5a85826e5c3ff5ab@mail.gmail.com> <05FA633B-C3C4-4CE2-B6BE-6CDCBA959BDA@gmail.com> Message-ID: <32b567370608231230u122c8d2dwc4892e6a57101cba@mail.gmail.com> built upon the singleton code, i made a little option :queue_run and modified Backgroundrb::rails to wait in that case. little hackery though, but it works. wanna have it? (as always not tested, no unit tests no nothing) 2006/8/23, Ezra Zygmuntowicz : > > > On Aug 23, 2006, at 11:33 AM, Michael Siebert wrote: > > Hello there, > I need a Worker Class to be Singleton-like, that means there must be at > maximum one Instance at a time. When a new Request is made to create another > worker of this type, it should return and wait for the other instance to be > destroyed. > > what i need to do: i have a worker who needs to do sth with java and > openoffice, and it seems when i have more tha one worker simultaneously, > everything goes terribly wrong... > > is there already sth in bgdrb or how would i do that? > > -- > Michael Siebert > > > Hey Michael- > > Yeah there is already singleton type workers in the plugin. It doesn't > work exactly as you want it to though. When you create a singleton worker it > will start doing its thing and any call to new_worker for the same singleton > will just return the key for the already existing singleton worker. So you > are always garaunteed to get the same singleton. There is not currently any > support for returning and then starting up after the singleton dies. But > check it out anyway as it might do what you want. > > There is also now an option to have a singleton worker so there is always > only one of a certain worker. Even if you call new_worker twice with this > option enabled you will get back the same instance of the class. Do not > create a worker class and use it as a singleton and a non singleton worker > at the same time. This is not supported. > > MiddleMan.new_worker(:class => :foo_worker, > :args => "Bar", > :job_key => :singleton_worker, > :singleton => true) > > > -Ezra > > -- Michael Siebert www.siebert-wd.de - Gedanken lesen www.stellar-legends.de - Weltraum-Browsergame im Alpha-Stadium -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060823/9d2fedab/attachment.html From ezmobius at gmail.com Wed Aug 23 15:44:23 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Wed, 23 Aug 2006 12:44:23 -0700 Subject: [Backgroundrb-devel] Singleton-like Worker In-Reply-To: <32b567370608231230u122c8d2dwc4892e6a57101cba@mail.gmail.com> References: <32b567370608231133x74c0076el5a85826e5c3ff5ab@mail.gmail.com> <05FA633B-C3C4-4CE2-B6BE-6CDCBA959BDA@gmail.com> <32b567370608231230u122c8d2dwc4892e6a57101cba@mail.gmail.com> Message-ID: <6A1E38F7-D179-48C4-8F22-6507FB5A82E9@brainspl.at> Sure I will take a look at it. But I might wait until after I implement a thread pool to add it in. I am planning to make a configurable thread pool for workers. So you could say that 30 workers can be running at a time max. Then when you already have 30 workers running, and you ask it to run another that one will go into the wait pool until one of the first 30 finishes and makes room. This will make it safer to run a lot of workers and you will be able to taylor the size of the pool to fit your situation. But yeah do send me the code so I can see what you did. Cheers- -Ezra On Aug 23, 2006, at 12:30 PM, Michael Siebert wrote: > built upon the singleton code, i made a little option :queue_run > and modified Backgroundrb::rails to wait in that case. little > hackery though, but it works. > wanna have it? (as always not tested, no unit tests no nothing) > > 2006/8/23, Ezra Zygmuntowicz : > > On Aug 23, 2006, at 11:33 AM, Michael Siebert wrote: > >> Hello there, >> I need a Worker Class to be Singleton-like, that means there must >> be at maximum one Instance at a time. When a new Request is made >> to create another worker of this type, it should return and wait >> for the other instance to be destroyed. >> >> what i need to do: i have a worker who needs to do sth with java >> and openoffice, and it seems when i have more tha one worker >> simultaneously, everything goes terribly wrong... >> >> is there already sth in bgdrb or how would i do that? >> >> -- >> Michael Siebert >> > > Hey Michael- > > Yeah there is already singleton type workers in the plugin. It > doesn't work exactly as you want it to though. When you create a > singleton worker it will start doing its thing and any call to > new_worker for the same singleton will just return the key for the > already existing singleton worker. So you are always garaunteed to > get the same singleton. There is not currently any support for > returning and then starting up after the singleton dies. But check > it out anyway as it might do what you want. > > There is also now an option to have a singleton worker so there is > always > only one of a certain worker. Even if you call new_worker twice > with this > option enabled you will get back the same instance of the class. Do > not > create a worker class and use it as a singleton and a non singleton > worker > at the same time. This is not supported. > > MiddleMan.new_worker(:class => :foo_worker, > :args => "Bar", > :job_key => :singleton_worker, > :singleton => true) > > > -Ezra > > > > > -- > Michael Siebert > > www.siebert-wd.de - Gedanken lesen > www.stellar-legends.de - Weltraum-Browsergame im Alpha-Stadium > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060823/9c0779e9/attachment.html From mark at martianrover.com Wed Aug 23 16:27:34 2006 From: mark at martianrover.com (P. Mark Anderson) Date: Wed, 23 Aug 2006 13:27:34 -0700 Subject: [Backgroundrb-devel] Singleton-like Worker In-Reply-To: <6A1E38F7-D179-48C4-8F22-6507FB5A82E9@brainspl.at> References: <32b567370608231133x74c0076el5a85826e5c3ff5ab@mail.gmail.com> <05FA633B-C3C4-4CE2-B6BE-6CDCBA959BDA@gmail.com> <32b567370608231230u122c8d2dwc4892e6a57101cba@mail.gmail.com> <6A1E38F7-D179-48C4-8F22-6507FB5A82E9@brainspl.at> Message-ID: +1 useful on the thread pool. I can't wait. Really. The queue I wrote has two priority levels so that quick or important tasks execute before those of normal priority. It'd be great if Backgroundrb managed it all. pmark On Aug 23, 2006, at 12:44 PM, Ezra Zygmuntowicz wrote: > > Sure I will take a look at it. But I might wait until after I > implement a thread pool to add it in. I am planning to make a > configurable thread pool for workers. So you could say that 30 > workers can be running at a time max. Then when you already have 30 > workers running, and you ask it to run another that one will go > into the wait pool until one of the first 30 finishes and makes > room. This will make it safer to run a lot of workers and you will > be able to taylor the size of the pool to fit your situation. But > yeah do send me the code so I can see what you did. > > Cheers- > -Ezra > > > On Aug 23, 2006, at 12:30 PM, Michael Siebert wrote: > >> built upon the singleton code, i made a little option :queue_run >> and modified Backgroundrb::rails to wait in that case. little >> hackery though, but it works. >> wanna have it? (as always not tested, no unit tests no nothing) >> >> 2006/8/23, Ezra Zygmuntowicz : >> >> On Aug 23, 2006, at 11:33 AM, Michael Siebert wrote: >> >>> Hello there, >>> I need a Worker Class to be Singleton-like, that means there must >>> be at maximum one Instance at a time. When a new Request is made >>> to create another worker of this type, it should return and wait >>> for the other instance to be destroyed. >>> >>> what i need to do: i have a worker who needs to do sth with java >>> and openoffice, and it seems when i have more tha one worker >>> simultaneously, everything goes terribly wrong... >>> >>> is there already sth in bgdrb or how would i do that? >>> >>> -- >>> Michael Siebert >>> >> >> Hey Michael- >> >> Yeah there is already singleton type workers in the plugin. It >> doesn't work exactly as you want it to though. When you create a >> singleton worker it will start doing its thing and any call to >> new_worker for the same singleton will just return the key for the >> already existing singleton worker. So you are always garaunteed to >> get the same singleton. There is not currently any support for >> returning and then starting up after the singleton dies. But check >> it out anyway as it might do what you want. >> >> There is also now an option to have a singleton worker so there is >> always >> only one of a certain worker. Even if you call new_worker twice >> with this >> option enabled you will get back the same instance of the class. >> Do not >> create a worker class and use it as a singleton and a non >> singleton worker >> at the same time. This is not supported. >> >> MiddleMan.new_worker(:class => :foo_worker, >> :args => "Bar", >> :job_key => :singleton_worker, >> :singleton => true) >> >> >> -Ezra >> >> >> >> >> -- >> Michael Siebert >> >> www.siebert-wd.de - Gedanken lesen >> www.stellar-legends.de - Weltraum-Browsergame im Alpha-Stadium >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060823/88b9cb5a/attachment.html From fernand.galiana at gmail.com Wed Aug 23 16:59:58 2006 From: fernand.galiana at gmail.com (Fernand Galiana) Date: Wed, 23 Aug 2006 14:59:58 -0600 Subject: [Backgroundrb-devel] Auto-Reload... In-Reply-To: <30784BB9-6A92-44B8-A935-D95EF8B4ABC9@gmail.com> References: <30ab479c0608222350n48957e9ak4a587b3573f3c0a5@mail.gmail.com> <30784BB9-6A92-44B8-A935-D95EF8B4ABC9@gmail.com> Message-ID: <30ab479c0608231359p55faf3beg3492762eea500cd@mail.gmail.com> Very cool !! Thanks for your on going effort... -Fernand On 8/23/06, Ezra Zygmuntowicz wrote: > > > On Aug 22, 2006, at 11:50 PM, Fernand Galiana wrote: > > > Hi, > > > > Not sure if this was posted before.... > > > > Is there a way to tell the backgroundrb server to reload > > workers a la rails in dev mode ? > > It seems that I have to cycle the server everytime I change my > > workers... > > > > Also has anyone cooked up a script for monitoring the > > background drb process on a nix OS ? > > ie rc init for server startup and making sure the process is > > happy thereafter... > > > > Thank you all, > > > > -Fernand > > _______________________________________________ > > Backgroundrb-devel mailing list > > Backgroundrb-devel at rubyforge.org > > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > > Hey Fernand- > > Yeah that will be in the next release. I have most of the code for > it done so it should be soon. It will also include breakpoint support. > > -Ezra > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060823/cac3262c/attachment-0001.html From david at lemstra.ca Wed Aug 23 18:21:39 2006 From: david at lemstra.ca (David Lemstra) Date: Wed, 23 Aug 2006 18:21:39 -0400 Subject: [Backgroundrb-devel] Singleton-like Worker In-Reply-To: References: <32b567370608231133x74c0076el5a85826e5c3ff5ab@mail.gmail.com> <05FA633B-C3C4-4CE2-B6BE-6CDCBA959BDA@gmail.com> <32b567370608231230u122c8d2dwc4892e6a57101cba@mail.gmail.com> <6A1E38F7-D179-48C4-8F22-6507FB5A82E9@brainspl.at> Message-ID: <44ECD4F3.9080500@lemstra.ca> Funny, I've been working on this for over a day now. It uses a autostart singleton worker QueueWorker that acts as an intermediary to your actual child workers. It sets up an array of watchdog threads that each grab {args} from a queue and spawns a new child worker when the previous child worker in the slot is finished. It takes several layers of mutexes to make it all work but thanks to singleton workers, its done w/ hardly any changes to the current code. The original question cold be resolved by using this w/ just one watchdog thread. Yeah, eventually plan to add a priority Q as well... Its's really messy right now though and am going on a week vacation tomorrow. will send the dirty version if I don't get it cleaned up by then. cheers, David Lemstra P. Mark Anderson wrote: > +1 useful on the thread pool. I can't wait. Really. The queue I wrote > has two priority levels so that quick or important tasks execute before > those of normal priority. It'd be great if Backgroundrb managed it all. > > pmark > > > > > On Aug 23, 2006, at 12:44 PM, Ezra Zygmuntowicz wrote: > >> >> Sure I will take a look at it. But I might wait until after I >> implement a thread pool to add it in. I am planning to make a >> configurable thread pool for workers. So you could say that 30 workers >> can be running at a time max. Then when you already have 30 workers >> running, and you ask it to run another that one will go into the wait >> pool until one of the first 30 finishes and makes room. This will make >> it safer to run a lot of workers and you will be able to taylor the >> size of the pool to fit your situation. But yeah do send me the code >> so I can see what you did. >> >> Cheers- >> -Ezra >> >> >> On Aug 23, 2006, at 12:30 PM, Michael Siebert wrote: >> >>> built upon the singleton code, i made a little option :queue_run and >>> modified Backgroundrb::rails to wait in that case. little hackery >>> though, but it works. >>> wanna have it? (as always not tested, no unit tests no nothing) >>> >>> 2006/8/23, Ezra Zygmuntowicz >> >: >>> >>> >>> On Aug 23, 2006, at 11:33 AM, Michael Siebert wrote: >>> >>>> Hello there, >>>> I need a Worker Class to be Singleton-like, that means there >>>> must be at maximum one Instance at a time. When a new Request is >>>> made to create another worker of this type, it should return and >>>> wait for the other instance to be destroyed. >>>> >>>> what i need to do: i have a worker who needs to do sth with java >>>> and openoffice, and it seems when i have more tha one worker >>>> simultaneously, everything goes terribly wrong... >>>> >>>> is there already sth in bgdrb or how would i do that? >>>> >>>> -- >>>> Michael Siebert > >>>> >>> >>> Hey Michael- >>> >>> Yeah there is already singleton type workers in the plugin. It >>> doesn't work exactly as you want it to though. When you create a >>> singleton worker it will start doing its thing and any call to >>> new_worker for the same singleton will just return the key for >>> the already existing singleton worker. So you are always >>> garaunteed to get the same singleton. There is not currently any >>> support for returning and then starting up after the singleton >>> dies. But check it out anyway as it might do what you want. >>> >>> There is also now an option to have a singleton worker so there >>> is always >>> only one of a certain worker. Even if you call new_worker twice >>> with this >>> option enabled you will get back the same instance of the class. >>> Do not >>> create a worker class and use it as a singleton and a non >>> singleton worker >>> at the same time. This is not supported. >>> >>> MiddleMan.new_worker(:class => :foo_worker, >>> :args => "Bar", >>> :job_key => :singleton_worker, >>> :singleton => true) >>> >>> >>> -Ezra >>> >>> From me at seebq.com Thu Aug 24 00:36:13 2006 From: me at seebq.com (Charles Brian Quinn) Date: Thu, 24 Aug 2006 00:36:13 -0400 Subject: [Backgroundrb-devel] Auto-Reload... In-Reply-To: <30ab479c0608231359p55faf3beg3492762eea500cd@mail.gmail.com> References: <30ab479c0608222350n48957e9ak4a587b3573f3c0a5@mail.gmail.com> <30784BB9-6A92-44B8-A935-D95EF8B4ABC9@gmail.com> <30ab479c0608231359p55faf3beg3492762eea500cd@mail.gmail.com> Message-ID: <3a2de0cd0608232136i4ffd882m247d30c17f0e00c0@mail.gmail.com> As for the process for keeping it running, that's interesting -- I would look into supervise -- a linux daemon for supervising processes to make sure they're up. qmail uses this pretty extensively.... On 8/23/06, Fernand Galiana wrote: > Very cool !! Thanks for your on going effort... > > -Fernand > > > On 8/23/06, Ezra Zygmuntowicz wrote: > > > > On Aug 22, 2006, at 11:50 PM, Fernand Galiana wrote: > > > > > Hi, > > > > > > Not sure if this was posted before.... > > > > > > Is there a way to tell the backgroundrb server to reload > > > workers a la rails in dev mode ? > > > It seems that I have to cycle the server everytime I change my > > > workers... > > > > > > Also has anyone cooked up a script for monitoring the > > > background drb process on a nix OS ? > > > ie rc init for server startup and making sure the process is > > > happy thereafter... > > > > > > Thank you all, > > > > > > -Fernand > > > _______________________________________________ > > > Backgroundrb-devel mailing list > > > Backgroundrb-devel at rubyforge.org > > > > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > > > > > Hey Fernand- > > > > Yeah that will be in the next release. I have most of the code for > > it done so it should be soon. It will also include breakpoint support. > > > > -Ezra > > > > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > -- Charles Brian Quinn self-promotion: www.seebq.com highgroove studios: www.highgroove.com slingshot hosting: www.slingshothosting.com From david at lemstra.ca Thu Aug 24 00:43:46 2006 From: david at lemstra.ca (David Lemstra) Date: Thu, 24 Aug 2006 00:43:46 -0400 Subject: [Backgroundrb-devel] A new QueueWorker class Message-ID: <44ED2E82.8070800@lemstra.ca> Hello all, I've come up w/ a worker class that manages queued jobs using a fixed number of child workers. Well, that's not quite true -- a new worker is spawned for each job, but you set the total number that may exist at once. There are three components: 1) queue_worker.rb: The singleton worker that manages the child workers. You probably want to auto start this. Make sure you give it :singleton=>true. Read this file for the methods to interact with your children. (ie. queue_job(), delete_job(), job_progress() ) 2) backgroundrb_rails_queue.rb: The super class for the "child workers" (and uses backgroundrb_rails.rb in turn). This file needs to be included in background.rb 3) Your child worker, which should be a subclass of BackgrounDRb::RailsQueue, is otherwise the about same as normal If it's a big loop, you probably want to use terminate?() on each iteration and update @progress. Use suicide() at the end to make room for the next child. Options: (probably in your backgroundrb.yml) autostart: :queue_key: class: queue_worker args: :num_child_workers: 2 :child_class: :cost_calculator_worker :reQ_on_finish: true :singleton: true :queue_key can be changed to what you want, but it is the permanent key of the QueueWorker :num_child_workers: is up to you! :child_class: your worker class you want as child workers. :reQ_on_finish: do you want results to be stored in the queue until you call job_progress!() ? Note: to be able to access your child jobs w/ the QueueWorker methods, include a unique :id in your {args} when you queue_job({args}) I'll attach the files. If they don't go through, I'll resend as text. BTW, This works well enough for me, but I'm learning as I go too, so no guarantees :) I don't use the fancy timing options, so ymmmv for :next_start and :interval. Let me know if you find any issues (though I'm off-line for a week after this post). I'm wondering myself if it might be better to reuse child workers instead of re-spawning new ones. Another day maybe. cheers, David Lemstra -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: queue_worker.rb Url: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060824/2349e2f4/attachment.ksh -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: backgroundrb_rails_queue.rb Url: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060824/2349e2f4/attachment-0001.ksh From ezmobius at gmail.com Thu Aug 24 02:41:13 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Wed, 23 Aug 2006 23:41:13 -0700 Subject: [Backgroundrb-devel] A new QueueWorker class In-Reply-To: <44ED2E82.8070800@lemstra.ca> References: <44ED2E82.8070800@lemstra.ca> Message-ID: Wow! Thats pretty thick ;) I'll work my way through it this weekend and see how it works. Thanks -Ezra On Aug 23, 2006, at 9:43 PM, David Lemstra wrote: > Hello all, > I've come up w/ a worker class that manages queued jobs using a fixed > number of child workers. Well, that's not quite true -- a new > worker is > spawned for each job, but you set the total number that may exist > at once. > > There are three components: > 1) queue_worker.rb: The singleton worker that manages the child > workers. > You probably want to auto start this. Make sure you give it > :singleton=>true. Read this file for the methods to interact with your > children. (ie. queue_job(), delete_job(), job_progress() ) > > 2) backgroundrb_rails_queue.rb: The super class for the "child > workers" > (and uses backgroundrb_rails.rb in turn). This file needs to be > included > in background.rb > > 3) Your child worker, which should be a subclass of > BackgrounDRb::RailsQueue, is otherwise the about same as normal > If it's a big loop, you probably want to use terminate?() on each > iteration and update @progress. Use suicide() at the end to make room > for the next child. > > Options: (probably in your backgroundrb.yml) > autostart: > :queue_key: > class: queue_worker > args: > :num_child_workers: 2 > :child_class: :cost_calculator_worker > :reQ_on_finish: true > :singleton: true > > :queue_key can be changed to what you want, but it is the permanent > key > of the QueueWorker > :num_child_workers: is up to you! > :child_class: your worker class you want as child workers. > :reQ_on_finish: do you want results to be stored in the queue until > you > call job_progress!() ? > Note: to be able to access your child jobs w/ the QueueWorker methods, > include a unique :id in your {args} when you queue_job({args}) > > I'll attach the files. If they don't go through, I'll resend as text. > > BTW, This works well enough for me, but I'm learning as I go too, > so no > guarantees :) I don't use the fancy timing options, so ymmmv for > :next_start and :interval. > > Let me know if you find any issues (though I'm off-line for a week > after > this post). I'm wondering myself if it might be better to reuse child > workers instead of re-spawning new ones. Another day maybe. > > cheers, > David Lemstra > # Put your code that runs your task inside the do_work method > # it will be run automatically in a thread. You have access to > # all of your rails models if you set load_rails to true in the > # config file. You also get @logger inside of this class by default. > require 'monitor.rb' > > class QueueWorker < BackgrounDRb::Rails > > attr_reader :q, :id_hash, :completed > def initialize(key, args={}) > super(key,args) > @num_child_workers = args[:num_child_workers] ? args > [:num_child_workers] : 1 > @child_workers = Array.new(@num_child_workers) {|i| Hash > [:job_key,nil,:args,nil,:s_thread,nil, :s_mutex,Mutex.new, :child, > i] } > > @q = [] > @q.extend(MonitorMixin) > @q_loaded_cv = @q.new_cond > @id_hash = {} > @id_hash_mutex = Mutex.new > > raise ArgumentError unless args.has_key?(:child_class) > @child_class = args[:child_class] > @reQ_on_finish = args[:reQ_on_finish] || false > @completed = 0 > end > > def queue_job(args) > return nil if @id_hash && args[:id] && @id_hash.has_key?(args > [:id]) > @q.synchronize do > @q.push args > @id_hash_mutex.synchronize { @id_hash[args[:id]] = > {:status=>:queued, :job_key=>nil, :results=>nil } } if args[:id] > @q_loaded_cv.signal > end > return true > end > > def job_in_progress?(job_id) > @id_hash.has_key?(job_id) > end > > def job_status?(job_id) > @id_hash_mutex.synchronize do > return nil unless @id_hash.has_key?(job_id) > return @id_hash[job_key][:status] > end > end > > def job_progress(job_id) > report_hsh = {} > @id_hash_mutex.synchronize do > return nil unless @id_hash.has_key?(job_id) > report_hsh[:status] = @id_hash[job_id][:status] > report_hsh[:progress] = case @id_hash[job_id][:status] > when :queued then > ahead = 0 > @q.each_index {|i| if @q[i][:id] == job_id then ahead = > i; break; end } > ahead > when :running > w = self[@id_hash[job_id][:job_key]] > w.nil? ? nil : w.progress > when :done > @id_hash[job_id][:results] > else nil > end > end > return report_hsh > end > > def job_progress!(job_id) > report_hsh = {} > @id_hash_mutex.synchronize do > return nil unless @id_hash.has_key?(job_id) > report_hsh[:status] = @id_hash[job_id][:status] > report_hsh[:progress] = case @id_hash[job_id][:status] > when :queued then > ahead = 0 > @q.each_index {|i| if @q[i][:id] == job_id then ahead = > i; break; end } > ahead > when :running > w = self[@id_hash[job_id][:job_key]] > if w.nil? > @id_hash.delete(job_id)[:results] > else > w.progress > end > when :done > @id_hash.delete(job_id)[:results] > else nil > end > end > return report_hsh > end > > def delete_job(job_id) > args = nil > @q.synchronize do @id_hash_mutex.synchronize do > args = @id_hash[job_id] > return true if args.nil? > if args[:status] == :queued > @q.delete_if {|h| h[:id] == job_id } > @id_hash.delete(job_id) > return true > elsif args[:status] == :done > @id_hash.delete(job_id) > return true > end > end > end > ::BackgrounDRb::MiddleMan.instance.delete_worker(args > [:job_key]) if args[:status] == :running > return true > end > > def do_work(args) > # You probably don't want to mess with this method unless you > know what's what. > @child_workers.each do |child_hash| > child_hash[:s_thread] = Thread.start do > loop do > # Wait for a new job in the @q > child_hash[:s_mutex].synchronize do > # get the Q mutex and wait for a job > @q.synchronize do > tl = Thread.list > @q_loaded_cv.wait_while { @q.empty? } > child_hash[:args] = @q.shift > if child_hash[:args][:id] > @id_hash_mutex.synchronize do > @id_hash[child_hash[:args][:id]][:status] = :running > child_hash[:job_key] = spawn_worker > ({:args=>child_hash[:args],:class=>@child_class}) > @id_hash[child_hash[:args][:id]][:job_key] = > child_hash[:job_key] > end > else > child_hash[:job_key] = spawn_worker(job_args.merge > (:class=>@child_class)) > end > end > self[child_hash[:job_key]].thread[:DQ_request].wait > (child_hash[:s_mutex]) > # grab and store the results > if child_hash[:args][:id] > @id_hash_mutex.synchronize do > if @reQ_on_finish > r = self[child_hash[:job_key]].results > @id_hash[child_hash[:args][:id]][:results] = r if r > @id_hash[child_hash[:args][:id]][:status] = :done > @id_hash[child_hash[:args][:id]][:done_at] = > Time.now > else > @id_hash.delete(child_hash[:args][:id]) > end > end > end > self[child_hash[:job_key]].thread[:DQed].signal > @completed += 1 > [:args,:job_key].each {|k| child_hash[k] = nil } > end > # Loop back and wait for the job_key to get killed again.... > end > end > end > end > > private > > def [](key) > # Use jobs to avoid the access time update w/ [] > ::BackgrounDRb::MiddleMan.instance.jobs[key] > end > > end > module BackgrounDRb > > class RailsQueue < BackgrounDRb::Rails > attr_reader :progress > def initialize(key, args={}) > super(key,args) > @job_ctrl = true > end > > def start_process > return if schedule_first_run && schedule_first_run.to_i > > Time.now.to_i > @thread = Thread.new do > Thread.current[:safe_to_kill] = ConditionVariable.new > Thread.current[:kill] = false > Thread.current[:DQ_request] = ConditionVariable.new > Thread.current[:DQed] = ConditionVariable.new > Thread.current[:mutex] = Mutex.new > begin > Thread.current[:mutex].synchronize do > do_work(@args) > end > rescue Exception => e > @logger.error "#{ e.message } - (#{ e.class })" << "\n" > << (e.backtrace or []).join("\n") > end > end > @next_start = @interval.from_now if schedule_repeat > end > > def results > # Overwrite this method and set reQ_on_finish = true (in the > queue worker args) > # to have a process put it's results in back in the queue > # for pickup before being killed > nil > end > > def before_DQ(args=nil) > # stub method that gets called before dequeue is run. > # Overwrite in your class instance > true > end > > def terminate(args=nil) > do_DQ(args) > super(args) > end > > def suicide(args=nil) > do_DQ(args) > kill > Thread.exit > end > > private > def do_DQ(args=nil) > before_DQ(args) > Thread.current[:DQ_request].signal > Thread.current[:DQed].wait(Thread.current[:mutex]) > end > end > end_______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel From lapomme00 at gmail.com Thu Aug 24 05:13:17 2006 From: lapomme00 at gmail.com (Brett Walker) Date: Thu, 24 Aug 2006 17:13:17 +0800 Subject: [Backgroundrb-devel] Unable to start with Capistrano... Message-ID: I've created a couple of tasks in my deploy.rb file for starting and stopping backgroundrb on the remote server. The stop task runs fine and kills the process. And the start task *seems* to run successfully, but backgroundrb is never started on the remote host. Has anyone had success with this? Here are my deploy tasks: desc 'Start backgroundrb server (default values)' task :start_backgroundrb do puts "Starting BackgrounDRb" #--- assumes the backgroundrb.yml is set up for production run "export RAILS_ENV=production && #{current_path}/script/backgroundrb/start -d" end desc 'Stop backgroundrb server (default values)' task :stop_backgroundrb do puts "Stopping BackgrounDRb" begin run "#{current_path}/script/backgroundrb/stop" rescue end end I'm no doubt missing something obvious. Cheers, Brett From codeslush at yahoo.com Thu Aug 24 14:44:11 2006 From: codeslush at yahoo.com (Michael) Date: Thu, 24 Aug 2006 11:44:11 -0700 (PDT) Subject: [Backgroundrb-devel] Regarding New QueueWorker class Message-ID: <20060824184411.61817.qmail@web30709.mail.mud.yahoo.com> Hi Ezra/Community, I posted a few comments last night on the rails forum and followed Ezra's advice to come here! Lot's of good stuff in the archives that probably saved me much headache (specifically, ActionMailer related items). I saw this post - and am wondering if anyone else has had a chance yet to download the two source files and try it out. Queuing of work items is definitely a big asset and something that I will be trying to do. I'll try the code as well, but if anyone has had any initial success/problems with it then please advise and save me some time! I look forward to working with this group! Regards, Michael --------------------------------- Get your own web address for just $1.99/1st yr. We'll help. Yahoo! Small Business. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060824/ca8a1ffd/attachment.html From ezmobius at gmail.com Thu Aug 24 14:50:27 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Thu, 24 Aug 2006 11:50:27 -0700 Subject: [Backgroundrb-devel] Regarding New QueueWorker class In-Reply-To: <20060824184411.61817.qmail@web30709.mail.mud.yahoo.com> References: <20060824184411.61817.qmail@web30709.mail.mud.yahoo.com> Message-ID: <118573E5-2C2C-484B-8488-81F208DDE6F0@gmail.com> On Aug 24, 2006, at 11:44 AM, Michael wrote: > Hi Ezra/Community, > > I posted a few comments last night on the rails forum and followed > Ezra's advice to come here! Lot's of good stuff in the archives > that probably saved me much headache (specifically, ActionMailer > related items). > > I saw this post - and am wondering if anyone else has had a chance > yet to download the two source files and try it out. Queuing of > work items is definitely a big asset and something that I will be > trying to do. I'll try the code as well, but if anyone has had any > initial success/problems with it then please advise and save me > some time! > > I look forward to working with this group! > > Regards, > > Michael The QueueWorker looks like it will work very well but I haven't had a chance to try it out yet though. I think that it might be better if the core MiddleMan held the Thread Pool. I hope to have time to work more on my thread pool impl now that I am done with my day job and on to greener pastures ;) We will see what I come up with. But for now by all means try out the new queue worker stuff and report back how it works for you. -Ezra P.S. Welcome to the list ;) From bjohnson at contuitive.com Thu Aug 24 17:08:36 2006 From: bjohnson at contuitive.com (Ben Johnson) Date: Thu, 24 Aug 2006 16:08:36 -0500 Subject: [Backgroundrb-devel] ActionMailer doesn't work in backgroundrb Message-ID: I am pretty confident that mailers do not work in backgroundrb. Can anyone confirm this? Because when I try to do Notifier.deliver_some_method I get an error saying it cant find the templates for my email. The funny thing is that when I run script/ runner Notifier.deliver_some_method it works just fine. Any ideas? Thank You, Ben Johnson E: bjohnson at contuitive.com O: 800-341-6826 M: 817-229-4258 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060824/9210b250/attachment.html From lapomme00 at gmail.com Fri Aug 25 01:18:23 2006 From: lapomme00 at gmail.com (Brett Walker) Date: Fri, 25 Aug 2006 13:18:23 +0800 Subject: [Backgroundrb-devel] repeat_every when a worker is not done yet... Message-ID: If I use repeat_every for a worker, if the do_work function has not completed by the time the next interval rolls around, will a new thread be started, resulting in 2 threads for the same worker? Looking at start_process, it seems like the thread gets created (which spins off the thread and then continues to execute the start_process method, right?), then sets the @next_start. So if the @next_start is in 10 minutes, but it takes 20 minutes to finish the current work, start_timer will attempt to call start_process on the job, and kick off another thread, resulting in 2 threads. Is this correct? What I want to happen is that the thread can't get kicked off again until the current one has completed/returned. Maybe the setting of @next_start needs to go in the thread block. Or did I totally misunderstand the code? Cheers, Brett From dgreig at magian.com Fri Aug 25 01:21:37 2006 From: dgreig at magian.com (Daniel Greig) Date: Fri, 25 Aug 2006 15:21:37 +1000 Subject: [Backgroundrb-devel] backgroundrb fails when running as win32 service Message-ID: <44EE88E1.1060009@magian.com> Hi, I've been playing with backgroundrb to batch process a bunch of images on a rails server. The process is working fine when backgroundrb is started from the command line, but seems to choke occasionally (well, quite often) when running as a service. It seems to get stuck on the following line when moving files: FileUtils.move(src, dest, :verbose => true) with the error: Bad file descriptor - (Errno::EBADF) For a while I thought it may have been a file permissions/file access issue, but I have seen another error where the process fails at a "puts" statement. I have a suspicion that the error may be occurring when ruby tries to write to the standard output using "puts". Since the "verbose" argument sends a message to the standard output, it could be failing for the same reason and just looking like a file move error. I have removed all puts statements and all verbose method calls and the service *seems* to be more reliable (although it's to early to say as I have to do a lot more testing). Has anyone seen any similar problems with the win32 service? My fingers are crossed that getting rid of the puts statements fixes the problem, but I'm nervous that it may be something else entirely - anyone have any ideas? thanks! Dan From gethemant at gmail.com Fri Aug 25 07:28:34 2006 From: gethemant at gmail.com (hemant) Date: Fri, 25 Aug 2006 16:58:34 +0530 Subject: [Backgroundrb-devel] uninitialised constant LoginEngine Message-ID: In my worker class, when i try to start the worker thread, it throws the error: uninitialized constant LoginEngine - (NameError) /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:123:in `const_missing' /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:131:in `const_missing' /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:133:in `const_missing' /home/foo_fighter/public_html/fooapp/config/../script/backgroundrb/../../config/../app/models/user.rb:3 /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:140:in `load' /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:56:in `require_or_load' /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:30:in `depend_on' /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:85:in `require_dependency' /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:98:in `const_missing' /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:131:in `const_missing' /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:133:in `const_missing' /home/foo_fighter/public_html/fooapp/config/../script/backgroundrb/../../config/../lib/workers/monitor_messageboards_worker.rb:18:in `do_work' /home/foo_fighter/public_html/fooapp/config/../script/backgroundrb/../../config/../vendor/plugins/backgroundrb/backgroundrb_rails.rb:35:in `start_process' /home/foo_fighter/public_html/fooapp/config/../script/backgroundrb/../../config/../vendor/plugins/backgroundrb/backgroundrb_rails.rb:31:in `start_process' /home/foo_fighter/public_html/fooapp/config/../script/backgroundrb/../../config/../vendor/plugins/backgroundrb/backgroundrb.rb:57:in `new_worker' /usr/lib/ruby/1.8/thread.rb:135:in `synchronize' /home/foo_fighter/public_html/fooapp/config/../script/backgroundrb/../../config/../vendor/plugins/backgroundrb/backgroundrb.rb:49:in `new_worker' /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' /home/foo_fighter/public_html/fooapp/config/../script/backgroundrb/start:96 /home/foo_fighter/public_html/fooapp/config/../script/backgroundrb/start:92 /home/foo_fighter/public_html/fooapp/config/../script/backgroundrb/start:84 any ideas why? and how to prevent this? -- nothing much to talk -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060825/6feaee72/attachment.html From gethemant at gmail.com Fri Aug 25 07:39:43 2006 From: gethemant at gmail.com (hemant) Date: Fri, 25 Aug 2006 17:09:43 +0530 Subject: [Backgroundrb-devel] uninitialised constant LoginEngine In-Reply-To: References: Message-ID: I geuss, I am the one, who is asking this question again.I tried to search through, the mails, but clearly...following solutions didn't work for me: $: << "#{RAILS_ROOT}/vendor/plugins/engines/lib/engines" $: << "#{RAILS_ROOT}/vendor/plugins/login_engine/lib" Also, i last time, I worked around the problem by simply not using loginengine model class.But this time, as it seems i need this. There is a price to be paid, for using some readymade component. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060825/04dc27c1/attachment-0001.html From mklein at bpl.org Fri Aug 25 09:19:27 2006 From: mklein at bpl.org (Klein, Michael) Date: Fri, 25 Aug 2006 09:19:27 -0400 Subject: [Backgroundrb-devel] ActionMailer doesn't work in backgroundrb In-Reply-To: Message-ID: I haven?t used ActionMailer much myself, but based on past experience with missing files and background processes, I?m guessing this is either a missing or mis-set RAILS_ROOT environment variable, or a permissions problem. On 8/24/06 5:08 PM, "Ben Johnson" wrote: > I am pretty confident that mailers do not work in backgroundrb. Can anyone > confirm this? Because when I try to do Notifier.deliver_some_method I get an > error saying it cant find the templates for my email. The funny thing is that > when I run script/runner?Notifier.deliver_some_method it works just fine. Any > ideas? > > > > Thank You, > Ben Johnson > E: bjohnson at contuitive.com > O: 800-341-6826 > M: 817-229-4258 > > > > > > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060825/c4163a51/attachment.html From ezmobius at gmail.com Fri Aug 25 11:13:24 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Fri, 25 Aug 2006 08:13:24 -0700 Subject: [Backgroundrb-devel] uninitialised constant LoginEngine In-Reply-To: References: Message-ID: <1C459FFB-2FF5-4F88-9DA3-EDFF93EEC14C@brainspl.at> On Aug 25, 2006, at 4:39 AM, hemant wrote: > I geuss, I am the one, who is asking this question again.I tried to > search through, the mails, but clearly...following solutions didn't > work for me: > > $: << "#{RAILS_ROOT}/vendor/plugins/engines/lib/engines" > $: << "#{RAILS_ROOT}/vendor/plugins/login_engine/lib" > > Also, i last time, I worked around the problem by simply not using > loginengine model class.But this time, as it seems i need this. > There is a price to be paid, for using some readymade component. > > > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel Hey Hemant- What I think you need to do is open up the start script and replace the line that require's boot.rb and make it require environment.rb., THat way it will go through rails normal load process and your engines will get loaded as well. This has bitten enough people that I am going to change it so that by default it will require all of rails if you set load_rails to true in your config file. Its an inconosistency on my part the way it works now. Cheers- -Ezra From ezmobius at gmail.com Fri Aug 25 11:14:56 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Fri, 25 Aug 2006 08:14:56 -0700 Subject: [Backgroundrb-devel] repeat_every when a worker is not done yet... In-Reply-To: References: Message-ID: <4667952B-0A86-40DD-876A-DA4E2B6C54EE@gmail.com> On Aug 24, 2006, at 10:18 PM, Brett Walker wrote: > If I use repeat_every for a worker, if the do_work function has not > completed by the time the next interval rolls around, will a new > thread be started, resulting in 2 threads for the same worker? > > Looking at start_process, it seems like the thread gets created (which > spins off the thread and then continues to execute the start_process > method, right?), then sets the @next_start. So if the @next_start is > in 10 minutes, but it takes 20 minutes to finish the current work, > start_timer will attempt to call start_process on the job, and kick > off another thread, resulting in 2 threads. > > Is this correct? What I want to happen is that the thread can't get > kicked off again until the current one has completed/returned. Maybe > the setting of @next_start needs to go in the thread block. Or did I > totally misunderstand the code? > > Cheers, > Brett Hey Brett- You know I think you are correct. It looks like if your worker lasts longer then the interval that you will end up with multiple worker threads. Hmm.. I personally haven't used the repeat_every stuff. I added unit tests for it when the patch was submitted but I don't think I considered the case you are talking about. I am in the middle of a wholesale refactoring of the codebase in order to add a thread pool and more precise timed workers. Hopefully sometime next week I will have a release. If you need this right away then I would happily accept a patch for more detailed cron style workers. Cheers- -Ezra From ezmobius at gmail.com Fri Aug 25 11:42:48 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Fri, 25 Aug 2006 08:42:48 -0700 Subject: [Backgroundrb-devel] Small Update: load_rails now actually loads rails Message-ID: <7CFA98A6-5069-451E-9F34-FAAEC187535E@gmail.com> Folks- After enough people running into problems because load_rails was misleading and actually only loaded your models and ActiveRecord, I have just pushed a small update> All that has changed is that in the config file load_rails is set to true by default and it actually loads all of rails by require'ing environment.rb. SO anything that works in your rails app should be available in your workers now including plugins and engines. Sorry for the confustion that was a seriously bad choice of a config option for what it used to do. Nothing else has changed sine the last release except for this. Thanks- -Ezra From ezmobius at gmail.com Fri Aug 25 11:46:21 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Fri, 25 Aug 2006 08:46:21 -0700 Subject: [Backgroundrb-devel] backgroundrb fails when running as win32 service In-Reply-To: <44EE88E1.1060009@magian.com> References: <44EE88E1.1060009@magian.com> Message-ID: On Aug 24, 2006, at 10:21 PM, Daniel Greig wrote: > Hi, > I've been playing with backgroundrb to batch process a bunch of images > on a rails server. The process is working fine when backgroundrb is > started from the command line, but seems to choke occasionally (well, > quite often) when running as a service. It seems to get stuck on the > following line when moving files: > FileUtils.move(src, dest, :verbose => true) > > with the error: > > Bad file descriptor - (Errno::EBADF) > > For a while I thought it may have been a file permissions/file access > issue, but I have seen another error where the process fails at a > "puts" > statement. I have a suspicion that the error may be occurring when > ruby > tries to write to the standard output using "puts". Since the > "verbose" > argument sends a message to the standard output, it could be failing > for the same reason and just looking like a file move error. > I have removed all puts statements and all verbose method calls and > the > service *seems* to be more reliable (although it's to early to say > as I > have to do a lot more testing). Has anyone seen any similar problems > with the win32 service? > > My fingers are crossed that getting rid of the puts statements > fixes the > problem, but I'm nervous that it may be something else entirely - > anyone > have any ideas? > thanks! > Dan Hey Dan- Unfortunately I don't have any windows boxes for testing the service part with. But from what I know of windows services you are correct, I think the errors you are getting are from using puts or print to stdout which seems to be closed or unavailable when running as a service. So as long as you don't use puts or print and instead use the @logger you should be fine. I also wonder if there is a way to redirect stdout to the log file on windows? If you get a chance to investigate then please let me know what you find out. For now you will probably have to not use the verbose option and make sure you use the log instead of puts. Cheers- -Ezra From gethemant at gmail.com Fri Aug 25 16:14:09 2006 From: gethemant at gmail.com (hemant) Date: Sat, 26 Aug 2006 01:44:09 +0530 Subject: [Backgroundrb-devel] uninitialised constant LoginEngine In-Reply-To: <1C459FFB-2FF5-4F88-9DA3-EDFF93EEC14C@brainspl.at> References: <1C459FFB-2FF5-4F88-9DA3-EDFF93EEC14C@brainspl.at> Message-ID: Again i managed to workaround the problem, by simply not using user.rbfiles(model class defined by LoginEngine). :) but...i had another issue, some of my model classes, were using some global connection objects( not to databases but Unix socket connections to some data vending servers).Now, this connection object was created in environment.rb file, so i required the file in that worker thread and it started to work. but, isn't it that loading the entire rails environment is a little heavy?? On 8/25/06, Ezra Zygmuntowicz wrote: > > > On Aug 25, 2006, at 4:39 AM, hemant wrote: > > > I geuss, I am the one, who is asking this question again.I tried to > > search through, the mails, but clearly...following solutions didn't > > work for me: > > > > $: << "#{RAILS_ROOT}/vendor/plugins/engines/lib/engines" > > $: << "#{RAILS_ROOT}/vendor/plugins/login_engine/lib" > > > > Also, i last time, I worked around the problem by simply not using > > loginengine model class.But this time, as it seems i need this. > > There is a price to be paid, for using some readymade component. > > > > > > > > _______________________________________________ > > Backgroundrb-devel mailing list > > Backgroundrb-devel at rubyforge.org > > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > Hey Hemant- > > What I think you need to do is open up the start script and > replace > the line that require's boot.rb and make it require environment.rb., > THat way it will go through rails normal load process and your > engines will get loaded as well. > > This has bitten enough people that I am going to change it so that > by default it will require all of rails if you set load_rails to true > in your config file. Its an inconosistency on my part the way it > works now. > > Cheers- > -Ezra > -- nothing much to talk -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060826/5323936b/attachment.html From hughes.james at gmail.com Fri Aug 25 17:22:27 2006 From: hughes.james at gmail.com (James Hughes) Date: Fri, 25 Aug 2006 14:22:27 -0700 Subject: [Backgroundrb-devel] Rake tasks fail silently, how to debug? Message-ID: <765a2c230608251422j4d3bba9bx91aa0650a7abdd88@mail.gmail.com> Hi, I've been using backgroundrb with some success up to now, but now when I do the start or setup tasks they just fail silently. Nothing in the log, no .pid file, and nothing in the process list. Rake with -trace just says, ** Invoke backgroundrb:setup (first_time) ** Execute backgroundrb:setup Any hints on how I can get more verbosity, or thoughts on what might be happening? (This happens with both the rails gem and edge; also, I'm tracking backgroundrb via svn:externals) thanks, jh -- James Hughes Web application developer Vancouver, BC "Developing a coherent political analysis is in many respects contingent upon an ability to connect one context to another, a process not dissimilar to playing the kid's game of dot-to-dot." - Ward Churchill, from '"A Government of Laws"?' From ezmobius at gmail.com Fri Aug 25 17:33:51 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Fri, 25 Aug 2006 14:33:51 -0700 Subject: [Backgroundrb-devel] Rake tasks fail silently, how to debug? In-Reply-To: <765a2c230608251422j4d3bba9bx91aa0650a7abdd88@mail.gmail.com> References: <765a2c230608251422j4d3bba9bx91aa0650a7abdd88@mail.gmail.com> Message-ID: <1F6E85D2-E1C7-4EBD-9CB4-84AE7AE4A7D6@gmail.com> On Aug 25, 2006, at 2:22 PM, James Hughes wrote: > Hi, > > I've been using backgroundrb with some success up to now, but now when > I do the start or setup tasks they just fail silently. Nothing in the > log, no .pid file, and nothing in the process list. Rake with -trace > just says, > > ** Invoke backgroundrb:setup (first_time) > ** Execute backgroundrb:setup > > Any hints on how I can get more verbosity, or thoughts on what might > be happening? > > (This happens with both the rails gem and edge; also, I'm tracking > backgroundrb via svn:externals) > > thanks, > jh > > -- > James Hughes > Web application developer > Vancouver, BC Hey James- When did this start happening? If you roll back one revision from trunk does that fix it? I just did a fresh svn co of trunk and was able to run the setup and start tasks without any issues. So please try again. Aslo can you use the start script directly instead of the rake task? Take a look in the rake task you can see what it does, it just copies the script dir up into your rails app and makes a config file. I don't know how to make rake more verbose in this case. Maybe open up the rakefile and add some puts statements and tell me what you find out would you? -Ezra From hughes.james at gmail.com Fri Aug 25 18:09:57 2006 From: hughes.james at gmail.com (James Hughes) Date: Fri, 25 Aug 2006 15:09:57 -0700 Subject: [Backgroundrb-devel] Rake tasks fail silently, how to debug? In-Reply-To: <1F6E85D2-E1C7-4EBD-9CB4-84AE7AE4A7D6@gmail.com> References: <765a2c230608251422j4d3bba9bx91aa0650a7abdd88@mail.gmail.com> <1F6E85D2-E1C7-4EBD-9CB4-84AE7AE4A7D6@gmail.com> Message-ID: <765a2c230608251509j4df07920v11d791f18e269708@mail.gmail.com> (Sorry, I keep forgetting reply-all on this list) On 8/25/06, Ezra Zygmuntowicz wrote: > > On Aug 25, 2006, at 2:22 PM, James Hughes wrote: > > > Any hints on how I can get more verbosity, or thoughts on what might > > be happening? > > > > (This happens with both the rails gem and edge; also, I'm tracking > > backgroundrb via svn:externals) > > Hey James- > > When did this start happening? If you roll back one revision from > trunk does that fix it? I just did a fresh svn co of trunk and was > able to run the setup and start tasks without any issues. So please > try again. Aslo can you use the start script directly instead of the > rake task? Take a look in the rake task you can see what it does, it > just copies the script dir up into your rails app and makes a config > file. I don't know how to make rake more verbose in this case. Maybe > open up the rakefile and add some puts statements and tell me what > you find out would you? > > -Ezra > Hi Ezra, I've done what you suggest. The setup task actually appears to be behaving correctly: all the scripts and config are already in place so it's not actually failing, it's just not doing anything. However, when I run the start script directly (i.e. not with rake), without the -d option, I get the following: /home/jhughes/work/media-catalog/trunk/config/../script/backgroundrb/start:99: private method `new' called for BackgrounDRb::MiddleMan:Class (NoMethodError). Something's not where it's supposed to be, but I'm not sure what. jh ps. Also possibly relevant, when I run the tests I get the following failure: 1) Failure: test_new_worker(BackgrounDRbTest) [./test/test_backgroundrb.rb:43]: expected to be kind_of? but was . -- James Hughes Web application developer Vancouver, BC "Developing a coherent political analysis is in many respects contingent upon an ability to connect one context to another, a process not dissimilar to playing the kid's game of dot-to-dot." - Ward Churchill, from '"A Government of Laws"?' From ezmobius at gmail.com Fri Aug 25 19:04:59 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Fri, 25 Aug 2006 16:04:59 -0700 Subject: [Backgroundrb-devel] Rake tasks fail silently, how to debug? In-Reply-To: <765a2c230608251509j4df07920v11d791f18e269708@mail.gmail.com> References: <765a2c230608251422j4d3bba9bx91aa0650a7abdd88@mail.gmail.com> <1F6E85D2-E1C7-4EBD-9CB4-84AE7AE4A7D6@gmail.com> <765a2c230608251509j4df07920v11d791f18e269708@mail.gmail.com> Message-ID: On Aug 25, 2006, at 3:09 PM, James Hughes wrote: > (Sorry, I keep forgetting reply-all on this list) > > On 8/25/06, Ezra Zygmuntowicz wrote: >> >> On Aug 25, 2006, at 2:22 PM, James Hughes wrote: >> >>> Any hints on how I can get more verbosity, or thoughts on what might >>> be happening? >>> >>> (This happens with both the rails gem and edge; also, I'm tracking >>> backgroundrb via svn:externals) >> >> Hey James- >> >> When did this start happening? If you roll back one >> revision from >> trunk does that fix it? I just did a fresh svn co of trunk and was >> able to run the setup and start tasks without any issues. So please >> try again. Aslo can you use the start script directly instead of the >> rake task? Take a look in the rake task you can see what it does, it >> just copies the script dir up into your rails app and makes a config >> file. I don't know how to make rake more verbose in this case. Maybe >> open up the rakefile and add some puts statements and tell me what >> you find out would you? >> >> -Ezra >> > > Hi Ezra, > > I've done what you suggest. The setup task actually appears to be > behaving correctly: all the scripts and config are already in place so > it's not actually failing, it's just not doing anything. However, when > I run the start script directly (i.e. not with rake), without the -d > option, I get the following: > > /home/jhughes/work/media-catalog/trunk/config/../script/ > backgroundrb/start:99: > private method `new' called for BackgrounDRb::MiddleMan:Class > (NoMethodError). > > Something's not where it's supposed to be, but I'm not sure what. > jh > > ps. Also possibly relevant, when I run the tests I get the > following failure: > > 1) Failure: > test_new_worker(BackgrounDRbTest) [./test/test_backgroundrb.rb:43]: > > expected to be kind_of? > but was > . Hey James- I think that somehow you are getting a few versions mixed up. I recently made the MiddleMan class be a singleton so that accounts for the error of private method new . I think what you should do is remove all traces of the plugin from your app. Except you can leave your workers in lib/workers. But remove everything esle , the plugin the config file and the script/backgroundrb dir from your rails app. Then reinstall the plugin from trunk and rerun the setup task and try again. I think that should fix the problem. It seems like you may have parts of two different versions installed and that is what is causing the errors. Cheers- -Ezra From hughes.james at gmail.com Fri Aug 25 19:08:13 2006 From: hughes.james at gmail.com (James Hughes) Date: Fri, 25 Aug 2006 16:08:13 -0700 Subject: [Backgroundrb-devel] Rake tasks fail silently, how to debug? In-Reply-To: <765a2c230608251509j4df07920v11d791f18e269708@mail.gmail.com> References: <765a2c230608251422j4d3bba9bx91aa0650a7abdd88@mail.gmail.com> <1F6E85D2-E1C7-4EBD-9CB4-84AE7AE4A7D6@gmail.com> <765a2c230608251509j4df07920v11d791f18e269708@mail.gmail.com> Message-ID: <765a2c230608251608l3f2ad80dv2f14faa77772850@mail.gmail.com> > > /home/jhughes/work/media-catalog/trunk/config/../script/backgroundrb/start:99: > private method `new' called for BackgrounDRb::MiddleMan:Class > (NoMethodError). Ok, above was due to old start/stop scripts lying around in rails' script directory. I didn't realize you have to clobber the entire script/backgroundrb directory in order for setup to recopy these. thanks, jh -- James Hughes Web application developer Vancouver, BC "Developing a coherent political analysis is in many respects contingent upon an ability to connect one context to another, a process not dissimilar to playing the kid's game of dot-to-dot." - Ward Churchill, from '"A Government of Laws"?' From lists at foorama.com Fri Aug 25 23:13:08 2006 From: lists at foorama.com (Tim Trautmann) Date: Fri, 25 Aug 2006 20:13:08 -0700 Subject: [Backgroundrb-devel] postgres-pr barfs at ActiveRecord::Base.allow_concurrency = true Message-ID: Hi List, first of thank you Ezra for BackgroundRB. Very cool plugin indeed! This is a public service FYI. In case you are running postgres as your database backend and you are seeing the following errors in your backgroundrb.log: ActiveRecord::StatementInvalid: NoMethodError: undefined method `async_query' for #: You are using the postgres-pr adapter which does not seem to support async_query. You need to install the compiled postgres db adapter wich can be found here: http://rubyforge.org/projects/ruby-postgres/ Or alternatively you could set ActiveRecord::Base.allow_concurrency to "false" in your backgroundrb start script. Though this could have adverse side effects. Thought this might be helpful to others in the future. Best, Tim From vicente.reig at gmail.com Sat Aug 26 13:55:45 2006 From: vicente.reig at gmail.com (Vicente Reig) Date: Sat, 26 Aug 2006 19:55:45 +0200 Subject: [Backgroundrb-devel] Dreamhost issues Message-ID: <835d58c60608261055k15e8eb64q522e47073a7dfc@mail.gmail.com> Hi there! I just have discovered this wonderful plugin a few days ago and I uploaded a little test to my Dreamhost account. I simply got a worker that resizes an image that the user uploads. So this morning I launched the BackgrounDRb server on my account with the rake task and began to upload some test images. The server daemon has been executing until a few minutes ago, I suppose because of Dreamhost's watchdog that killed the process. So my RoR application could be capable of restarting the server if it detects that is down, right? I mean that I think I can code it, but I would like to hear someone's else experience or if it's just a crazy idea :-) I know I could put a spinner near the file_field until the image gets resized and forget about the Workers... but I like the idea of the web application doing some work behind the scenes. If anyone cares, I'm coding a flickr-like application for fun and for my sister's wedding. :-) -- Vicente Reig Rinc?n de Arellano http://www.vicentereig.info/ mvl: 679 980 411 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060826/92afc758/attachment.html From mklein at bpl.org Sun Aug 27 20:52:21 2006 From: mklein at bpl.org (Klein, Michael) Date: Sun, 27 Aug 2006 20:52:21 -0400 Subject: [Backgroundrb-devel] Dreamhost issues References: <835d58c60608261055k15e8eb64q522e47073a7dfc@mail.gmail.com> Message-ID: An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060827/3c937681/attachment.html From bjohnson at contuitive.com Mon Aug 28 03:02:17 2006 From: bjohnson at contuitive.com (Ben Johnson) Date: Mon, 28 Aug 2006 02:02:17 -0500 Subject: [Backgroundrb-devel] Just a tip Message-ID: I figured instead of asking a question I would share some useful knowledge I learned this week. Maybe you already know this, maybe you dont. If you use backgroundrb to run an "infinite" loop, or create many threads, or do quite a bit of processor intensive tasks, and you notice that its running kind of slow, it might not be a bad idea to have multiple backgroundrb processes running, especially if you have a dual core processor or more than 1 processor. I am on a server with 2 dual core processors. I noticed under the "top" command my background process was reaching 99% of the CPU. This was because it was only utilizing 1/4 of the power the server had to offer. Anyways, I modified backgroundrb quite a bit. To the point where I can make a change in the configuration file and allow the start script to create as many processes as I need. Needless to say I saw speed increases of over 10 times what it used to be. All that you have to do is evenly distribute your tasks among the many processes. Maybe this will help someone, who know, but its worth putting into this list. Thank You, Ben Johnson E: bjohnson at contuitive.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060828/7330b756/attachment.html From ezmobius at gmail.com Mon Aug 28 03:28:05 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Mon, 28 Aug 2006 00:28:05 -0700 Subject: [Backgroundrb-devel] Just a tip In-Reply-To: References: Message-ID: <56C58C88-0108-455B-A0FF-77744E4D997D@gmail.com> On Aug 28, 2006, at 12:02 AM, Ben Johnson wrote: > I figured instead of asking a question I would share some useful > knowledge I learned this week. Maybe you already know this, maybe > you dont. > > If you use backgroundrb to run an "infinite" loop, or create many > threads, or do quite a bit of processor intensive tasks, and you > notice that its running kind of slow, it might not be a bad idea to > have multiple backgroundrb processes running, especially if you > have a dual core processor or more than 1 processor. I am on a > server with 2 dual core processors. I noticed under the "top" > command my background process was reaching 99% of the CPU. This was > because it was only utilizing 1/4 of the power the server had to > offer. Anyways, I modified backgroundrb quite a bit. To the point > where I can make a change in the configuration file and allow the > start script to create as many processes as I need. > > Needless to say I saw speed increases of over 10 times what it used > to be. > > All that you have to do is evenly distribute your tasks among the > many processes. > > Maybe this will help someone, who know, but its worth putting into > this list. > > Thank You, > Ben Johnson > E: bjohnson at contuitive.com > > Hey Ben- Care to share your enhancements? -Ezra -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060828/e1744247/attachment-0001.html From martin at nosleepmedia.com Mon Aug 28 21:21:05 2006 From: martin at nosleepmedia.com (Martin May) Date: Mon, 28 Aug 2006 19:21:05 -0600 Subject: [Backgroundrb-devel] Tasks get executed twice with repeat_every Message-ID: <21F9013A-C945-495D-9ED1-83675F7C7ABF@nosleepmedia.com> I have a simple worker that I set up with repeat_every: class DueNoteQueuerWorker < BackgrounDRb::Rails repeat_every 1.minutes first_run Time.now def do_work(args) @logger.debug("Sending due notes. The current time is #{Time.now}") end end I want this to run every minute, all the time. The first run is fine, but it seems that subsequent runs get executed twice. From the logfile: # Logfile created on Mon Aug 28 19:18:01 MDT 2006 by logger.rb/1.5.2.7 Sending due notes. The current time is Mon Aug 28 19:18:03 MDT 2006 Sending due notes. The current time is Mon Aug 28 19:19:03 MDT 2006 Sending due notes. The current time is Mon Aug 28 19:19:03 MDT 2006 Sending due notes. The current time is Mon Aug 28 19:20:03 MDT 2006 Sending due notes. The current time is Mon Aug 28 19:20:03 MDT 2006 Is there any way to fix that? Here is my configuration file: --- port: "22222" timer_sleep: 60 load_rails: true environment: development host: localhost database_yml: config/database.yml acl: deny: all allow: localhost 127.0.0.1 order: deny,allow autostart: 1: job_key: due_note_queuer1 class: due_note_queuer_worker Thanks, Martin From ezmobius at gmail.com Tue Aug 29 14:40:43 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Tue, 29 Aug 2006 11:40:43 -0700 Subject: [Backgroundrb-devel] Tasks get executed twice with repeat_every In-Reply-To: <21F9013A-C945-495D-9ED1-83675F7C7ABF@nosleepmedia.com> References: <21F9013A-C945-495D-9ED1-83675F7C7ABF@nosleepmedia.com> Message-ID: <1350A024-E359-422B-814B-506523019813@gmail.com> Hi~ On Aug 28, 2006, at 6:21 PM, Martin May wrote: > > I have a simple worker that I set up with repeat_every: > > class DueNoteQueuerWorker < BackgrounDRb::Rails > > repeat_every 1.minutes > first_run Time.now > > def do_work(args) > @logger.debug("Sending due notes. The current time is # > {Time.now}") > end > > end > > > I want this to run every minute, all the time. The first run is fine, > but it seems that subsequent runs get executed twice. From the > logfile: > > # Logfile created on Mon Aug 28 19:18:01 MDT 2006 by logger.rb/1.5.2.7 > Sending due notes. The current time is Mon Aug 28 19:18:03 MDT 2006 > Sending due notes. The current time is Mon Aug 28 19:19:03 MDT 2006 > Sending due notes. The current time is Mon Aug 28 19:19:03 MDT 2006 > Sending due notes. The current time is Mon Aug 28 19:20:03 MDT 2006 > Sending due notes. The current time is Mon Aug 28 19:20:03 MDT 2006 > > Is there any way to fix that? Here is my configuration file: > > --- > port: "22222" > timer_sleep: 60 > load_rails: true > environment: development > host: localhost > database_yml: config/database.yml > acl: > deny: all > allow: localhost 127.0.0.1 > order: deny,allow > autostart: > 1: > job_key: due_note_queuer1 > class: due_note_queuer_worker > > Thanks, > Martin Hey Martin- I am working on rewriting the timing mechanism for cron like workers. I had one other person report a similar issue when one worker is still running and the next one kicks off. So for now you can either try to patch it yourself or wait a bit longer until the next release. Cheers- -Ezra From ezmobius at gmail.com Tue Aug 29 14:43:17 2006 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Tue, 29 Aug 2006 11:43:17 -0700 Subject: [Backgroundrb-devel] Dreamhost issues In-Reply-To: References: <835d58c60608261055k15e8eb64q522e47073a7dfc@mail.gmail.com> Message-ID: <08EA73D3-1B96-4082-8D24-5E07492F3747@gmail.com> Hmm yeah that sucks. Dreamhost is know for killing long running tasks. There is a frao handler someone made for fcgi so that when dreamhost tries to tell your fcgi's to die, they will just trap the kill signal and continue on running. It is possible to use this technique with bgdrb. I am busy this week setting up engine yard but next week when I get back home I could try to add something to handle this so it doesn't get killed off. Cheers- -Ezra On Aug 27, 2006, at 5:52 PM, Klein, Michael wrote: > I use RoR/BackgrounDRb on Dreamhost, and I do just what you're > saying -- if I get an "unable to connect" error from the MiddleMan, > I restart the server and retry. After 3 failed retries, I bubble > the exception up. > > Ezra -- is there a way to code this (optional, obviously) autostart > behavior into the MiddleMan class itself? I can never remember > whether the MiddleMan is local and the workers remote, or whether > the MiddleMan itself is undumped. > > Michael > > > -----Original Message----- > From: backgroundrb-devel-bounces at rubyforge.org on behalf of Vicente > Reig > Sent: Sat 8/26/2006 1:55 PM > To: backgroundrb-devel at rubyforge.org > Subject: [Backgroundrb-devel] Dreamhost issues > > Hi there! > I just have discovered this wonderful plugin a few days ago and I > uploaded a > little test to my Dreamhost account. I simply got a worker that > resizes an > image that the user uploads. > So this morning I launched the BackgrounDRb server on my account > with the > rake task and began to upload some test images. The server daemon > has been > executing until a few minutes ago, I suppose because of Dreamhost's > watchdog > that killed the process. So my RoR application could be capable of > restarting the server if it detects that is down, right? I mean > that I think > I can code it, but I would like to hear someone's else experience > or if it's > just a crazy idea :-) > > I know I could put a spinner near the file_field until the image gets > resized and forget about the Workers... but I like the idea of the web > application doing some work behind the scenes. > > If anyone cares, I'm coding a flickr-like application for fun and > for my > sister's wedding. :-) > > -- > Vicente Reig Rinc?n de Arellano > http://www.vicentereig.info/ > mvl: 679 980 411 > > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060829/f74063e2/attachment.html From martin at nosleepmedia.com Tue Aug 29 14:55:00 2006 From: martin at nosleepmedia.com (Martin May) Date: Tue, 29 Aug 2006 12:55:00 -0600 Subject: [Backgroundrb-devel] Environment Setting Message-ID: <86AB824A-0C8D-4FB8-99D9-00988177E0E9@nosleepmedia.com> Looks like the environment setting isn't being applied, it always starts up in development mode for me. Could this have something to do with the recent change to loading the full Rails environment? Martin