[Backgroundrb-devel] best approach to managing workers and getting status

Sean O'Hara sohara at sohara.com
Sun May 4 12:56:15 EDT 2008


I am using backgroundrb to process audio files from a rails  
controller. Currently a new worker gets created every time the method  
is called on the worker, using this code:

@job_key = MiddleMan.new_worker(:worker  
=> :audio_file_worker, :job_key => Time.now.to_i)

I need to create the new worker each time in order to be able to get  
the status from the worker using the job key (this info is returned to  
the client using ajax requests). But this means that I end up with  
many workers eating up memory, and just hanging around after their  
jobs are complete.

I am planning to just kill them from the controller each time the  
status returns that they are complete. This will prevent the extra  
processes from hanging around and using memory, but I guess it still  
entails some 'costs' in starting up the new worker each time, since it  
contains a rails instance?

Is there a better approach, e.g. just having one worker, and sending  
the jobs to the same worker? If so, how do we keep track of statuses  
of unique jobs in this case? Since the job_key is created when  
creating a new worker, isn't in effect a worker key, rather than a job  

I did read about this different approach discussed here (using  
thread_pool.defer) but it doesn't seem to allow for getting status of  
the unique threads, as far as I can tell: http://rubyforge.org/pipermail/backgroundrb-devel/2008-February/001532.html

Any guidance is appreciated.


More information about the Backgroundrb-devel mailing list