[Backgroundrb-devel] A new QueueWorker class

David Lemstra david at lemstra.ca
Thu Aug 24 00:43:46 EDT 2006


Hello all,
I've come up w/ a worker class that manages queued jobs using a fixed
number of child workers. Well, that's not quite true -- a new worker is
spawned for each job, but you set the total number that may exist at once.

There are three components:
1) queue_worker.rb: The singleton worker that manages the child workers.
You probably want to auto start this. Make sure you give it
:singleton=>true. Read this file for the methods to interact with your
children. (ie. queue_job(), delete_job(), job_progress() )

2) backgroundrb_rails_queue.rb: The super class for the "child workers"
(and uses backgroundrb_rails.rb in turn). This file needs to be included
in background.rb

3) Your child worker, which should be a subclass of
BackgrounDRb::RailsQueue, is otherwise the about same as normal
If it's a big loop, you probably want to use terminate?() on each
iteration and update @progress. Use suicide() at the end to make room
for the next child.

Options: (probably in your backgroundrb.yml)
autostart:
  :queue_key:
    class: queue_worker
    args:
      :num_child_workers: 2
      :child_class: :cost_calculator_worker
      :reQ_on_finish: true
      :singleton: true

:queue_key can be changed to what you want, but it is the permanent key
of the QueueWorker
:num_child_workers: is up to you!
:child_class: your worker class you want as child workers.
:reQ_on_finish: do you want results to be stored in the queue until you
call job_progress!() ?
Note: to be able to access your child jobs w/ the QueueWorker methods,
include a unique :id in your {args} when you queue_job({args})

I'll attach the files. If they don't go through, I'll resend as text.

BTW, This works well enough for me, but I'm learning as I go too, so no
guarantees :) I don't use the fancy timing options, so ymmmv for
:next_start and :interval.

Let me know if you find any issues (though I'm off-line for a week after
this post). I'm wondering myself if it might be better to reuse child
workers instead of re-spawning new ones. Another day maybe.

cheers,
David Lemstra
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: queue_worker.rb
Url: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060824/2349e2f4/attachment.ksh 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: backgroundrb_rails_queue.rb
Url: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060824/2349e2f4/attachment-0001.ksh 


More information about the Backgroundrb-devel mailing list