[Backgroundrb-devel] best approach to managing workers and getting status

hemant gethemant at gmail.com
Wed Jul 16 14:19:44 EDT 2008

On Wed, Jul 16, 2008 at 11:40 PM, Frank Schwach <f.schwach at uea.ac.uk> wrote:
> Hi Hemant and others,
> Once again thank you for your reply to my last post and the excellent
> work you are doing! Having seen your announcement of all those new
> features really convinced me to go ahead and use backgroundrb for my
> app. However, I must admit being quite confused now with all the latest
> changes to the API and would appreciate some help getting my head around
> this. So, basically, I have an app with a database table of long running
> tasks. There are several different types of tasks, so in my table I have
> a column for the type of job to be run and another for serialized data
> for the job (basically some IDs and args). I want to run the actual jobs
> on a remote machine, which the new version of backgroundrb seems to have
> covered nicely now.
> So this is what I thought I could do:
> Have a pool of x RemoteJobWorkers on the remote machine waiting to
> accept jobs:
> class RemoteJobWorker
>  def enque_job (job_type, job_args)
>    thread_pool.defer(:job_type, job_args)
>  end
>  def job_type1 (args)
>    # update job table with status
>    Jobs.update(job_key, :status => running)
>    # perform the long running task
>    large_dataset.each do
>      # update job table with progress
>      Jobs.update(job_key, :progress => x)
>    end
>    # job done: update status again
>    Jobs.update(job_key, :status => complete)
>  end
>  def jobtype2 (args) ... end
>  def jobtype3 (args) ... end
>  .. and so on ..
> end
> So when a job is submitted by a user, the controller sends it to the
> queue on the remote machine like this(?):
> MiddleMan.worker(:remote_job_worker).async_enque_job(:arg =>
> data,:job_key => new_job.id,:host => "a.b.c.d:XXX")
> where new_job is the newly created instance of my Jobs model.

I see you are rolling out your own table based queue here, why not
just use inbuilt persistent database queue?

For the worker, read this page:


Persistent job queue is explained at the bottom.

For adding tasks to the queue you need to do:

MiddleMan.worker(:remote_worker).enq_some_job(:arg => some_arg)

Task will be added to the db queue and automatically picked up.
Handles hairy race conditions for you and stuff like that.

More information about the Backgroundrb-devel mailing list