[Backgroundrb-devel] Best way to have several single-threaded workers handling job queue?

Cassidy Forbes caseyforbes at comcast.net
Mon Jul 28 16:04:23 EDT 2008


Hi all,

I have jobs that need to be queued up and run in a single thread 
(the labor intensive part of the job uses a library that is not thread 
safe)

Right now, I have a single worker. The thread pool_size = 1 but jobs 
are put in using thread_pool.defer so that backgroundrb takes care of 
the queue for me.  This works fine except that the queue gets larger 
than I would like.  I'd like to have multiple identical (single 
threaded, own Rais environment) workers sharing a queue and processing 
the jobs.

I could start up several workers with different worker_keys and 
distribute jobs on the client side, but this seems a little messy. Are 
there any better options?

I'm using the latest backgroundrb/packet from git and memcached for 
the result cache.

Casey


More information about the Backgroundrb-devel mailing list