[Backgroundrb-devel] Configure multiple workers for persistent job queue
gethemant at gmail.com
Tue Jan 6 21:47:07 EST 2009
2009/1/7 Mark D <buenos79 at hotmail.com>:
> I'm using a persistent job queue with a single worker thread to process the
> jobs. However the jobs are being created faster than the worker can handle
> them. Is it possible to configure backgroundrb to start multiple copies of
> the same worker to pick up these jobs?
1. You can use one worker as master (the one that takes jobs out of
the queue) and have that worker start multiple copies of itself (using
MiddleMan.new_worker) and hand over the jobs to these workers.
2. There isn't a builtin functionality of auto-adjusting worker count
if number of tasks in the queue is high. But if your tasks are
inherently IO bound, you can use remove item from queue and run it
inside thread pool.
More information about the Backgroundrb-devel