[Backgroundrb-devel] Need help with singleton worker

Isak Hansen isak.hansen at gmail.com
Tue May 15 08:16:26 EDT 2007


We have a fairly expensive task that we'd rather not schedule too many
instances of in parallel.

Couldn't get pool_size to limit the amount of workers, and I figure
that's hardly optimal anyway, so would like to have the work queued up
for a single named worker instead, running as soon as possible.

Using MiddleMan.schedule_worker() with :job_key does start off a
worker, but I can't manage to add another task to the same worker and
have it queue up. It's not just a matter of adding :worker_method =>
:do_work/:worker_method_args => whatever to schedule_worker()..? (see
example below)

Ideally using the same interface for running the first and any subsequent tasks.




Here's a debug run, which may or may not make any sense.

Task a, b and c were started in 10-20 sec total,  d was started after
a and b finished.

As you can see, b isn't queued up, and c/d never start. I've also had
'b' start but never finish. (May have been due to do_work deleting the
worker as 'a' finished in that case.. figured delete was neccessary to
have the other jobs start, but no..)




lib/workers/my_worker:
class MyWorker < BackgrounDRb::Worker::RailsBase

  def do_work(args)
    logger.info("MyWorker#do_work. args: #{args}")
    sleep(60)
    logger.info("MyWorker#work done, deleting self. args: #{args}")
    # probably shouldn't call self.delete here, as we reuse the worker..?
  end

end
MyWorker.register




script/console:
>> MiddleMan.schedule_worker(:class => :my_worker, :job_key => :foo,
:worker_method => :do_work, :worker_method_args => "a", :args =>
"_a_", :trigger_args => { :start => Time.now })
=> 0
>> MiddleMan.schedule_worker(:class => :my_worker, :job_key => :foo,
:worker_method => :do_work, :worker_method_args => "b", :args =>
"_b_", :trigger_args => { :start => Time.now })
=> 1
>> MiddleMan.schedule_worker(:class => :my_worker, :job_key => :foo,
:worker_method => :do_work, :worker_method_args => "c", :args =>
"_c_", :trigger_args => { :start => Time.now })
=> 2
>> MiddleMan.schedule_worker(:class => :my_worker, :job_key => :foo,
:worker_method => :do_work, :worker_method_args => "d", :args =>
"_d_", :trigger_args => { :start => Time.now })
=> 3
>>




bdrb log #1:
20070515-13:55:27 (10865) MyWorker#do_work. args: _a_
20070515-13:55:32 (10865) MyWorker#do_work. args: b
20070515-13:56:27 (10865) MyWorker#work done, deleting self. args: _a_
20070515-13:56:32 (10865) MyWorker#work done, deleting self. args: b




bdrb log #2:
20070515-13:55:22 (10864) Loading Sechedule:
job_keyfooargs_a_classmy_worker
trigger_typetriggertrigger_argsstartTue May 15 13:55:22 CEST
2007worker_methoddo_workworker_method_argsajob_keyfooclassmy_workerargs_a_
#<BackgrounDRb::Trigger:0xb7b60054>
20070515-13:55:23 (10864) Schedule triggered: #<struct
#<Class:0xb7ddb4f0>
job=#<Proc:0xb7cfb7d8@/var/www/html/apps/xxx/vendor/plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:319>,
trigger=#<BackgrounDRb::Trigger:0xb7b60054 @start_time=Tue May 15
13:55:22 CEST 2007, @repeat_interval=1, @end_time=Tue May 15 13:55:23
CEST 2007>, earliest=Tue May 15 13:55:23 CEST 2007, last=Tue May 15
13:55:23 CEST 2007>
20070515-13:55:27 (10864) Starting worker: my_worker foo (my_worker_foo) (_a_)
20070515-13:55:31 (10864) Loading Sechedule:
job_keyfooargs_b_classmy_worker
trigger_typetriggertrigger_argsstartTue May 15 13:55:31 CEST
2007worker_methoddo_workworker_method_argsbjob_keyfooclassmy_workerargs_b_
#<BackgrounDRb::Trigger:0xb7874ae8>
20070515-13:55:32 (10864) Schedule triggered: #<struct
#<Class:0xb7ddb4f0>
job=#<Proc:0xb7cfb7d8@/var/www/html/apps/xxx/vendor/plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:319>,
trigger=#<BackgrounDRb::Trigger:0xb7874ae8 @start_time=Tue May 15
13:55:31 CEST 2007, @repeat_interval=1, @end_time=Tue May 15 13:55:32
CEST 2007>, earliest=Tue May 15 13:55:32 CEST 2007, last=Tue May 15
13:55:32 CEST 2007>
20070515-13:55:40 (10864) Loading Sechedule:
job_keyfooargs_c_classmy_worker
trigger_typetriggertrigger_argsstartTue May 15 13:55:40 CEST
2007worker_methoddo_workworker_method_argscjob_keyfooclassmy_workerargs_c_
#<BackgrounDRb::Trigger:0xb786eb48>
20070515-13:56:03 (10864) Loading Sechedule:
job_keyfooargs_d_classmy_worker
trigger_typetriggertrigger_argsstartTue May 15 13:56:03 CEST
2007worker_methoddo_workworker_method_argsdjob_keyfooclassmy_workerargs_d_
#<BackgrounDRb::Trigger:0xb786c44c>


More information about the Backgroundrb-devel mailing list