[Backgroundrb-devel] Need help with understanding how thread_pool/pool_size work

Raghu Srinivasan raghu.srinivasan at gmail.com
Wed Dec 31 12:15:49 EST 2008


I'm on BDRB v 1.0.3 and am trying to understand how thread_pool and
pool_size work. I have to kick of dozens of jobs on schedule (each hour or
so). Other than the fact that they're all accessing the same DB there's no
reason for them to not run in parallel. thread_pool/pool_size should be the
way to go right?

I've put in sample code with its results below:

My Rails controller kicks off 5 jobs in a loop - each calling the
run_concurrent method in my foo worker. run_concurrent then calls
my_actual_method which just logger.infos a message with a time stamp and
sleeps for 5 seconds to simulate a long running job. I did this as per
http://backgroundrb.rubyforge.org/workers/#thread_pool
<http://backgroundrb.rubyforge.org/workers/#thread_pool>Since
I'm calling this via a defer and have a pool size of 10, I expect to see
that my_actual_method actually gets called 5 times in quick succession
(since the pool size is greater than the # of calls). However I find that
run_concurrent doesn't even call my_actual_method. Here's the output from my
backgroundrb log when I go to http://mysite.com/some/foobar

Can someone help me understand what I'm doing wrong here?

Thanks,

Raghu
====================================================
Rails controller code
class SomeController < ApplicationController
        def foobar
                i = 0
                while i < 5
                        worker = MiddleMan.worker(:foo_worker)
                        result = worker.run_concurrent(:job_key =>
random_string(10))
                        i += 1
                end
                render :text => 'all done at ' + Time.now.to_s

        end
end
====================================================
Worker code
class FooWorker < BackgrounDRb::MetaWorker

        set_worker_name :foo_worker
        pool_size 10

        def create
                # this method is called, when worker is loaded for the first
time
        end

        def run_concurrent(args)
                logger.info "*** FOO_WORKER/RUN_CONCURRENT at " +
Time.now.to_s
                thread_pool.defer(:my_actual_method)
        end

        def my_actual_method(args)
                logger.info "*** FOO_WORKER/MY_ACTUAL_METHOD at " +
Time.now.to_s
                sleep 5
        end
end
====================================================
Here's the output
# Logfile created on Wed Dec 31 09:15:21 +0000 2008 by /
foo_worker started (pid:5087)
Schedules for worker loaded (pid:5087)
run_concurrent job_keydfvq5o4m0s (pid:5087)
*** FOO_WORKER/RUN_CONCURRENT at Wed Dec 31 09:15:26 +0000 2008 (pid:5087)
run_concurrent job_keyy4tascam6k (pid:5087)
*** FOO_WORKER/RUN_CONCURRENT at Wed Dec 31 09:15:26 +0000 2008 (pid:5087)
run_concurrent job_key8gw2eegeqs (pid:5087)
*** FOO_WORKER/RUN_CONCURRENT at Wed Dec 31 09:15:26 +0000 2008 (pid:5087)
run_concurrent job_keyywb1oop73t (pid:5087)
*** FOO_WORKER/RUN_CONCURRENT at Wed Dec 31 09:15:26 +0000 2008 (pid:5087)
run_concurrent job_key17gfkqtzjh (pid:5087)
*** FOO_WORKER/RUN_CONCURRENT at Wed Dec 31 09:15:26 +0000 2008 (pid:5087)
====================================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081231/12a7e9bf/attachment.html>


More information about the Backgroundrb-devel mailing list