[Backgroundrb-devel] Need help with understanding how thread_pool/pool_size work

Raghu Srinivasan raghu.srinivasan at gmail.com
Sun Jan 4 03:20:59 EST 2009


Does anyone else have any ideas for how to fix this? If someone has sample
code that they don't mind sharing, I'd really appreciate taking a look at
some real code that does the pool_size/thread_pool magic.
Raghu

On Wed, Dec 31, 2008 at 3:00 PM, Raghu Srinivasan <
raghu.srinivasan at gmail.com> wrote:

> No luck Hemant. I changed my worker as follows, no error, I get the exact
> same o/p as before. Where might I be messing up?Thanks!
> Raghu
> ============================
> class FooWorker < BackgrounDRb::MetaWorker
>
>         set_worker_name :foo_worker
>         pool_size 10
>
>         def create
>                 # this method is called, when worker is loaded for the
> first time
>         end
>
>         def run_concurrent(args)
>                 logger.info "*** FOO_WORKER/RUN_CONCURRENT at " +
> Time.now.to_s
>                 thread_pool.defer(:my_actual_method,:x => args[:job_key])
>         end
>
>         def my_actual_method(args)
>                 logger.info "*** FOO_WORKER/MY_ACTUAL_METHOD " + args[:x]
> + " at " + Time.now.to_s
>                 sleep 5
>         end
> end
> ============================
>
> On Wed, Dec 31, 2008 at 2:43 PM, Raghu Srinivasan <
> raghu.srinivasan at gmail.com> wrote:
>
>> No, I don't get any errors at all. Why is it mandatory for the method that
>> I want to run concurrently to receive an argument? In the real case of
>> course I'll be passing some equivalent of an ID but let me try with that and
>> see...
>>
>>
>> On Wed, Dec 31, 2008 at 1:48 PM, hemant <gethemant at gmail.com> wrote:
>>
>>> Could it be, because method that runs in thread expects an argument
>>> and you are not passing that argument when using thread_pool.defer()
>>> ??
>>>
>>> Did you not get any errors in log file?
>>>
>>>
>>>
>>> On Wed, Dec 31, 2008 at 10:45 PM, Raghu Srinivasan
>>> <raghu.srinivasan at gmail.com> wrote:
>>> > I'm on BDRB v 1.0.3 and am trying to understand how thread_pool and
>>> > pool_size work. I have to kick of dozens of jobs on schedule (each hour
>>> or
>>> > so). Other than the fact that they're all accessing the same DB there's
>>> no
>>> > reason for them to not run in parallel. thread_pool/pool_size should be
>>> the
>>> > way to go right?
>>> > I've put in sample code with its results below:
>>> > My Rails controller kicks off 5 jobs in a loop - each calling the
>>> > run_concurrent method in my foo worker. run_concurrent then calls
>>> > my_actual_method which just logger.infos a message with a time stamp
>>> and
>>> > sleeps for 5 seconds to simulate a long running job. I did this as
>>> > per http://backgroundrb.rubyforge.org/workers/#thread_pool Since I'm
>>> calling
>>> > this via a defer and have a pool size of 10, I expect to see that
>>> > my_actual_method actually gets called 5 times in quick succession
>>> (since the
>>> > pool size is greater than the # of calls). However I find that
>>> > run_concurrent doesn't even call my_actual_method. Here's the output
>>> from my
>>> > backgroundrb log when I go to http://mysite.com/some/foobar
>>> > Can someone help me understand what I'm doing wrong here?
>>> > Thanks,
>>> > Raghu
>>> > ====================================================
>>> > Rails controller code
>>> > class SomeController < ApplicationController
>>> >         def foobar
>>> >                 i = 0
>>> >                 while i < 5
>>> >                         worker = MiddleMan.worker(:foo_worker)
>>> >                         result = worker.run_concurrent(:job_key =>
>>> > random_string(10))
>>> >                         i += 1
>>> >                 end
>>> >                 render :text => 'all done at ' + Time.now.to_s
>>> >         end
>>> > end
>>> > ====================================================
>>> > Worker code
>>> > class FooWorker < BackgrounDRb::MetaWorker
>>> >         set_worker_name :foo_worker
>>> >         pool_size 10
>>> >         def create
>>> >                 # this method is called, when worker is loaded for the
>>> first
>>> > time
>>> >         end
>>> >         def run_concurrent(args)
>>> >                 logger.info "*** FOO_WORKER/RUN_CONCURRENT at " +
>>> > Time.now.to_s
>>> >                 thread_pool.defer(:my_actual_method)
>>> >         end
>>> >         def my_actual_method(args)
>>> >                 logger.info "*** FOO_WORKER/MY_ACTUAL_METHOD at " +
>>> > Time.now.to_s
>>> >                 sleep 5
>>> >         end
>>> > end
>>> > ====================================================
>>> > Here's the output
>>> > # Logfile created on Wed Dec 31 09:15:21 +0000 2008 by /
>>> > foo_worker started (pid:5087)
>>> > Schedules for worker loaded (pid:5087)
>>> > run_concurrent job_keydfvq5o4m0s (pid:5087)
>>> > *** FOO_WORKER/RUN_CONCURRENT at Wed Dec 31 09:15:26 +0000 2008
>>> (pid:5087)
>>> > run_concurrent job_keyy4tascam6k (pid:5087)
>>> > *** FOO_WORKER/RUN_CONCURRENT at Wed Dec 31 09:15:26 +0000 2008
>>> (pid:5087)
>>> > run_concurrent job_key8gw2eegeqs (pid:5087)
>>> > *** FOO_WORKER/RUN_CONCURRENT at Wed Dec 31 09:15:26 +0000 2008
>>> (pid:5087)
>>> > run_concurrent job_keyywb1oop73t (pid:5087)
>>> > *** FOO_WORKER/RUN_CONCURRENT at Wed Dec 31 09:15:26 +0000 2008
>>> (pid:5087)
>>> > run_concurrent job_key17gfkqtzjh (pid:5087)
>>> > *** FOO_WORKER/RUN_CONCURRENT at Wed Dec 31 09:15:26 +0000 2008
>>> (pid:5087)
>>> > ====================================================
>>> >
>>> > _______________________________________________
>>> > Backgroundrb-devel mailing list
>>> > Backgroundrb-devel at rubyforge.org
>>> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>> >
>>>
>>>
>>>
>>> --
>>> Let them talk of their oriental summer climes of everlasting
>>> conservatories; give me the privilege of making my own summer with my
>>> own coals.
>>>
>>> http://gnufied.org
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20090104/c79388ed/attachment.html>


More information about the Backgroundrb-devel mailing list