[Backgroundrb-devel] 1.0.4, memcache, ask_result, and reload_on_schedule

hemant gethemant at gmail.com
Fri Aug 29 18:37:27 EDT 2008


Well this is because, when a worker is restarted on schedule, its
normally started with a unique worker key and what gets used in
memcache as key is a combination of worker name, worker key(if it
exists). So for workers with reload_on_schedule workers, you don't
really know the key and hence you can't query results, just using
worker name.

Its quite possible to change this behaviour. For example, you can
modify master_proxy.rb so as it doesn't use any unique key for workers
started on schedule (see load_and_invoke method).

But this will be obtrusive, hence, I will try to think of a mechanism
so as it co-exists cleanly.


On Fri, Aug 29, 2008 at 10:33 PM, Tom Wood <thomas.a.wood at uconn.edu> wrote:
> Hi,
>
>
>
> Am finally getting around to upgrading our backgroundrb from 1.0.1 to the
> current SVN release (1.0.4?).  Most things are working quite well.
>
>
>
> But …  I was looking forward to using the reload_on_schedule option on
> several memory hungry workers that don't run that often, but am running into
> a problem: even using memcached for result storage, I'm unable to see cached
> results from workers started with 'reload_on_schedule true'.
>
>
>
> I created two nearly identical workers:
>
>
>
> class HarvesterWorker < BackgrounDRb::MetaWorker
>
>   set_worker_name :harvester_worker
>
>   reload_on_schedule true
>
>
>
>   def create(args = nil)
>
>     @counter = 0
>
>     logger.info "Harvester created (pid #{Process.pid})"
>
>     cache['result'] = @counter
>
>   end
>
>
>
>   def run(args = nil)
>
>     @counter += 1
>
>     logger.info "Harvester run called: #{@counter}"
>
>     cache['result'] = @counter
>
>   end
>
> end
>
>
>
> and
>
>
>
> class Harvester2Worker < BackgrounDRb::MetaWorker
>
>   set_worker_name :harvester2_worker
>
>
>
>   def create(args = nil)
>
>     @counter = 0
>
>     logger.info "Harvester2 created (pid #{Process.pid})"
>
>     cache['result'] = @counter
>
>   end
>
>
>
>   def run(args = nil)
>
>     @counter += 1
>
>     logger.info "Harvester2 run called: #{@counter}"
>
>     cache['result'] = @counter
>
>   end
>
> end
>
>
>
> The only real difference between the workers is that HarvesterWorker uses
> the reload_on_schedule option, and Harvester2Worker does not.
>
> I scheduled each worker to exec the run method every 15 seconds, and
> configured backgroundrb to use memcached for result storage.   The log file
> shows the workers are running correctly.  However:
>
>
>
> MiddleMan.worker(:harvester_worker).ask_result('result') always returns nil,
> while
>
>
>
> MiddleMan.worker(:harvester2_worker).ask_result('result') returns a correct
> result.
>
>
>
> I.e., the worker running with 'reload_on_schedule true' doesn't seem to be
> caching the result.
>
>
>
> Any thoughts?  Thanks!
>
>
>
> Tom Wood
>
> thomas.a.wood at uconn.edu
>
> ITS Applications Developer
>
> University of Connecticut Libraries
>
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org


More information about the Backgroundrb-devel mailing list