[Backgroundrb-devel] How to obtain task progress for long-running methods

seth b subimage at gmail.com
Wed Nov 19 20:57:24 EST 2008

It'd be nice if you had a comprehensive tutorial covering the latest
"up to date" way to handle these things.

I'm pulling my fucking hair out trying to figure out how to upgrade
from 0.2.1 and the doc site isn't doing much but confusing me more.

Used to be - you'd create one worker for each task you were running.
All work was run from my "do_work" methods, and I'd grab progress by
calling methods on the worker from Rails. Simple enough....

I'm trying to transition to the latest BDRB right now and I'm utterly
confused. Perhaps something has changed drastically and I'm missing a
huge concept shift. Right now I've moved my "do_work" method to the
"create" method, which seems to run fine - but I can't reference any
of my workers or get any progress back.

Can anyone point to a full tutorial or code example of running an
async process and grabbing multiple results WITHOUT memcached? Or is
memcached required now?

Has the paradigm shifted? Do you only create one worker and call jobs
against that worker and grab results using the cache now?

Someone please point me in the right direction. My emails to the list
and questions on the irc channel have gone unanswered for the last
couple of days.

seth - subimage llc
Cashboard - Estimates, invoices, and time tracking software - for free!
Substruct - Open source RoR e-commerce software.

On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at gmail.com> wrote:
> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at skarven.net> wrote:
>> Hmm... I'm using the git version of backgroundrb, and currently up-to-
>> date with the repository; I don't have an "ask_worker" method; I have
>> "ask_result." Maybe I should be asking: where can I obtain the version
>> of Backgroundrb that's covered in "The Rails Way," because it seems the
>> API has changed significantly.
>> Anyway, I'm creating new workers via something like
>> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key => 'some key
>> ', :data => {:arguments => values})
>> The worker is coded to such that the process logic is called inside the
>> create method, and it posts progress via cache[worker_key]
>>  def create(args = nil)
>>        run_my_process(args)
>>  end
>>  def run_my_process(args)
>>    cache[worker_key] = {}
>>    ... do stuff
>>    cache[worker_key][:progress] = 25
>>    ... do more stuff
>>    cache[worker_key][:progress] = 50
>>    ... etc
>>  end
>> While this runs, I can see the worker chugging away (because it logs as
>> it runs,) but if I call
>> MiddleMan.worker(:foobar_worker).ask_result('some key')
>> while it is running, I get nil.
>> Prior to this, I had a prior version of the worker that stayed in
>> menory. To use that one, I invoked the process via
>> MiddleMan.worker(:foobar_worker).async_run_my_process('some key')
>> The only difference in the coding of the worker for that version was
>> that the cache key was job_key rather than worker_key. Calling
>> ask_results('some key') produced nil until the task was completed, at
>> which point it contained my end-state results.
> We sort of grew very conservative about memory usage in 1.1 release
> and decided to handover this task to memcache. Earlier, bdrb master
> process used to cache all these results and hence they were readily
> available. But unless I maintain an LRU based access, I will have hard
> time keeping memory usage of master process under check.
> Hence this responsibility was shelved off to worker. But hey, while a
> worker is processing a task, it can't server cache result, and thus
> use Memcache, which is out of process. Now even if worker is
> processing task, you still get your cache result requests served
> alright.
> More details can be found here:
> http://backgroundrb.rubyforge.org/workers/#result_caching
> Lastly, compared to any book, that page is usually more up to date about stuff.
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

More information about the Backgroundrb-devel mailing list