[Backgroundrb-devel] How to obtain task progress for long-running methods

hemant gethemant at gmail.com
Wed Nov 19 22:25:36 EST 2008


You need memcache, if you want to use result storing and retrieval to
work, when worker is processing a task.

http://backgroundrb.rubyforge.org/workers/#result_caching

AFAIK, thats all, thats needed to get started with result caching.
All documentation of how/what/why is on backgroundrb.rubyforge.org ,
so read whats there on the website carefully.

At last, Do not swear on mailing lists. You can do that on your blog,
irc (some channels allow, others don't). But NEVER on mailing list, i
hate that.



On Thu, Nov 20, 2008 at 7:27 AM, seth b <subimage at gmail.com> wrote:
> It'd be nice if you had a comprehensive tutorial covering the latest
> "up to date" way to handle these things.
>
> I'm pulling my fucking hair out trying to figure out how to upgrade
> from 0.2.1 and the doc site isn't doing much but confusing me more.
>
> Used to be - you'd create one worker for each task you were running.
> All work was run from my "do_work" methods, and I'd grab progress by
> calling methods on the worker from Rails. Simple enough....
>
> I'm trying to transition to the latest BDRB right now and I'm utterly
> confused. Perhaps something has changed drastically and I'm missing a
> huge concept shift. Right now I've moved my "do_work" method to the
> "create" method, which seems to run fine - but I can't reference any
> of my workers or get any progress back.
>
> Can anyone point to a full tutorial or code example of running an
> async process and grabbing multiple results WITHOUT memcached? Or is
> memcached required now?
>
> Has the paradigm shifted? Do you only create one worker and call jobs
> against that worker and grab results using the cache now?
>
> Someone please point me in the right direction. My emails to the list
> and questions on the irc channel have gone unanswered for the last
> couple of days.
>
> --------------------
> seth - subimage llc
> -----
> http://sublog.subimage.com
> -----
> Cashboard - Estimates, invoices, and time tracking software - for free!
> http://www.getcashboard.com
> -----
> Substruct - Open source RoR e-commerce software.
> http://code.google.com/p/substruct/
>
>
>
> On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at gmail.com> wrote:
>> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at skarven.net> wrote:
>>> Hmm... I'm using the git version of backgroundrb, and currently up-to-
>>> date with the repository; I don't have an "ask_worker" method; I have
>>> "ask_result." Maybe I should be asking: where can I obtain the version
>>> of Backgroundrb that's covered in "The Rails Way," because it seems the
>>> API has changed significantly.
>>>
>>> Anyway, I'm creating new workers via something like
>>>
>>> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key => 'some key
>>> ', :data => {:arguments => values})
>>>
>>> The worker is coded to such that the process logic is called inside the
>>> create method, and it posts progress via cache[worker_key]
>>>
>>>  def create(args = nil)
>>>        run_my_process(args)
>>>  end
>>>
>>>  def run_my_process(args)
>>>    cache[worker_key] = {}
>>>    ... do stuff
>>>    cache[worker_key][:progress] = 25
>>>    ... do more stuff
>>>    cache[worker_key][:progress] = 50
>>>    ... etc
>>>  end
>>>
>>> While this runs, I can see the worker chugging away (because it logs as
>>> it runs,) but if I call
>>>
>>> MiddleMan.worker(:foobar_worker).ask_result('some key')
>>>
>>> while it is running, I get nil.
>>>
>>> Prior to this, I had a prior version of the worker that stayed in
>>> menory. To use that one, I invoked the process via
>>>
>>> MiddleMan.worker(:foobar_worker).async_run_my_process('some key')
>>>
>>> The only difference in the coding of the worker for that version was
>>> that the cache key was job_key rather than worker_key. Calling
>>> ask_results('some key') produced nil until the task was completed, at
>>> which point it contained my end-state results.
>>
>> We sort of grew very conservative about memory usage in 1.1 release
>> and decided to handover this task to memcache. Earlier, bdrb master
>> process used to cache all these results and hence they were readily
>> available. But unless I maintain an LRU based access, I will have hard
>> time keeping memory usage of master process under check.
>>
>> Hence this responsibility was shelved off to worker. But hey, while a
>> worker is processing a task, it can't server cache result, and thus
>> use Memcache, which is out of process. Now even if worker is
>> processing task, you still get your cache result requests served
>> alright.
>>
>> More details can be found here:
>>
>> http://backgroundrb.rubyforge.org/workers/#result_caching
>>
>> Lastly, compared to any book, that page is usually more up to date about stuff.
>> _______________________________________________
>> Backgroundrb-devel mailing list
>> Backgroundrb-devel at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org


More information about the Backgroundrb-devel mailing list