[Backgroundrb-devel] How to obtain task progress for long-running methods

Samer Masry enzodm at gmail.com
Thu Nov 20 01:31:05 EST 2008


There is no language policy.  Just curtosy.  We are all  
hackers( reverse engineers ).  It's not always easy but we're  
persistent.  If you can help with documentation or code please do it's  
how apps evolve.

On Nov 19, 2008, at 7:45 PM, "seth b" <subimage at gmail.com> wrote:

> Give me a fucking break. What are you the internet language police?
>
> No wonder everyone I know is switching to workling / starling. Not
> only does the BDRB documentation suck, the support on the mailing list
> sucks even worse.
>
> I'll go with Ezra's recommendation and move along. Thanks for the
> non-help, dickhead.
>
> --------------------
> seth - subimage llc
> -----
> http://sublog.subimage.com
> -----
> Cashboard - Estimates, invoices, and time tracking software - for  
> free!
> http://www.getcashboard.com
> -----
> Substruct - Open source RoR e-commerce software.
> http://code.google.com/p/substruct/
>
>
>
> On Wed, Nov 19, 2008 at 7:25 PM, hemant <gethemant at gmail.com> wrote:
>> You need memcache, if you want to use result storing and retrieval to
>> work, when worker is processing a task.
>>
>> http://backgroundrb.rubyforge.org/workers/#result_caching
>>
>> AFAIK, thats all, thats needed to get started with result caching.
>> All documentation of how/what/why is on backgroundrb.rubyforge.org ,
>> so read whats there on the website carefully.
>>
>> At last, Do not swear on mailing lists. You can do that on your blog,
>> irc (some channels allow, others don't). But NEVER on mailing list, i
>> hate that.
>>
>>
>>
>> On Thu, Nov 20, 2008 at 7:27 AM, seth b <subimage at gmail.com> wrote:
>>> It'd be nice if you had a comprehensive tutorial covering the latest
>>> "up to date" way to handle these things.
>>>
>>> I'm pulling my fucking hair out trying to figure out how to upgrade
>>> from 0.2.1 and the doc site isn't doing much but confusing me more.
>>>
>>> Used to be - you'd create one worker for each task you were running.
>>> All work was run from my "do_work" methods, and I'd grab progress by
>>> calling methods on the worker from Rails. Simple enough....
>>>
>>> I'm trying to transition to the latest BDRB right now and I'm  
>>> utterly
>>> confused. Perhaps something has changed drastically and I'm  
>>> missing a
>>> huge concept shift. Right now I've moved my "do_work" method to the
>>> "create" method, which seems to run fine - but I can't reference any
>>> of my workers or get any progress back.
>>>
>>> Can anyone point to a full tutorial or code example of running an
>>> async process and grabbing multiple results WITHOUT memcached? Or is
>>> memcached required now?
>>>
>>> Has the paradigm shifted? Do you only create one worker and call  
>>> jobs
>>> against that worker and grab results using the cache now?
>>>
>>> Someone please point me in the right direction. My emails to the  
>>> list
>>> and questions on the irc channel have gone unanswered for the last
>>> couple of days.
>>>
>>> --------------------
>>> seth - subimage llc
>>> -----
>>> http://sublog.subimage.com
>>> -----
>>> Cashboard - Estimates, invoices, and time tracking software - for  
>>> free!
>>> http://www.getcashboard.com
>>> -----
>>> Substruct - Open source RoR e-commerce software.
>>> http://code.google.com/p/substruct/
>>>
>>>
>>>
>>> On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at gmail.com> wrote:
>>>> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at skarven.net 
>>>> > wrote:
>>>>> Hmm... I'm using the git version of backgroundrb, and currently  
>>>>> up-to-
>>>>> date with the repository; I don't have an "ask_worker" method; I  
>>>>> have
>>>>> "ask_result." Maybe I should be asking: where can I obtain the  
>>>>> version
>>>>> of Backgroundrb that's covered in "The Rails Way," because it  
>>>>> seems the
>>>>> API has changed significantly.
>>>>>
>>>>> Anyway, I'm creating new workers via something like
>>>>>
>>>>> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key =>  
>>>>> 'some key
>>>>> ', :data => {:arguments => values})
>>>>>
>>>>> The worker is coded to such that the process logic is called  
>>>>> inside the
>>>>> create method, and it posts progress via cache[worker_key]
>>>>>
>>>>> def create(args = nil)
>>>>>       run_my_process(args)
>>>>> end
>>>>>
>>>>> def run_my_process(args)
>>>>>   cache[worker_key] = {}
>>>>>   ... do stuff
>>>>>   cache[worker_key][:progress] = 25
>>>>>   ... do more stuff
>>>>>   cache[worker_key][:progress] = 50
>>>>>   ... etc
>>>>> end
>>>>>
>>>>> While this runs, I can see the worker chugging away (because it  
>>>>> logs as
>>>>> it runs,) but if I call
>>>>>
>>>>> MiddleMan.worker(:foobar_worker).ask_result('some key')
>>>>>
>>>>> while it is running, I get nil.
>>>>>
>>>>> Prior to this, I had a prior version of the worker that stayed in
>>>>> menory. To use that one, I invoked the process via
>>>>>
>>>>> MiddleMan.worker(:foobar_worker).async_run_my_process('some key')
>>>>>
>>>>> The only difference in the coding of the worker for that version  
>>>>> was
>>>>> that the cache key was job_key rather than worker_key. Calling
>>>>> ask_results('some key') produced nil until the task was  
>>>>> completed, at
>>>>> which point it contained my end-state results.
>>>>
>>>> We sort of grew very conservative about memory usage in 1.1 release
>>>> and decided to handover this task to memcache. Earlier, bdrb master
>>>> process used to cache all these results and hence they were readily
>>>> available. But unless I maintain an LRU based access, I will have  
>>>> hard
>>>> time keeping memory usage of master process under check.
>>>>
>>>> Hence this responsibility was shelved off to worker. But hey,  
>>>> while a
>>>> worker is processing a task, it can't server cache result, and thus
>>>> use Memcache, which is out of process. Now even if worker is
>>>> processing task, you still get your cache result requests served
>>>> alright.
>>>>
>>>> More details can be found here:
>>>>
>>>> http://backgroundrb.rubyforge.org/workers/#result_caching
>>>>
>>>> Lastly, compared to any book, that page is usually more up to  
>>>> date about stuff.
>>>> _______________________________________________
>>>> Backgroundrb-devel mailing list
>>>> Backgroundrb-devel at rubyforge.org
>>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>>>
>>> _______________________________________________
>>> Backgroundrb-devel mailing list
>>> Backgroundrb-devel at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>>
>>
>>
>>
>> --
>> Let them talk of their oriental summer climes of everlasting
>> conservatories; give me the privilege of making my own summer with my
>> own coals.
>>
>> http://gnufied.org
>>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel


More information about the Backgroundrb-devel mailing list