[Backgroundrb-devel] How to obtain task progress for long-running methods

seth b subimage at gmail.com
Wed Nov 19 22:45:00 EST 2008


Give me a fucking break. What are you the internet language police?

No wonder everyone I know is switching to workling / starling. Not
only does the BDRB documentation suck, the support on the mailing list
sucks even worse.

I'll go with Ezra's recommendation and move along. Thanks for the
non-help, dickhead.

--------------------
seth - subimage llc
-----
http://sublog.subimage.com
-----
Cashboard - Estimates, invoices, and time tracking software - for free!
http://www.getcashboard.com
-----
Substruct - Open source RoR e-commerce software.
http://code.google.com/p/substruct/



On Wed, Nov 19, 2008 at 7:25 PM, hemant <gethemant at gmail.com> wrote:
> You need memcache, if you want to use result storing and retrieval to
> work, when worker is processing a task.
>
> http://backgroundrb.rubyforge.org/workers/#result_caching
>
> AFAIK, thats all, thats needed to get started with result caching.
> All documentation of how/what/why is on backgroundrb.rubyforge.org ,
> so read whats there on the website carefully.
>
> At last, Do not swear on mailing lists. You can do that on your blog,
> irc (some channels allow, others don't). But NEVER on mailing list, i
> hate that.
>
>
>
> On Thu, Nov 20, 2008 at 7:27 AM, seth b <subimage at gmail.com> wrote:
>> It'd be nice if you had a comprehensive tutorial covering the latest
>> "up to date" way to handle these things.
>>
>> I'm pulling my fucking hair out trying to figure out how to upgrade
>> from 0.2.1 and the doc site isn't doing much but confusing me more.
>>
>> Used to be - you'd create one worker for each task you were running.
>> All work was run from my "do_work" methods, and I'd grab progress by
>> calling methods on the worker from Rails. Simple enough....
>>
>> I'm trying to transition to the latest BDRB right now and I'm utterly
>> confused. Perhaps something has changed drastically and I'm missing a
>> huge concept shift. Right now I've moved my "do_work" method to the
>> "create" method, which seems to run fine - but I can't reference any
>> of my workers or get any progress back.
>>
>> Can anyone point to a full tutorial or code example of running an
>> async process and grabbing multiple results WITHOUT memcached? Or is
>> memcached required now?
>>
>> Has the paradigm shifted? Do you only create one worker and call jobs
>> against that worker and grab results using the cache now?
>>
>> Someone please point me in the right direction. My emails to the list
>> and questions on the irc channel have gone unanswered for the last
>> couple of days.
>>
>> --------------------
>> seth - subimage llc
>> -----
>> http://sublog.subimage.com
>> -----
>> Cashboard - Estimates, invoices, and time tracking software - for free!
>> http://www.getcashboard.com
>> -----
>> Substruct - Open source RoR e-commerce software.
>> http://code.google.com/p/substruct/
>>
>>
>>
>> On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at gmail.com> wrote:
>>> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at skarven.net> wrote:
>>>> Hmm... I'm using the git version of backgroundrb, and currently up-to-
>>>> date with the repository; I don't have an "ask_worker" method; I have
>>>> "ask_result." Maybe I should be asking: where can I obtain the version
>>>> of Backgroundrb that's covered in "The Rails Way," because it seems the
>>>> API has changed significantly.
>>>>
>>>> Anyway, I'm creating new workers via something like
>>>>
>>>> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key => 'some key
>>>> ', :data => {:arguments => values})
>>>>
>>>> The worker is coded to such that the process logic is called inside the
>>>> create method, and it posts progress via cache[worker_key]
>>>>
>>>>  def create(args = nil)
>>>>        run_my_process(args)
>>>>  end
>>>>
>>>>  def run_my_process(args)
>>>>    cache[worker_key] = {}
>>>>    ... do stuff
>>>>    cache[worker_key][:progress] = 25
>>>>    ... do more stuff
>>>>    cache[worker_key][:progress] = 50
>>>>    ... etc
>>>>  end
>>>>
>>>> While this runs, I can see the worker chugging away (because it logs as
>>>> it runs,) but if I call
>>>>
>>>> MiddleMan.worker(:foobar_worker).ask_result('some key')
>>>>
>>>> while it is running, I get nil.
>>>>
>>>> Prior to this, I had a prior version of the worker that stayed in
>>>> menory. To use that one, I invoked the process via
>>>>
>>>> MiddleMan.worker(:foobar_worker).async_run_my_process('some key')
>>>>
>>>> The only difference in the coding of the worker for that version was
>>>> that the cache key was job_key rather than worker_key. Calling
>>>> ask_results('some key') produced nil until the task was completed, at
>>>> which point it contained my end-state results.
>>>
>>> We sort of grew very conservative about memory usage in 1.1 release
>>> and decided to handover this task to memcache. Earlier, bdrb master
>>> process used to cache all these results and hence they were readily
>>> available. But unless I maintain an LRU based access, I will have hard
>>> time keeping memory usage of master process under check.
>>>
>>> Hence this responsibility was shelved off to worker. But hey, while a
>>> worker is processing a task, it can't server cache result, and thus
>>> use Memcache, which is out of process. Now even if worker is
>>> processing task, you still get your cache result requests served
>>> alright.
>>>
>>> More details can be found here:
>>>
>>> http://backgroundrb.rubyforge.org/workers/#result_caching
>>>
>>> Lastly, compared to any book, that page is usually more up to date about stuff.
>>> _______________________________________________
>>> Backgroundrb-devel mailing list
>>> Backgroundrb-devel at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>>
>> _______________________________________________
>> Backgroundrb-devel mailing list
>> Backgroundrb-devel at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>


More information about the Backgroundrb-devel mailing list