From gethemant at gmail.com  Mon Nov  3 00:18:10 2008
From: gethemant at gmail.com (hemant)
Date: Mon, 3 Nov 2008 10:48:10 +0530
Subject: [Backgroundrb-devel] Performance problems running Prawn under
	BackgrounDRb
In-Reply-To: <1ca1c1410810282316hf5bff90we2e9511e775377f7@mail.gmail.com>
References: <1ca1c1410810262219h3303f3d6p236613b0d3dc953c@mail.gmail.com>
	<fb283d5e0810272004r28a0b58ey69b941b844a7cbf4@mail.gmail.com>
	<1ca1c1410810281433t6452b3ffqc5e380c463613071@mail.gmail.com>
	<fb283d5e0810282239h6a5fe6cs7ce177dccc8448ee@mail.gmail.com>
	<1ca1c1410810282316hf5bff90we2e9511e775377f7@mail.gmail.com>
Message-ID: <fb283d5e0811022118u15004ad9qa18b2e7795b48154@mail.gmail.com>

On Wed, Oct 29, 2008 at 11:46 AM, Aaron Turner <synfinatic at gmail.com> wrote:
> I have a RoR application which I'm using Prawn via BackgrounDRb to
> generate PDF's.  I've determined there seems to be a 5-6x performance
> hit compared to running Prawn directly as a script from the command
> line (~12sec to 60+sec) on my OS X system.


Okay, I profiled your worker code and results can be found here:

http://backgroundrb.gnufied.org/worker_result.html

What this indicates is, worker is spending quite a bit of time inside,
ActiveSupport::Dependencies#new_constants_in, what this means is,
ActiveSupport really screws constant loading in a big way and slows it
down.

I do not have a solution since this is hardly bdrb problem.

From jnicholson at skarven.net  Thu Nov  6 11:13:02 2008
From: jnicholson at skarven.net (Jim Nicholson)
Date: Thu, 06 Nov 2008 11:13:02 -0500
Subject: [Backgroundrb-devel] How to obtain task progress for long-running
	methods
Message-ID: <1225987982.25767.7.camel@jnick.skarven.net>

We've used previous versions of backgroundrb to handle long-running
tasks, like ingesting large XML uploads into the database. In the
previous API, we could set accessors on workers and use them to provide
status and progress indicators as the main worker method churned away.

With the recent API changes, I can't see how this sort of thing would be
done. It seems that calls to "ask_results" while a method is running
always return nil, rather than whatever the current contents of the
cache are.

Is there some mechanism I'm missing for doing this sort of thing? I can
see the usefulness of the new API and support for clustering, etc, but
if I can't give the user feedback on an active worker's progress, I'm
going to have to go back to the previous incarnation of backgroundrb.




From kieran776 at gmail.com  Thu Nov  6 18:57:39 2008
From: kieran776 at gmail.com (Kieran P)
Date: Fri, 7 Nov 2008 12:57:39 +1300
Subject: [Backgroundrb-devel] How to obtain task progress for
	long-running methods
In-Reply-To: <1225987982.25767.7.camel@jnick.skarven.net>
References: <1225987982.25767.7.camel@jnick.skarven.net>
Message-ID: <fc3b32cf0811061557x202eb952m28b10c7680ce71c5@mail.gmail.com>

Hello Jim,

We had the same problem when we upgraded Backgroundrb. The solution provided
by hemant was to add these line in your backgroundrb.yml config

:backgroundrb:
  :result_storage: memcache

:memcache: "127.0.0.1:11211"


With that in place, ask_worker should start working.

Regards
Kieran


On Fri, Nov 7, 2008 at 5:13 AM, Jim Nicholson <jnicholson at skarven.net>wrote:

> We've used previous versions of backgroundrb to handle long-running
> tasks, like ingesting large XML uploads into the database. In the
> previous API, we could set accessors on workers and use them to provide
> status and progress indicators as the main worker method churned away.
>
> With the recent API changes, I can't see how this sort of thing would be
> done. It seems that calls to "ask_results" while a method is running
> always return nil, rather than whatever the current contents of the
> cache are.
>
> Is there some mechanism I'm missing for doing this sort of thing? I can
> see the usefulness of the new API and support for clustering, etc, but
> if I can't give the user feedback on an active worker's progress, I'm
> going to have to go back to the previous incarnation of backgroundrb.
>
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081107/549b87c7/attachment-0001.html>

From jnicholson at skarven.net  Fri Nov  7 11:04:14 2008
From: jnicholson at skarven.net (Jim Nicholson)
Date: Fri, 07 Nov 2008 11:04:14 -0500
Subject: [Backgroundrb-devel] How to obtain task progress
	for	long-running methods
In-Reply-To: <fc3b32cf0811061557x202eb952m28b10c7680ce71c5@mail.gmail.com>
References: <1225987982.25767.7.camel@jnick.skarven.net>
	<fc3b32cf0811061557x202eb952m28b10c7680ce71c5@mail.gmail.com>
Message-ID: <1226073854.25767.21.camel@jnick.skarven.net>

Hmm... I'm using the git version of backgroundrb, and currently up-to-
date with the repository; I don't have an "ask_worker" method; I have
"ask_result." Maybe I should be asking: where can I obtain the version
of Backgroundrb that's covered in "The Rails Way," because it seems the
API has changed significantly.

Anyway, I'm creating new workers via something like

MiddleMan.new_worker(:worker => :foobar_worker, :worker_key => 'some key
', :data => {:arguments => values})

The worker is coded to such that the process logic is called inside the
create method, and it posts progress via cache[worker_key]

  def create(args = nil)
	run_my_process(args)
  end

  def run_my_process(args)
    cache[worker_key] = {}
    ... do stuff
    cache[worker_key][:progress] = 25
    ... do more stuff
    cache[worker_key][:progress] = 50
    ... etc
  end

While this runs, I can see the worker chugging away (because it logs as
it runs,) but if I call

MiddleMan.worker(:foobar_worker).ask_result('some key') 

while it is running, I get nil.

Prior to this, I had a prior version of the worker that stayed in
menory. To use that one, I invoked the process via

MiddleMan.worker(:foobar_worker).async_run_my_process('some key')

The only difference in the coding of the worker for that version was
that the cache key was job_key rather than worker_key. Calling
ask_results('some key') produced nil until the task was completed, at
which point it contained my end-state results.





On Fri, 2008-11-07 at 12:57 +1300, Kieran P wrote:
> Hello Jim,
> 
> We had the same problem when we upgraded Backgroundrb. The solution
> provided by hemant was to add these line in your backgroundrb.yml
> config
> 
> :backgroundrb:
>   :result_storage: memcache
> 
> :memcache: "127.0.0.1:11211"
> 
> 
> With that in place, ask_worker should start working.
> 
> Regards
> Kieran
> 
> 
> On Fri, Nov 7, 2008 at 5:13 AM, Jim Nicholson <jnicholson at skarven.net>
> wrote:
>         We've used previous versions of backgroundrb to handle long-
>         running
>         tasks, like ingesting large XML uploads into the database. In
>         the
>         previous API, we could set accessors on workers and use them
>         to provide
>         status and progress indicators as the main worker method
>         churned away.
>         
>         With the recent API changes, I can't see how this sort of
>         thing would be
>         done. It seems that calls to "ask_results" while a method is
>         running
>         always return nil, rather than whatever the current contents
>         of the
>         cache are.
>         
>         Is there some mechanism I'm missing for doing this sort of
>         thing? I can
>         see the usefulness of the new API and support for clustering,
>         etc, but
>         if I can't give the user feedback on an active worker's
>         progress, I'm
>         going to have to go back to the previous incarnation of
>         backgroundrb.
>         
>         
>         
>         _______________________________________________
>         Backgroundrb-devel mailing list
>         Backgroundrb-devel at rubyforge.org
>         http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> 


From gethemant at gmail.com  Fri Nov  7 13:04:10 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 7 Nov 2008 23:34:10 +0530
Subject: [Backgroundrb-devel] How to obtain task progress for
	long-running methods
In-Reply-To: <1226073854.25767.21.camel@jnick.skarven.net>
References: <1225987982.25767.7.camel@jnick.skarven.net>
	<fc3b32cf0811061557x202eb952m28b10c7680ce71c5@mail.gmail.com>
	<1226073854.25767.21.camel@jnick.skarven.net>
Message-ID: <fb283d5e0811071004q2232d264ra7d685dc25a6253b@mail.gmail.com>

On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at skarven.net> wrote:
> Hmm... I'm using the git version of backgroundrb, and currently up-to-
> date with the repository; I don't have an "ask_worker" method; I have
> "ask_result." Maybe I should be asking: where can I obtain the version
> of Backgroundrb that's covered in "The Rails Way," because it seems the
> API has changed significantly.
>
> Anyway, I'm creating new workers via something like
>
> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key => 'some key
> ', :data => {:arguments => values})
>
> The worker is coded to such that the process logic is called inside the
> create method, and it posts progress via cache[worker_key]
>
>  def create(args = nil)
>        run_my_process(args)
>  end
>
>  def run_my_process(args)
>    cache[worker_key] = {}
>    ... do stuff
>    cache[worker_key][:progress] = 25
>    ... do more stuff
>    cache[worker_key][:progress] = 50
>    ... etc
>  end
>
> While this runs, I can see the worker chugging away (because it logs as
> it runs,) but if I call
>
> MiddleMan.worker(:foobar_worker).ask_result('some key')
>
> while it is running, I get nil.
>
> Prior to this, I had a prior version of the worker that stayed in
> menory. To use that one, I invoked the process via
>
> MiddleMan.worker(:foobar_worker).async_run_my_process('some key')
>
> The only difference in the coding of the worker for that version was
> that the cache key was job_key rather than worker_key. Calling
> ask_results('some key') produced nil until the task was completed, at
> which point it contained my end-state results.

We sort of grew very conservative about memory usage in 1.1 release
and decided to handover this task to memcache. Earlier, bdrb master
process used to cache all these results and hence they were readily
available. But unless I maintain an LRU based access, I will have hard
time keeping memory usage of master process under check.

Hence this responsibility was shelved off to worker. But hey, while a
worker is processing a task, it can't server cache result, and thus
use Memcache, which is out of process. Now even if worker is
processing task, you still get your cache result requests served
alright.

More details can be found here:

http://backgroundrb.rubyforge.org/workers/#result_caching

Lastly, compared to any book, that page is usually more up to date about stuff.

From shagymoe at yahoo.com  Tue Nov 11 08:56:44 2008
From: shagymoe at yahoo.com (Shaggy Moe)
Date: Tue, 11 Nov 2008 05:56:44 -0800 (PST)
Subject: [Backgroundrb-devel] attachment_fu, aws-s3 and backgroundrb
Message-ID: <360093.77480.qm@web90401.mail.mud.yahoo.com>

Hi,

I'm using attachment_fu to store files on amazon s3.  A background job is currently performing the upload and I'm having a strange issue where the very first file uploaded/created (attachment_fu uploads when creating the object) does not get uploaded.  The db information is all all good. The model and attachment model associated with it are saved to the db, but the file is not uploaded. However, every subsequent file is uploaded successfully with no issues.

It seems that maybe the rails environment isn't being loaded on the first shot, but the load_worker_env.rb file has the line: require RAILS_HOME + "/config/environment"

Any suggestions? I've tried manually adding 'require aws-s3' to the load_worker_env.rb file and it didn't work.  Is it maybe something with attachment_fu not getting loaded or maybe the attachment_fu config file?

Thanks for the help.

Shagy



      

From subimage at gmail.com  Tue Nov 11 20:32:29 2008
From: subimage at gmail.com (seth b)
Date: Tue, 11 Nov 2008 17:32:29 -0800
Subject: [Backgroundrb-devel] Upgrading a crusty version of BDRB.
Message-ID: <7aff9b4c0811111732m5a2e1020ma085a0942777af0e@mail.gmail.com>

Yo gang, I'm running BDRB 0.2.1 on one of my sites. I know, old old
old... I'm going to attempt an upgrade to the latest version & full
test coverage this week.

Can anyone offer blog posts or words of wisdom for this upgrade?

Many thanks.

--------------------
seth - subimage llc
-----
http://sublog.subimage.com
-----
Cashboard - Estimates, invoices, and time tracking software - for free!
http://www.getcashboard.com
-----
Substruct - Open source RoR e-commerce software.
http://code.google.com/p/substruct/

From gethemant at gmail.com  Wed Nov 12 01:59:27 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 12 Nov 2008 12:29:27 +0530
Subject: [Backgroundrb-devel] attachment_fu, aws-s3 and backgroundrb
In-Reply-To: <360093.77480.qm@web90401.mail.mud.yahoo.com>
References: <360093.77480.qm@web90401.mail.mud.yahoo.com>
Message-ID: <fb283d5e0811112259i74ffd756h8f0d81903f960251@mail.gmail.com>

On Tue, Nov 11, 2008 at 7:26 PM, Shaggy Moe <shagymoe at yahoo.com> wrote:
> Hi,
>
> I'm using attachment_fu to store files on amazon s3.  A background job is currently performing the upload and I'm having a strange issue where the very first file uploaded/created (attachment_fu uploads when creating the object) does not get uploaded.  The db information is all all good. The model and attachment model associated with it are saved to the db, but the file is not uploaded. However, every subsequent file is uploaded successfully with no issues.
>
> It seems that maybe the rails environment isn't being loaded on the first shot, but the load_worker_env.rb file has the line: require RAILS_HOME + "/config/environment"
>
> Any suggestions? I've tried manually adding 'require aws-s3' to the load_worker_env.rb file and it didn't work.  Is it maybe something with attachment_fu not getting loaded or maybe the attachment_fu config file?
>
> Thanks for the help.

Is the request coming to bdrb worker? (put some logger statements)
Also, can you paste your worker code and how are you invoking upload
task in worker from rails?

From gethemant at gmail.com  Wed Nov 12 02:02:23 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 12 Nov 2008 12:32:23 +0530
Subject: [Backgroundrb-devel] Upgrading a crusty version of BDRB.
In-Reply-To: <7aff9b4c0811111732m5a2e1020ma085a0942777af0e@mail.gmail.com>
References: <7aff9b4c0811111732m5a2e1020ma085a0942777af0e@mail.gmail.com>
Message-ID: <fb283d5e0811112302ve63354fl8fe8820b0bded2b5@mail.gmail.com>

There have been huge amount of changes and most up2date source of
information is the website itself (http://backgroundrb.rubyforge.org)



On Wed, Nov 12, 2008 at 7:02 AM, seth b <subimage at gmail.com> wrote:
> Yo gang, I'm running BDRB 0.2.1 on one of my sites. I know, old old
> old... I'm going to attempt an upgrade to the latest version & full
> test coverage this week.
>
> Can anyone offer blog posts or words of wisdom for this upgrade?
>
> Many thanks.
>
> --------------------
> seth - subimage llc
> -----
> http://sublog.subimage.com
> -----
> Cashboard - Estimates, invoices, and time tracking software - for free!
> http://www.getcashboard.com
> -----
> Substruct - Open source RoR e-commerce software.
> http://code.google.com/p/substruct/
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From jeremyc at merjis.com  Sun Nov 16 13:51:43 2008
From: jeremyc at merjis.com (Jeremy Chatfield)
Date: Sun, 16 Nov 2008 18:51:43 +0000
Subject: [Backgroundrb-devel] Mac, ActiveScaffold and BackgroundRB
Message-ID: <e6097b140811161051k7f22ae5dp21211a7536acf516@mail.gmail.com>

Hi - backgroundrb is extremely useful, except in one case, Mac OS X with
ActiveScaffold. Even when I compile a test environment from Ruby source, I
still get an error on the Mac, but not on Linux.
I don't know whether this is a backgroundrb problem, or an ActiveScaffold
problem, or indeed a Mac problem... I'm sufficiently new to RoR that I don't
know where to start debugging and disentangling the ownerships of the
problem :)

Here's the test case:

rails awcore
cd awcore
script/generate model Something foo:string bar:integer
git clone git://github.com/gnufied/backgroundrb.gitvendor/plugins/backgroundrb
rake backgroundrb:setup
script/generate worker listsorter
(edit listsorter to include a new methid "refresh" that runs logger.info to
register the invocation)
(edit config/background.yml to include a periodic task)
script/backgroundrb start

This works just fine. Brilliantly so. One more step and it falls apart.

script/backgroundrb stop
git clone git://github.com/activescaffold/active_scaffold.gitvendor/plugins/active_scaffold
&& rm -rf vendor/plugins/active_scaffold/.git
script/backgroundrb start

This will fail with either the Mac OS X Leaopard (10.5.5) Xcode 3.1 included
Ruby 1.8.6 or a fresh compilation of Ruby 1.8.7 (with the various gems). I
continues working on Debian (the only other test environment I tried - I
reckon that if it works there and fails on Mac, the Mac is implicated
somehow).

My gem list is:

actionmailer (2.1.2)
actionpack (2.1.2)
activerecord (2.1.2)
activeresource (2.1.2)
activesupport (2.1.2)
capistrano (2.5.2)
cgi_multipart_eof_fix (2.5.0)
chronic (0.2.3)
daemons (1.0.10)
fastthread (1.0.1)
gem_plugin (0.2.3)
highline (1.5.0)
hoe (1.8.2)
httpclient (2.1.2)
mongrel (1.1.5)
net-scp (1.0.1)
net-sftp (2.0.1)
net-ssh (2.0.4)
net-ssh-gateway (1.0.0)
packet (0.1.14)
rails (2.1.2)
rake (0.8.3)
RedCloth (4.1.0)
rubyforge (1.0.1)
soap4r (1.5.8)
sqlite3-ruby (1.2.4)

The error report is:

 awcore]$ script/backgroundrb start
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/core_ext/module/aliasing.rb:31:in
`alias_method': undefined method `datetime_selector' for class
`ActionView::Helpers::InstanceTag' (NameError)
from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/core_ext/module/aliasing.rb:31:in
`alias_method_chain'
from
/Users/jeremyc/merjis/tools/tawse/awcore/vendor/plugins/active_scaffold/lib/extensions/name_option_for_datetime.rb:20
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
`gem_original_require'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
`require'
from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in
`require'
from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:355:in
`new_constants_in'
from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in
`require'
from
/Users/jeremyc/merjis/tools/tawse/awcore/vendor/plugins/active_scaffold/environment.rb:63
from
/Users/jeremyc/merjis/tools/tawse/awcore/vendor/plugins/active_scaffold/environment.rb:63:in
`each'
from
/Users/jeremyc/merjis/tools/tawse/awcore/vendor/plugins/active_scaffold/environment.rb:63
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
`gem_original_require'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
`require'
from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in
`require'
from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:355:in
`new_constants_in'
from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in
`require'
from
/Users/jeremyc/merjis/tools/tawse/awcore/vendor/plugins/active_scaffold/init.rb:6:in
`evaluate_init_rb'
from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/rails/plugin.rb:95:in
`evaluate_init_rb'
from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/core_ext/kernel/reporting.rb:11:in
`silence_warnings'
from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/rails/plugin.rb:91:in
`evaluate_init_rb'
from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/rails/plugin.rb:44:in
`load'
from
/usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/rails/plugin/loader.rb:33:in
`load_plugins'
from
/usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/rails/plugin/loader.rb:32:in
`each'
from
/usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/rails/plugin/loader.rb:32:in
`load_plugins'
from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/initializer.rb:292:in
`load_plugins'
from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/initializer.rb:142:in
`process'
from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/initializer.rb:97:in
`send'
from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/initializer.rb:97:in
`run'
from /Users/jeremyc/merjis/tools/tawse/awcore/config/environment.rb:13
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
`gem_original_require'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
`require'
from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in
`require'
from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:355:in
`new_constants_in'
from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in
`require'
from script/backgroundrb:25

The worker is:

class ListsorterWorker < BackgrounDRb::MetaWorker
  set_worker_name :listsorter_worker
#  reload_on_schedule true
#  pool_size 1

  def create(args=nil)
    # this method is called, when worker is loaded for the first time
    logger.info 'listsorter initiated'
  end

  def refresh
    logger.info 'listsorterer refresh'
  end
end

The backgroundrb.yml is:

---
:backgroundrb:
  :port: 11006
  :ip: 0.0.0.0

:schedules:
  :listsorter_worker:
    :refresh:
      :trigger_args:
        :start: <%= Time.now + 5.seconds %>
        :repeat_interval: <%= 15.minutes %>
        :end: <%= Time.now + 1.month %>

I'm not sure how to further narrow the problem.

Obviously, removing script/backgroundrb results in the service working (with
the extra steps needed for an ActiveScaffold to work - but I *haven't* done
these, for this test - I was looking for a simple test case).

Suggestions for how to kill this problem, or what to look at next, are
welcome. I'd really like to develop on my Mac and then move it over to the
production servers. Having to develop and test remotely is do-able, but ugly
slow.

Cheers, JeremyC.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081116/2c032fed/attachment-0001.html>

From jeremyc at merjis.com  Sun Nov 16 18:54:21 2008
From: jeremyc at merjis.com (Jeremy Chatfield)
Date: Sun, 16 Nov 2008 23:54:21 +0000
Subject: [Backgroundrb-devel] Mac, ActiveScaffold and BackgroundRB
In-Reply-To: <e6097b140811161051k7f22ae5dp21211a7536acf516@mail.gmail.com>
References: <e6097b140811161051k7f22ae5dp21211a7536acf516@mail.gmail.com>
Message-ID: <e6097b140811161554p7009e2dbh45f892a940b6ac2d@mail.gmail.com>

Following a suggestion by Samer Masry, I checked the Ruby versions. The
Linux system that worked was running Ruby 1.8.5. After updating to 1.8.7, it
fails in the same way as the macs (on 1.8.6 and 1.8.7).
I'm now tolerably convinced that the Ruby version is affecting the
failure... But now I don't know whether to try asking on Ruby lists,
BackgroundRB list, or ActveScaffold list!

Cheers, JeremyC.

On Sun, Nov 16, 2008 at 6:51 PM, Jeremy Chatfield <jeremyc at merjis.com>wrote:

> Hi - backgroundrb is extremely useful, except in one case, Mac OS X with
> ActiveScaffold. Even when I compile a test environment from Ruby source, I
> still get an error on the Mac, but not on Linux.
> I don't know whether this is a backgroundrb problem, or an ActiveScaffold
> problem, or indeed a Mac problem... I'm sufficiently new to RoR that I don't
> know where to start debugging and disentangling the ownerships of the
> problem :)
>
> Here's the test case:
>
> rails awcore
> cd awcore
> script/generate model Something foo:string bar:integer
> git clone git://github.com/gnufied/backgroundrb.gitvendor/plugins/backgroundrb
> rake backgroundrb:setup
> script/generate worker listsorter
> (edit listsorter to include a new methid "refresh" that runs logger.infoto register the invocation)
> (edit config/background.yml to include a periodic task)
> script/backgroundrb start
>
> This works just fine. Brilliantly so. One more step and it falls apart.
>
> script/backgroundrb stop
> git clone git://github.com/activescaffold/active_scaffold.gitvendor/plugins/active_scaffold && rm -rf vendor/plugins/active_scaffold/.git
> script/backgroundrb start
>
> This will fail with either the Mac OS X Leaopard (10.5.5) Xcode 3.1
> included Ruby 1.8.6 or a fresh compilation of Ruby 1.8.7 (with the various
> gems). I continues working on Debian (the only other test environment I
> tried - I reckon that if it works there and fails on Mac, the Mac is
> implicated somehow).
>
> My gem list is:
>
> actionmailer (2.1.2)
> actionpack (2.1.2)
> activerecord (2.1.2)
> activeresource (2.1.2)
> activesupport (2.1.2)
> capistrano (2.5.2)
> cgi_multipart_eof_fix (2.5.0)
> chronic (0.2.3)
> daemons (1.0.10)
> fastthread (1.0.1)
> gem_plugin (0.2.3)
> highline (1.5.0)
> hoe (1.8.2)
> httpclient (2.1.2)
> mongrel (1.1.5)
> net-scp (1.0.1)
> net-sftp (2.0.1)
> net-ssh (2.0.4)
> net-ssh-gateway (1.0.0)
> packet (0.1.14)
> rails (2.1.2)
> rake (0.8.3)
> RedCloth (4.1.0)
> rubyforge (1.0.1)
> soap4r (1.5.8)
> sqlite3-ruby (1.2.4)
>
> The error report is:
>
>  awcore]$ script/backgroundrb start
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/core_ext/module/aliasing.rb:31:in
> `alias_method': undefined method `datetime_selector' for class
> `ActionView::Helpers::InstanceTag' (NameError)
>  from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/core_ext/module/aliasing.rb:31:in
> `alias_method_chain'
> from
> /Users/jeremyc/merjis/tools/tawse/awcore/vendor/plugins/active_scaffold/lib/extensions/name_option_for_datetime.rb:20
>  from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
> `gem_original_require'
> from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
> `require'
>  from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in
> `require'
> from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:355:in
> `new_constants_in'
>  from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in
> `require'
> from
> /Users/jeremyc/merjis/tools/tawse/awcore/vendor/plugins/active_scaffold/environment.rb:63
>  from
> /Users/jeremyc/merjis/tools/tawse/awcore/vendor/plugins/active_scaffold/environment.rb:63:in
> `each'
> from
> /Users/jeremyc/merjis/tools/tawse/awcore/vendor/plugins/active_scaffold/environment.rb:63
>  from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
> `gem_original_require'
> from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
> `require'
>  from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in
> `require'
> from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:355:in
> `new_constants_in'
>  from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in
> `require'
> from
> /Users/jeremyc/merjis/tools/tawse/awcore/vendor/plugins/active_scaffold/init.rb:6:in
> `evaluate_init_rb'
>  from
> /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/rails/plugin.rb:95:in
> `evaluate_init_rb'
> from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/core_ext/kernel/reporting.rb:11:in
> `silence_warnings'
>  from
> /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/rails/plugin.rb:91:in
> `evaluate_init_rb'
> from
> /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/rails/plugin.rb:44:in
> `load'
>  from
> /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/rails/plugin/loader.rb:33:in
> `load_plugins'
> from
> /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/rails/plugin/loader.rb:32:in
> `each'
>  from
> /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/rails/plugin/loader.rb:32:in
> `load_plugins'
> from
> /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/initializer.rb:292:in
> `load_plugins'
>  from
> /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/initializer.rb:142:in
> `process'
> from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/initializer.rb:97:in
> `send'
>  from
> /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.2/lib/initializer.rb:97:in `run'
> from /Users/jeremyc/merjis/tools/tawse/awcore/config/environment.rb:13
>  from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
> `gem_original_require'
> from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
> `require'
>  from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in
> `require'
> from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:355:in
> `new_constants_in'
>  from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in
> `require'
> from script/backgroundrb:25
>
> The worker is:
>
> class ListsorterWorker < BackgrounDRb::MetaWorker
>   set_worker_name :listsorter_worker
> #  reload_on_schedule true
> #  pool_size 1
>
>   def create(args=nil)
>     # this method is called, when worker is loaded for the first time
>     logger.info 'listsorter initiated'
>   end
>
>   def refresh
>     logger.info 'listsorterer refresh'
>   end
> end
>
> The backgroundrb.yml is:
>
> ---
> :backgroundrb:
>   :port: 11006
>   :ip: 0.0.0.0
>
> :schedules:
>   :listsorter_worker:
>     :refresh:
>       :trigger_args:
>         :start: <%= Time.now + 5.seconds %>
>         :repeat_interval: <%= 15.minutes %>
>         :end: <%= Time.now + 1.month %>
>
> I'm not sure how to further narrow the problem.
>
> Obviously, removing script/backgroundrb results in the service working
> (with the extra steps needed for an ActiveScaffold to work - but I *haven't*
> done these, for this test - I was looking for a simple test case).
>
> Suggestions for how to kill this problem, or what to look at next, are
> welcome. I'd really like to develop on my Mac and then move it over to the
> production servers. Having to develop and test remotely is do-able, but ugly
> slow.
>
> Cheers, JeremyC.
>
>


-- 
Jeremy Chatfield - Merjis: internet marketing strategy, audits and execution
Skype: jezchatfield, Mobile: +44(0)7780 607613, SkypeIn +44 203 290 9977
Merjis Ltd is registered in England with the number 4727672, VAT number
855472011
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081116/a2111cc4/attachment.html>

From subimage at gmail.com  Mon Nov 17 22:07:43 2008
From: subimage at gmail.com (seth b)
Date: Mon, 17 Nov 2008 19:07:43 -0800
Subject: [Backgroundrb-devel] Upgrading & testing
Message-ID: <7aff9b4c0811171907o305b65e7q334053b1673f0975@mail.gmail.com>

Hello again BDRB folks.

I'm attempting to push forward with my upgrade from 0.2.1 of DRB. At
the moment I'm trying to run with TDD and test my workers properly
before putting them into action.

However, I'm running into some questions while doing so.

First, I ran into the issue that my DRB server was not running prior
to my tests. I started this up manually, but for times when I run
autotest I'd like to automate this process (startup and shutdown). Is
anyone doing such a thing and could provide some insight into a slick
way to accomplish it?

Thanks in advance.

--------------------
seth - subimage llc
-----
http://sublog.subimage.com
-----
Cashboard - Estimates, invoices, and time tracking software - for free!
http://www.getcashboard.com
-----
Substruct - Open source RoR e-commerce software.
http://code.google.com/p/substruct/

From subimage at gmail.com  Mon Nov 17 22:12:54 2008
From: subimage at gmail.com (seth b)
Date: Mon, 17 Nov 2008 19:12:54 -0800
Subject: [Backgroundrb-devel] Upgrading & testing
In-Reply-To: <7aff9b4c0811171907o305b65e7q334053b1673f0975@mail.gmail.com>
References: <7aff9b4c0811171907o305b65e7q334053b1673f0975@mail.gmail.com>
Message-ID: <7aff9b4c0811171912i497bb2f1xebf4bbd6d760d62f@mail.gmail.com>

PS: Here's what I'm doing right now - but it's not working. I the test
runs, but gets stuck in an infinite loop selecting from
bdrb_job_queues. Any help would be wonderful. I'm pretty stuck.

# TEST CASE ==========

require File.dirname(__FILE__) + '/../test_helper'
require File.join(File.dirname(__FILE__) + "/../bdrb_test_helper")
require "cancel_account_worker"

class CancelAccountWorkerTest < Test::Unit::TestCase
  fixtures(:all)

  def setup
    `ruby #{File.join(RAILS_ROOT, 'script/backgroundrb')} start -e test`
  end

  def teardown
    `ruby #{File.join(RAILS_ROOT, 'script/backgroundrb')} stop`
  end

  def test_cancel_account
    acct = accounts(:paid)
    worker = MiddleMan.worker(:cancel_account_worker).do_work(:account_id
=> acct.id)
    assert_nil Account.find_by_id(acct.id)
  end
end

# INFINITE SQL LOOP FROM LOGS ==========

  SQL (0.000290)   BEGIN
  BdrbJobQueue Load (0.000417)   SELECT * FROM `bdrb_job_queues` WHERE
( worker_name = 'send_document_worker' AND taken = 0 AND scheduled_at
<= '2008-11-18 03:12:26' ) LIMIT 1 FOR UPDATE
  SQL (0.000098)   COMMIT
  SQL (0.000392)   BEGIN
  BdrbJobQueue Load (0.000743)   SELECT * FROM `bdrb_job_queues` WHERE
( worker_name = 'exporter_worker' AND taken = 0 AND scheduled_at <=
'2008-11-18 03:12:30' ) LIMIT 1 FOR UPDATE
  SQL (0.000129)   COMMIT
  SQL (0.000502)   BEGIN
  BdrbJobQueue Load (0.000770)   SELECT * FROM `bdrb_job_queues` WHERE
( worker_name = 'basecamp_sync_worker' AND taken = 0 AND scheduled_at
<= '2008-11-18 03:12:31' ) LIMIT 1 FOR UPDATE
  SQL (0.000195)   COMMIT

.....etc, etc etc.

--------------------
seth - subimage llc
-----
http://sublog.subimage.com
-----
Cashboard - Estimates, invoices, and time tracking software - for free!
http://www.getcashboard.com
-----
Substruct - Open source RoR e-commerce software.
http://code.google.com/p/substruct/



On Mon, Nov 17, 2008 at 7:07 PM, seth b <subimage at gmail.com> wrote:
> Hello again BDRB folks.
>
> I'm attempting to push forward with my upgrade from 0.2.1 of DRB. At
> the moment I'm trying to run with TDD and test my workers properly
> before putting them into action.
>
> However, I'm running into some questions while doing so.
>
> First, I ran into the issue that my DRB server was not running prior
> to my tests. I started this up manually, but for times when I run
> autotest I'd like to automate this process (startup and shutdown). Is
> anyone doing such a thing and could provide some insight into a slick
> way to accomplish it?
>
> Thanks in advance.
>
> --------------------
> seth - subimage llc
> -----
> http://sublog.subimage.com
> -----
> Cashboard - Estimates, invoices, and time tracking software - for free!
> http://www.getcashboard.com
> -----
> Substruct - Open source RoR e-commerce software.
> http://code.google.com/p/substruct/
>

From ramon.tayag at gmail.com  Tue Nov 18 03:13:43 2008
From: ramon.tayag at gmail.com (Ramon Tayag)
Date: Tue, 18 Nov 2008 16:13:43 +0800
Subject: [Backgroundrb-devel] Invalid worker with name [worker name] and key
Message-ID: <f25d5ad20811180013p78a634f3i55ef6fa28d72438e@mail.gmail.com>

Hey everyone.  I started to wonder why no mail was being sent from my
app and I checked the log and saw this:

I haven't upgraded my backgroundrb so I can't/shouldn't be a version
error.  I decided to restart my cluster (mongrels and backgroundrb
server) and tried to send again.  It worked.  What causes this
problem?  And what can I do so that I'll be notified when this problem
arises?

Invalid worker with name mail_worker and key
/usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_connection.rb:52:in
`ask_worker'
/home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_worker.rb:103:in
`async_method_invoke'
/home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_worker.rb:37:in
`receive_data'
/usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_parser.rb:44:in
`extract'
/usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_parser.rb:26:in
`loop'
/usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_parser.rb:26:in
`extract'
/home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_worker.rb:32:in
`receive_data'
/usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:232:in
`read_external_socket'
/usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:222:in
`handle_external_messages'
/usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:196:in
`handle_read_event'
/usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:192:in
`each'
/usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:192:in
`handle_read_event'
/usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:146:in
`start_reactor'
/usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:139:in
`loop'
/usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:139:in
`start_reactor'
/usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_master.rb:21:in
`run'
/home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_proxy.rb:14:in
`initialize'
/home/ramon/my-app/current/script/backgroundrb:47:in `new'
/home/ramon/my-app/current/script/backgroundrb:47
Client disconected

Thanks,
Ramon Tayag

From jnutting at gmail.com  Tue Nov 18 03:48:24 2008
From: jnutting at gmail.com (Jack Nutting)
Date: Tue, 18 Nov 2008 09:48:24 +0100
Subject: [Backgroundrb-devel] Invalid worker with name [worker name] and
	key
In-Reply-To: <f25d5ad20811180013p78a634f3i55ef6fa28d72438e@mail.gmail.com>
References: <f25d5ad20811180013p78a634f3i55ef6fa28d72438e@mail.gmail.com>
Message-ID: <f84841020811180048j9218002k140983a70d2a9fa4@mail.gmail.com>

On Tue, Nov 18, 2008 at 9:13 AM, Ramon Tayag <ramon.tayag at gmail.com> wrote:

> I decided to restart my cluster (mongrels and backgroundrb
> server) and tried to send again.  It worked.  What causes this
> problem?  And what can I do so that I'll be notified when this problem
> arises?
>

I have a similar problem occurring occasionally under heavy load, which
leads to the whole backgroundrb system crashing.  I've configured monit to
restart backgroundrb if it goes down, which is "good enough" for me at the
moment (but still somewhat unsatisfying since it is more of a band-aid than
a cure).

-- 
// jack
// http://www.nuthole.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081118/e49879e0/attachment.html>

From ramon.tayag at gmail.com  Tue Nov 18 04:17:00 2008
From: ramon.tayag at gmail.com (Ramon Tayag)
Date: Tue, 18 Nov 2008 17:17:00 +0800
Subject: [Backgroundrb-devel] Invalid worker with name [worker name] and
	key
In-Reply-To: <f25d5ad20811180058s58eb8e37pb0b2e1fe777ed9fa@mail.gmail.com>
References: <f25d5ad20811180013p78a634f3i55ef6fa28d72438e@mail.gmail.com>
	<f84841020811180048j9218002k140983a70d2a9fa4@mail.gmail.com>
	<f25d5ad20811180049j61b8c75ft648d07be83712e32@mail.gmail.com>
	<f84841020811180055g52120525ya6add579ec911fc7@mail.gmail.com>
	<f25d5ad20811180058s58eb8e37pb0b2e1fe777ed9fa@mail.gmail.com>
Message-ID: <f25d5ad20811180117t1f4395b1m1402f21dcdcbc982@mail.gmail.com>

Did more surfing and tried Middleman.all_worker_info on my server

{"0.0.0.0:11006"=>[{:worker_key=>"", :status=>:running,
:worker=>:mail_worker}, {:worker_key=>"", :status=>:running,
:worker=>:log_worker}]}

Should the worker_key have a value?

Ramon Tayag

From enzodm at gmail.com  Tue Nov 18 12:54:01 2008
From: enzodm at gmail.com (Samer Masry)
Date: Tue, 18 Nov 2008 09:54:01 -0800
Subject: [Backgroundrb-devel] Invalid worker with name [worker name] and
	key
In-Reply-To: <f25d5ad20811180013p78a634f3i55ef6fa28d72438e@mail.gmail.com>
References: <f25d5ad20811180013p78a634f3i55ef6fa28d72438e@mail.gmail.com>
Message-ID: <c9517bd80811180954t675efc32ne9e8cb0d166c67e0@mail.gmail.com>

By default when you start the server it finds all of the workers and inserts
one in.

Make sure when you create a new worker with a key you are using the
new_worker method
MiddleMan.new_worker(:worker => #{worker_name}, :worker_key =>
#{worker_key})

You'll then be able to access the worker by the key.

PS.  in the console use MiddleMan.all_worker_info to get a list of all the
worker objects.  Once you use the new_worker method you will see the newly
created worker with the key specified.


On Tue, Nov 18, 2008 at 12:13 AM, Ramon Tayag <ramon.tayag at gmail.com> wrote:

> Hey everyone.  I started to wonder why no mail was being sent from my
> app and I checked the log and saw this:
>
> I haven't upgraded my backgroundrb so I can't/shouldn't be a version
> error.  I decided to restart my cluster (mongrels and backgroundrb
> server) and tried to send again.  It worked.  What causes this
> problem?  And what can I do so that I'll be notified when this problem
> arises?
>
> Invalid worker with name mail_worker and key
>
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_connection.rb:52:in
> `ask_worker'
>
> /home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_worker.rb:103:in
> `async_method_invoke'
>
> /home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_worker.rb:37:in
> `receive_data'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_parser.rb:44:in
> `extract'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_parser.rb:26:in
> `loop'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_parser.rb:26:in
> `extract'
>
> /home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_worker.rb:32:in
> `receive_data'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:232:in
> `read_external_socket'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:222:in
> `handle_external_messages'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:196:in
> `handle_read_event'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:192:in
> `each'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:192:in
> `handle_read_event'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:146:in
> `start_reactor'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:139:in
> `loop'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:139:in
> `start_reactor'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_master.rb:21:in
> `run'
>
> /home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_proxy.rb:14:in
> `initialize'
> /home/ramon/my-app/current/script/backgroundrb:47:in `new'
> /home/ramon/my-app/current/script/backgroundrb:47
> Client disconected
>
> Thanks,
> Ramon Tayag
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081118/537df5bb/attachment.html>

From ramon.tayag at gmail.com  Tue Nov 18 13:09:09 2008
From: ramon.tayag at gmail.com (Ramon Tayag)
Date: Wed, 19 Nov 2008 02:09:09 +0800
Subject: [Backgroundrb-devel] Invalid worker with name [worker name] and
	key
In-Reply-To: <c9517bd80811180954t675efc32ne9e8cb0d166c67e0@mail.gmail.com>
References: <f25d5ad20811180013p78a634f3i55ef6fa28d72438e@mail.gmail.com>
	<c9517bd80811180954t675efc32ne9e8cb0d166c67e0@mail.gmail.com>
Message-ID: <f25d5ad20811181009q2c2b73fahe87aba2227f62efb@mail.gmail.com>

Thank you.

Sorry need a bit more help.

I see what you typed in "Start a new worker from controller".  Will it
work with the model?  Also, is it because I didn't call this that this
error is happening?

Thanks,
Ramon Tayag



On Wed, Nov 19, 2008 at 1:54 AM, Samer Masry <enzodm at gmail.com> wrote:
> By default when you start the server it finds all of the workers and inserts
> one in.
>
> Make sure when you create a new worker with a key you are using the
> new_worker method
> MiddleMan.new_worker(:worker => #{worker_name}, :worker_key =>
> #{worker_key})
>
> You'll then be able to access the worker by the key.
>
> PS.  in the console use MiddleMan.all_worker_info to get a list of all the
> worker objects.  Once you use the new_worker method you will see the newly
> created worker with the key specified.
>
>
> On Tue, Nov 18, 2008 at 12:13 AM, Ramon Tayag <ramon.tayag at gmail.com> wrote:
>>
>> Hey everyone.  I started to wonder why no mail was being sent from my
>> app and I checked the log and saw this:
>>
>> I haven't upgraded my backgroundrb so I can't/shouldn't be a version
>> error.  I decided to restart my cluster (mongrels and backgroundrb
>> server) and tried to send again.  It worked.  What causes this
>> problem?  And what can I do so that I'll be notified when this problem
>> arises?
>>
>> Invalid worker with name mail_worker and key
>>
>> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_connection.rb:52:in
>> `ask_worker'
>>
>> /home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_worker.rb:103:in
>> `async_method_invoke'
>>
>> /home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_worker.rb:37:in
>> `receive_data'
>>
>> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_parser.rb:44:in
>> `extract'
>>
>> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_parser.rb:26:in
>> `loop'
>>
>> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_parser.rb:26:in
>> `extract'
>>
>> /home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_worker.rb:32:in
>> `receive_data'
>> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:232:in
>> `read_external_socket'
>> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:222:in
>> `handle_external_messages'
>> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:196:in
>> `handle_read_event'
>> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:192:in
>> `each'
>> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:192:in
>> `handle_read_event'
>> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:146:in
>> `start_reactor'
>> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:139:in
>> `loop'
>> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:139:in
>> `start_reactor'
>>
>> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_master.rb:21:in
>> `run'
>>
>> /home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_proxy.rb:14:in
>> `initialize'
>> /home/ramon/my-app/current/script/backgroundrb:47:in `new'
>> /home/ramon/my-app/current/script/backgroundrb:47
>> Client disconected
>>
>> Thanks,
>> Ramon Tayag
>> _______________________________________________
>> Backgroundrb-devel mailing list
>> Backgroundrb-devel at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
>

From raghu.srinivasan at gmail.com  Wed Nov 19 00:35:30 2008
From: raghu.srinivasan at gmail.com (Raghu Srinivasan)
Date: Tue, 18 Nov 2008 21:35:30 -0800
Subject: [Backgroundrb-devel] Error starting up Backgroundrb
Message-ID: <6482c06a0811182135v539c2748uc740bee382e5c9ff@mail.gmail.com>

Hello everyone, I am hoping to get some help with my upgrade of BDRB.

As per one of Hemant's earlier emails, I started following along as per the
instructions at http://backgroundrb.rubyforge.org

1) git clone for bdrb (after I removed the older backgroundrb directory from
/vendor/plugins)
2) upgraded packet to 0.1.7 from 0.1.5
3) removed backgroundrb and load_worker_env.rb from the script directory
4) did a rake backgroundrb:setup
5) did a rake db:migrate and ensured that the table got created in the db
6) copied over my original backgroundrb.yml

So far so good. Now when I try to start bdrb, I get this error:
======
ffapi ~/public_html/feedflix.com/ff: script/backgroundrb start
/usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:263:in
`load_missing_constant': uninitialized constant BackgrounDRb::StartStop
(NameError)
        from
/usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:453:in
`const_missing'
        from script/backgroundrb:32
ffapi ~/public_html/feedflix.com/ff:
=======
I feel like I am missing something obvious here. Can someone help?

Thanks!
Raghu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081118/752e1048/attachment-0001.html>

From ramon.tayag at gmail.com  Wed Nov 19 01:49:17 2008
From: ramon.tayag at gmail.com (Ramon Tayag)
Date: Wed, 19 Nov 2008 14:49:17 +0800
Subject: [Backgroundrb-devel] Invalid worker with name [worker name] and
	key
In-Reply-To: <c9517bd80811181219t38a6dd69u6fb247e41da36551@mail.gmail.com>
References: <f25d5ad20811180013p78a634f3i55ef6fa28d72438e@mail.gmail.com>
	<c9517bd80811180954t675efc32ne9e8cb0d166c67e0@mail.gmail.com>
	<f25d5ad20811181009q2c2b73fahe87aba2227f62efb@mail.gmail.com>
	<c9517bd80811181011n490e8749v8261755a5091dc5@mail.gmail.com>
	<f25d5ad20811181131u189e665dk9d0e11d40c9fbf26@mail.gmail.com>
	<c9517bd80811181219t38a6dd69u6fb247e41da36551@mail.gmail.com>
Message-ID: <f25d5ad20811182249w647d23a4p38c74493798473aa@mail.gmail.com>

Thanks for this.. but is this what's causes the error of tasks not
running?  I'd rather not implement code if it isn't going to fix the
problem :)

Because on the backgroundrb site it says "You must use worker_key if
worker was started with a worker_key" which tells me that this worker
key is optional.  And it seems so because most of my backgroundrb
tasks run.

Ramon Tayag



On Wed, Nov 19, 2008 at 4:19 AM, Samer Masry <enzodm at gmail.com> wrote:
> Here's an example
> If you are using a worker with a key then you need to use something like
>
> worker = MiddleMan.new_worker( :worker => :mail_worker, :worker_key =>
> 'test')
> now if you do a MiddleMan.all_worker_info you will then see your worker with
> the key
>
>
> To access your worker simply do MiddleMan.worker( :mail_worker, 'test' )
>
>
> On Tue, Nov 18, 2008 at 11:31 AM, Ramon Tayag <ramon.tayag at gmail.com> wrote:
>>
>> Actually, I'm not doing any of that... I just followed the code below
>> here: http://backgroundrb.rubyforge.org/
>>
>> It worked, at least for the most part.
>>
>> Ramon Tayag
>>
>> On Wed, Nov 19, 2008 at 2:11 AM, Samer Masry <enzodm at gmail.com> wrote:
>> > How are you creating the worker?
>
>

From gethemant at gmail.com  Wed Nov 19 01:50:40 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 19 Nov 2008 12:20:40 +0530
Subject: [Backgroundrb-devel] Error starting up Backgroundrb
In-Reply-To: <6482c06a0811182135v539c2748uc740bee382e5c9ff@mail.gmail.com>
References: <6482c06a0811182135v539c2748uc740bee382e5c9ff@mail.gmail.com>
Message-ID: <fb283d5e0811182250i254348f0sfe7eaae25819b5d7@mail.gmail.com>

1. You need 0.1.14 packet version
2. which rails version you are using? Are you having config.plugins in
environment.rb?
3. Do you have vendor/plugins/backgroundrb/lib/backgroundrb/bdrb_start_stop.rb
file?



On Wed, Nov 19, 2008 at 11:05 AM, Raghu Srinivasan
<raghu.srinivasan at gmail.com> wrote:
> Hello everyone, I am hoping to get some help with my upgrade of BDRB.
>
> As per one of Hemant's earlier emails, I started following along as per the
> instructions at http://backgroundrb.rubyforge.org
>
> 1) git clone for bdrb (after I removed the older backgroundrb directory from
> /vendor/plugins)
> 2) upgraded packet to 0.1.7 from 0.1.5
> 3) removed backgroundrb and load_worker_env.rb from the script directory
> 4) did a rake backgroundrb:setup
> 5) did a rake db:migrate and ensured that the table got created in the db
> 6) copied over my original backgroundrb.yml
>
> So far so good. Now when I try to start bdrb, I get this error:
> ======
> ffapi ~/public_html/feedflix.com/ff: script/backgroundrb start
> /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:263:in
> `load_missing_constant': uninitialized constant BackgrounDRb::StartStop
> (NameError)
>         from
> /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:453:in
> `const_missing'
>         from script/backgroundrb:32
> ffapi ~/public_html/feedflix.com/ff:
> =======
> I feel like I am missing something obvious here. Can someone help?
>
> Thanks!
> Raghu
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From raghu.srinivasan at gmail.com  Wed Nov 19 01:59:18 2008
From: raghu.srinivasan at gmail.com (Raghu Srinivasan)
Date: Tue, 18 Nov 2008 22:59:18 -0800
Subject: [Backgroundrb-devel] Error starting up Backgroundrb
In-Reply-To: <fb283d5e0811182250i254348f0sfe7eaae25819b5d7@mail.gmail.com>
References: <6482c06a0811182135v539c2748uc740bee382e5c9ff@mail.gmail.com>
	<fb283d5e0811182250i254348f0sfe7eaae25819b5d7@mail.gmail.com>
Message-ID: <6482c06a0811182259g2effd93ci569ad821821b0df@mail.gmail.com>

Hi Hemant,

1. I installed this version (0.1.14) of packet and tried - I still get the
same error
2. My Rails version is 2.0.2. No, config.plans is commented out in my env.rb
(which means all plugins including BDRB must be getting loaded)
3. Yes, I have this file

I am on Ubuntu (Hardy) and the earlier BDRB version I've used all along runs
with no issues.

Thanks,

Raghu

On Tue, Nov 18, 2008 at 10:50 PM, hemant <gethemant at gmail.com> wrote:

> 1. You need 0.1.14 packet version
> 2. which rails version you are using? Are you having config.plugins in
> environment.rb?
> 3. Do you have
> vendor/plugins/backgroundrb/lib/backgroundrb/bdrb_start_stop.rb
> file?
>
>
>
> On Wed, Nov 19, 2008 at 11:05 AM, Raghu Srinivasan
> <raghu.srinivasan at gmail.com> wrote:
> > Hello everyone, I am hoping to get some help with my upgrade of BDRB.
> >
> > As per one of Hemant's earlier emails, I started following along as per
> the
> > instructions at http://backgroundrb.rubyforge.org
> >
> > 1) git clone for bdrb (after I removed the older backgroundrb directory
> from
> > /vendor/plugins)
> > 2) upgraded packet to 0.1.7 from 0.1.5
> > 3) removed backgroundrb and load_worker_env.rb from the script directory
> > 4) did a rake backgroundrb:setup
> > 5) did a rake db:migrate and ensured that the table got created in the db
> > 6) copied over my original backgroundrb.yml
> >
> > So far so good. Now when I try to start bdrb, I get this error:
> > ======
> > ffapi ~/public_html/feedflix.com/ff: script/backgroundrb start
> >
> /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:263:in
> > `load_missing_constant': uninitialized constant BackgrounDRb::StartStop
> > (NameError)
> >         from
> >
> /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:453:in
> > `const_missing'
> >         from script/backgroundrb:32
> > ffapi ~/public_html/feedflix.com/ff:
> > =======
> > I feel like I am missing something obvious here. Can someone help?
> >
> > Thanks!
> > Raghu
> >
> >
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081118/743834d0/attachment.html>

From gethemant at gmail.com  Wed Nov 19 02:03:55 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 19 Nov 2008 12:33:55 +0530
Subject: [Backgroundrb-devel] Error starting up Backgroundrb
In-Reply-To: <6482c06a0811182259g2effd93ci569ad821821b0df@mail.gmail.com>
References: <6482c06a0811182135v539c2748uc740bee382e5c9ff@mail.gmail.com>
	<fb283d5e0811182250i254348f0sfe7eaae25819b5d7@mail.gmail.com>
	<6482c06a0811182259g2effd93ci569ad821821b0df@mail.gmail.com>
Message-ID: <fb283d5e0811182303h161bd849wf978f6c7f6d2340f@mail.gmail.com>

I am puzzled. But anyways, just then open backgroundrb script from
script directory and add require "bdrb_start_stop" after require
"backgroundrb_server" and try.



On Wed, Nov 19, 2008 at 12:29 PM, Raghu Srinivasan
<raghu.srinivasan at gmail.com> wrote:
> Hi Hemant,
>
> 1. I installed this version (0.1.14) of packet and tried - I still get the
> same error
> 2. My Rails version is 2.0.2. No, config.plans is commented out in my env.rb
> (which means all plugins including BDRB must be getting loaded)
> 3. Yes, I have this file
>
> I am on Ubuntu (Hardy) and the earlier BDRB version I've used all along runs
> with no issues.
>
> Thanks,
>
> Raghu
>
> On Tue, Nov 18, 2008 at 10:50 PM, hemant <gethemant at gmail.com> wrote:
>>
>> 1. You need 0.1.14 packet version
>> 2. which rails version you are using? Are you having config.plugins in
>> environment.rb?
>> 3. Do you have
>> vendor/plugins/backgroundrb/lib/backgroundrb/bdrb_start_stop.rb
>> file?
>>
>>
>>
>> On Wed, Nov 19, 2008 at 11:05 AM, Raghu Srinivasan
>> <raghu.srinivasan at gmail.com> wrote:
>> > Hello everyone, I am hoping to get some help with my upgrade of BDRB.
>> >
>> > As per one of Hemant's earlier emails, I started following along as per
>> > the
>> > instructions at http://backgroundrb.rubyforge.org
>> >
>> > 1) git clone for bdrb (after I removed the older backgroundrb directory
>> > from
>> > /vendor/plugins)
>> > 2) upgraded packet to 0.1.7 from 0.1.5
>> > 3) removed backgroundrb and load_worker_env.rb from the script directory
>> > 4) did a rake backgroundrb:setup
>> > 5) did a rake db:migrate and ensured that the table got created in the
>> > db
>> > 6) copied over my original backgroundrb.yml
>> >
>> > So far so good. Now when I try to start bdrb, I get this error:
>> > ======
>> > ffapi ~/public_html/feedflix.com/ff: script/backgroundrb start
>> >
>> > /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:263:in
>> > `load_missing_constant': uninitialized constant BackgrounDRb::StartStop
>> > (NameError)
>> >         from
>> >
>> > /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:453:in
>> > `const_missing'
>> >         from script/backgroundrb:32
>> > ffapi ~/public_html/feedflix.com/ff:
>> > =======
>> > I feel like I am missing something obvious here. Can someone help?
>> >
>> > Thanks!
>> > Raghu
>> >
>> >
>> > _______________________________________________
>> > Backgroundrb-devel mailing list
>> > Backgroundrb-devel at rubyforge.org
>> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>> >
>>
>>
>>
>> --
>> Let them talk of their oriental summer climes of everlasting
>> conservatories; give me the privilege of making my own summer with my
>> own coals.
>>
>> http://gnufied.org
>
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From raghu.srinivasan at gmail.com  Wed Nov 19 02:21:54 2008
From: raghu.srinivasan at gmail.com (Raghu Srinivasan)
Date: Tue, 18 Nov 2008 23:21:54 -0800
Subject: [Backgroundrb-devel] Error starting up Backgroundrb
In-Reply-To: <fb283d5e0811182303h161bd849wf978f6c7f6d2340f@mail.gmail.com>
References: <6482c06a0811182135v539c2748uc740bee382e5c9ff@mail.gmail.com>
	<fb283d5e0811182250i254348f0sfe7eaae25819b5d7@mail.gmail.com>
	<6482c06a0811182259g2effd93ci569ad821821b0df@mail.gmail.com>
	<fb283d5e0811182303h161bd849wf978f6c7f6d2340f@mail.gmail.com>
Message-ID: <6482c06a0811182321s188229c6jae6efb662de9ae95@mail.gmail.com>

I added that require and now the 'start' command seems to work fine but the
log has an error in it for "uninitialized constant
BackgrounDRb::MetaWorker::BdrbJobQueue (NameError)". Here it is:
============
ffapi ~/public_html/feedflix.com/ff: ./script/backgroundrb start
Starting BackgrounDRb ....
ffapi ~/public_html/feedflix.com/ff: ls -lrt log/ba*
-rw-r--r-- 1 raghus raghus   57 Nov 19 07:19 log/backgroundrb_11007.log
-rw-r--r-- 1 raghus raghus 1366 Nov 19 07:19
log/backgroundrb_debug_11007.log
ffapi ~/public_html/feedflix.com/ff: cat log/backgroundrb_debug_11007.log
/usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:478:in
`const_missing': uninitialized constant
BackgrounDRb::MetaWorker::BdrbJobQueue (NameError)
        from /home/raghus/public_html/
feedflix.com/ff/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:271:in`check_for_enqueued_tasks'
        from /home/raghus/public_html/
feedflix.com/ff/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:133:in`worker_init'
        from
/usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_periodic_event.rb:23:in
`call'
        from
/usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_periodic_event.rb:23:in
`run'
        from
/usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_core.rb:301:in
`check_for_timer_events'
        from
/usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_core.rb:301:in
`each'
        from
/usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_core.rb:301:in
`check_for_timer_events'
        from /home/raghus/public_html/
feedflix.com/ff/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:296:in`check_for_timer_events'
         ... 6 levels...
        from
/usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/packet_worker_runner:47:in
`new'
        from
/usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/packet_worker_runner:47
        from /usr/bin/packet_worker_runner:19:in `load'
        from /usr/bin/packet_worker_runner:19
ffapi ~/public_html/feedflix.com/ff:
==============


On Tue, Nov 18, 2008 at 11:03 PM, hemant <gethemant at gmail.com> wrote:

> I am puzzled. But anyways, just then open backgroundrb script from
> script directory and add require "bdrb_start_stop" after require
> "backgroundrb_server" and try.
>
>
>
> On Wed, Nov 19, 2008 at 12:29 PM, Raghu Srinivasan
> <raghu.srinivasan at gmail.com> wrote:
> > Hi Hemant,
> >
> > 1. I installed this version (0.1.14) of packet and tried - I still get
> the
> > same error
> > 2. My Rails version is 2.0.2. No, config.plans is commented out in my
> env.rb
> > (which means all plugins including BDRB must be getting loaded)
> > 3. Yes, I have this file
> >
> > I am on Ubuntu (Hardy) and the earlier BDRB version I've used all along
> runs
> > with no issues.
> >
> > Thanks,
> >
> > Raghu
> >
> > On Tue, Nov 18, 2008 at 10:50 PM, hemant <gethemant at gmail.com> wrote:
> >>
> >> 1. You need 0.1.14 packet version
> >> 2. which rails version you are using? Are you having config.plugins in
> >> environment.rb?
> >> 3. Do you have
> >> vendor/plugins/backgroundrb/lib/backgroundrb/bdrb_start_stop.rb
> >> file?
> >>
> >>
> >>
> >> On Wed, Nov 19, 2008 at 11:05 AM, Raghu Srinivasan
> >> <raghu.srinivasan at gmail.com> wrote:
> >> > Hello everyone, I am hoping to get some help with my upgrade of BDRB.
> >> >
> >> > As per one of Hemant's earlier emails, I started following along as
> per
> >> > the
> >> > instructions at http://backgroundrb.rubyforge.org
> >> >
> >> > 1) git clone for bdrb (after I removed the older backgroundrb
> directory
> >> > from
> >> > /vendor/plugins)
> >> > 2) upgraded packet to 0.1.7 from 0.1.5
> >> > 3) removed backgroundrb and load_worker_env.rb from the script
> directory
> >> > 4) did a rake backgroundrb:setup
> >> > 5) did a rake db:migrate and ensured that the table got created in the
> >> > db
> >> > 6) copied over my original backgroundrb.yml
> >> >
> >> > So far so good. Now when I try to start bdrb, I get this error:
> >> > ======
> >> > ffapi ~/public_html/feedflix.com/ff: script/backgroundrb start
> >> >
> >> >
> /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:263:in
> >> > `load_missing_constant': uninitialized constant
> BackgrounDRb::StartStop
> >> > (NameError)
> >> >         from
> >> >
> >> >
> /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:453:in
> >> > `const_missing'
> >> >         from script/backgroundrb:32
> >> > ffapi ~/public_html/feedflix.com/ff:
> >> > =======
> >> > I feel like I am missing something obvious here. Can someone help?
> >> >
> >> > Thanks!
> >> > Raghu
> >> >
> >> >
> >> > _______________________________________________
> >> > Backgroundrb-devel mailing list
> >> > Backgroundrb-devel at rubyforge.org
> >> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >> >
> >>
> >>
> >>
> >> --
> >> Let them talk of their oriental summer climes of everlasting
> >> conservatories; give me the privilege of making my own summer with my
> >> own coals.
> >>
> >> http://gnufied.org
> >
> >
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081118/c9ddf3e6/attachment-0001.html>

From gethemant at gmail.com  Wed Nov 19 06:29:29 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 19 Nov 2008 16:59:29 +0530
Subject: [Backgroundrb-devel] Error starting up Backgroundrb
In-Reply-To: <6482c06a0811182321s188229c6jae6efb662de9ae95@mail.gmail.com>
References: <6482c06a0811182135v539c2748uc740bee382e5c9ff@mail.gmail.com>
	<fb283d5e0811182250i254348f0sfe7eaae25819b5d7@mail.gmail.com>
	<6482c06a0811182259g2effd93ci569ad821821b0df@mail.gmail.com>
	<fb283d5e0811182303h161bd849wf978f6c7f6d2340f@mail.gmail.com>
	<6482c06a0811182321s188229c6jae6efb662de9ae95@mail.gmail.com>
Message-ID: <fb283d5e0811190329o34395f7eg2a81795a4e8f9404@mail.gmail.com>

There is something seriously wrong with the your setup. Can you create
a sample rails application with bdrb plugin and send the zip file, if
it doesn't work.

Looks like, none of the stuff from lib directory are getting picked.
This is a symptom of plugins not getting loaded when using
./script/backgroundrb start command. Plugins should be loaded because
we are loading environment.rb file in backgroundrb start/stop script.



On Wed, Nov 19, 2008 at 12:51 PM, Raghu Srinivasan
<raghu.srinivasan at gmail.com> wrote:
> I added that require and now the 'start' command seems to work fine but the
> log has an error in it for "uninitialized constant
> BackgrounDRb::MetaWorker::BdrbJobQueue (NameError)". Here it is:
> ============
> ffapi ~/public_html/feedflix.com/ff: ./script/backgroundrb start
> Starting BackgrounDRb ....
> ffapi ~/public_html/feedflix.com/ff: ls -lrt log/ba*
> -rw-r--r-- 1 raghus raghus   57 Nov 19 07:19 log/backgroundrb_11007.log
> -rw-r--r-- 1 raghus raghus 1366 Nov 19 07:19
> log/backgroundrb_debug_11007.log
> ffapi ~/public_html/feedflix.com/ff: cat log/backgroundrb_debug_11007.log
> /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:478:in
> `const_missing': uninitialized constant
> BackgrounDRb::MetaWorker::BdrbJobQueue (NameError)
>         from
> /home/raghus/public_html/feedflix.com/ff/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:271:in
> `check_for_enqueued_tasks'
>         from
> /home/raghus/public_html/feedflix.com/ff/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:133:in
> `worker_init'
>         from
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_periodic_event.rb:23:in
> `call'
>         from
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_periodic_event.rb:23:in
> `run'
>         from
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_core.rb:301:in
> `check_for_timer_events'
>         from
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_core.rb:301:in
> `each'
>         from
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_core.rb:301:in
> `check_for_timer_events'
>         from
> /home/raghus/public_html/feedflix.com/ff/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:296:in
> `check_for_timer_events'
>          ... 6 levels...
>         from
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/packet_worker_runner:47:in
> `new'
>         from
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/packet_worker_runner:47
>         from /usr/bin/packet_worker_runner:19:in `load'
>         from /usr/bin/packet_worker_runner:19
> ffapi ~/public_html/feedflix.com/ff:
> ==============
>
>
> On Tue, Nov 18, 2008 at 11:03 PM, hemant <gethemant at gmail.com> wrote:
>>
>> I am puzzled. But anyways, just then open backgroundrb script from
>> script directory and add require "bdrb_start_stop" after require
>> "backgroundrb_server" and try.
>>
>>
>>
>> On Wed, Nov 19, 2008 at 12:29 PM, Raghu Srinivasan
>> <raghu.srinivasan at gmail.com> wrote:
>> > Hi Hemant,
>> >
>> > 1. I installed this version (0.1.14) of packet and tried - I still get
>> > the
>> > same error
>> > 2. My Rails version is 2.0.2. No, config.plans is commented out in my
>> > env.rb
>> > (which means all plugins including BDRB must be getting loaded)
>> > 3. Yes, I have this file
>> >
>> > I am on Ubuntu (Hardy) and the earlier BDRB version I've used all along
>> > runs
>> > with no issues.
>> >
>> > Thanks,
>> >
>> > Raghu
>> >
>> > On Tue, Nov 18, 2008 at 10:50 PM, hemant <gethemant at gmail.com> wrote:
>> >>
>> >> 1. You need 0.1.14 packet version
>> >> 2. which rails version you are using? Are you having config.plugins in
>> >> environment.rb?
>> >> 3. Do you have
>> >> vendor/plugins/backgroundrb/lib/backgroundrb/bdrb_start_stop.rb
>> >> file?
>> >>
>> >>
>> >>
>> >> On Wed, Nov 19, 2008 at 11:05 AM, Raghu Srinivasan
>> >> <raghu.srinivasan at gmail.com> wrote:
>> >> > Hello everyone, I am hoping to get some help with my upgrade of BDRB.
>> >> >
>> >> > As per one of Hemant's earlier emails, I started following along as
>> >> > per
>> >> > the
>> >> > instructions at http://backgroundrb.rubyforge.org
>> >> >
>> >> > 1) git clone for bdrb (after I removed the older backgroundrb
>> >> > directory
>> >> > from
>> >> > /vendor/plugins)
>> >> > 2) upgraded packet to 0.1.7 from 0.1.5
>> >> > 3) removed backgroundrb and load_worker_env.rb from the script
>> >> > directory
>> >> > 4) did a rake backgroundrb:setup
>> >> > 5) did a rake db:migrate and ensured that the table got created in
>> >> > the
>> >> > db
>> >> > 6) copied over my original backgroundrb.yml
>> >> >
>> >> > So far so good. Now when I try to start bdrb, I get this error:
>> >> > ======
>> >> > ffapi ~/public_html/feedflix.com/ff: script/backgroundrb start
>> >> >
>> >> >
>> >> > /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:263:in
>> >> > `load_missing_constant': uninitialized constant
>> >> > BackgrounDRb::StartStop
>> >> > (NameError)
>> >> >         from
>> >> >
>> >> >
>> >> > /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:453:in
>> >> > `const_missing'
>> >> >         from script/backgroundrb:32
>> >> > ffapi ~/public_html/feedflix.com/ff:
>> >> > =======
>> >> > I feel like I am missing something obvious here. Can someone help?
>> >> >
>> >> > Thanks!
>> >> > Raghu
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > Backgroundrb-devel mailing list
>> >> > Backgroundrb-devel at rubyforge.org
>> >> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Let them talk of their oriental summer climes of everlasting
>> >> conservatories; give me the privilege of making my own summer with my
>> >> own coals.
>> >>
>> >> http://gnufied.org
>> >
>> >
>>
>>
>>
>> --
>> Let them talk of their oriental summer climes of everlasting
>> conservatories; give me the privilege of making my own summer with my
>> own coals.
>>
>> http://gnufied.org
>
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From raghu.srinivasan at gmail.com  Wed Nov 19 10:36:10 2008
From: raghu.srinivasan at gmail.com (Raghu Srinivasan)
Date: Wed, 19 Nov 2008 07:36:10 -0800
Subject: [Backgroundrb-devel] Error starting up Backgroundrb
In-Reply-To: <fb283d5e0811190329o34395f7eg2a81795a4e8f9404@mail.gmail.com>
References: <6482c06a0811182135v539c2748uc740bee382e5c9ff@mail.gmail.com>
	<fb283d5e0811182250i254348f0sfe7eaae25819b5d7@mail.gmail.com>
	<6482c06a0811182259g2effd93ci569ad821821b0df@mail.gmail.com>
	<fb283d5e0811182303h161bd849wf978f6c7f6d2340f@mail.gmail.com>
	<6482c06a0811182321s188229c6jae6efb662de9ae95@mail.gmail.com>
	<fb283d5e0811190329o34395f7eg2a81795a4e8f9404@mail.gmail.com>
Message-ID: <6482c06a0811190736ma04210amf7cc2fb744c3a7e7@mail.gmail.com>

I'll build a setup from scratch and see what happens and if that doesn't
work, I'll email you the zip file.
Thanks for your help,
Raghu

On Wed, Nov 19, 2008 at 3:29 AM, hemant <gethemant at gmail.com> wrote:

> There is something seriously wrong with the your setup. Can you create
> a sample rails application with bdrb plugin and send the zip file, if
> it doesn't work.
>
> Looks like, none of the stuff from lib directory are getting picked.
> This is a symptom of plugins not getting loaded when using
> ./script/backgroundrb start command. Plugins should be loaded because
> we are loading environment.rb file in backgroundrb start/stop script.
>
>
>
> On Wed, Nov 19, 2008 at 12:51 PM, Raghu Srinivasan
> <raghu.srinivasan at gmail.com> wrote:
> > I added that require and now the 'start' command seems to work fine but
> the
> > log has an error in it for "uninitialized constant
> > BackgrounDRb::MetaWorker::BdrbJobQueue (NameError)". Here it is:
> > ============
> > ffapi ~/public_html/feedflix.com/ff: ./script/backgroundrb start
> > Starting BackgrounDRb ....
> > ffapi ~/public_html/feedflix.com/ff: ls -lrt log/ba*
> > -rw-r--r-- 1 raghus raghus   57 Nov 19 07:19 log/backgroundrb_11007.log
> > -rw-r--r-- 1 raghus raghus 1366 Nov 19 07:19
> > log/backgroundrb_debug_11007.log
> > ffapi ~/public_html/feedflix.com/ff: cat
> log/backgroundrb_debug_11007.log
> >
> /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:478:in
> > `const_missing': uninitialized constant
> > BackgrounDRb::MetaWorker::BdrbJobQueue (NameError)
> >         from
> > /home/raghus/public_html/
> feedflix.com/ff/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:271:in
> > `check_for_enqueued_tasks'
> >         from
> > /home/raghus/public_html/
> feedflix.com/ff/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:133:in
> > `worker_init'
> >         from
> >
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_periodic_event.rb:23:in
> > `call'
> >         from
> >
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_periodic_event.rb:23:in
> > `run'
> >         from
> >
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_core.rb:301:in
> > `check_for_timer_events'
> >         from
> >
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_core.rb:301:in
> > `each'
> >         from
> >
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/../lib/packet/packet_core.rb:301:in
> > `check_for_timer_events'
> >         from
> > /home/raghus/public_html/
> feedflix.com/ff/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:296:in
> > `check_for_timer_events'
> >          ... 6 levels...
> >         from
> > /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/packet_worker_runner:47:in
> > `new'
> >         from
> > /usr/lib/ruby/gems/1.8/gems/packet-0.1.14/bin/packet_worker_runner:47
> >         from /usr/bin/packet_worker_runner:19:in `load'
> >         from /usr/bin/packet_worker_runner:19
> > ffapi ~/public_html/feedflix.com/ff:
> > ==============
> >
> >
> > On Tue, Nov 18, 2008 at 11:03 PM, hemant <gethemant at gmail.com> wrote:
> >>
> >> I am puzzled. But anyways, just then open backgroundrb script from
> >> script directory and add require "bdrb_start_stop" after require
> >> "backgroundrb_server" and try.
> >>
> >>
> >>
> >> On Wed, Nov 19, 2008 at 12:29 PM, Raghu Srinivasan
> >> <raghu.srinivasan at gmail.com> wrote:
> >> > Hi Hemant,
> >> >
> >> > 1. I installed this version (0.1.14) of packet and tried - I still get
> >> > the
> >> > same error
> >> > 2. My Rails version is 2.0.2. No, config.plans is commented out in my
> >> > env.rb
> >> > (which means all plugins including BDRB must be getting loaded)
> >> > 3. Yes, I have this file
> >> >
> >> > I am on Ubuntu (Hardy) and the earlier BDRB version I've used all
> along
> >> > runs
> >> > with no issues.
> >> >
> >> > Thanks,
> >> >
> >> > Raghu
> >> >
> >> > On Tue, Nov 18, 2008 at 10:50 PM, hemant <gethemant at gmail.com> wrote:
> >> >>
> >> >> 1. You need 0.1.14 packet version
> >> >> 2. which rails version you are using? Are you having config.plugins
> in
> >> >> environment.rb?
> >> >> 3. Do you have
> >> >> vendor/plugins/backgroundrb/lib/backgroundrb/bdrb_start_stop.rb
> >> >> file?
> >> >>
> >> >>
> >> >>
> >> >> On Wed, Nov 19, 2008 at 11:05 AM, Raghu Srinivasan
> >> >> <raghu.srinivasan at gmail.com> wrote:
> >> >> > Hello everyone, I am hoping to get some help with my upgrade of
> BDRB.
> >> >> >
> >> >> > As per one of Hemant's earlier emails, I started following along as
> >> >> > per
> >> >> > the
> >> >> > instructions at http://backgroundrb.rubyforge.org
> >> >> >
> >> >> > 1) git clone for bdrb (after I removed the older backgroundrb
> >> >> > directory
> >> >> > from
> >> >> > /vendor/plugins)
> >> >> > 2) upgraded packet to 0.1.7 from 0.1.5
> >> >> > 3) removed backgroundrb and load_worker_env.rb from the script
> >> >> > directory
> >> >> > 4) did a rake backgroundrb:setup
> >> >> > 5) did a rake db:migrate and ensured that the table got created in
> >> >> > the
> >> >> > db
> >> >> > 6) copied over my original backgroundrb.yml
> >> >> >
> >> >> > So far so good. Now when I try to start bdrb, I get this error:
> >> >> > ======
> >> >> > ffapi ~/public_html/feedflix.com/ff: script/backgroundrb start
> >> >> >
> >> >> >
> >> >> >
> /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:263:in
> >> >> > `load_missing_constant': uninitialized constant
> >> >> > BackgrounDRb::StartStop
> >> >> > (NameError)
> >> >> >         from
> >> >> >
> >> >> >
> >> >> >
> /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:453:in
> >> >> > `const_missing'
> >> >> >         from script/backgroundrb:32
> >> >> > ffapi ~/public_html/feedflix.com/ff:
> >> >> > =======
> >> >> > I feel like I am missing something obvious here. Can someone help?
> >> >> >
> >> >> > Thanks!
> >> >> > Raghu
> >> >> >
> >> >> >
> >> >> > _______________________________________________
> >> >> > Backgroundrb-devel mailing list
> >> >> > Backgroundrb-devel at rubyforge.org
> >> >> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Let them talk of their oriental summer climes of everlasting
> >> >> conservatories; give me the privilege of making my own summer with my
> >> >> own coals.
> >> >>
> >> >> http://gnufied.org
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >> Let them talk of their oriental summer climes of everlasting
> >> conservatories; give me the privilege of making my own summer with my
> >> own coals.
> >>
> >> http://gnufied.org
> >
> >
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081119/b7c733ea/attachment.html>

From subimage at gmail.com  Wed Nov 19 20:57:24 2008
From: subimage at gmail.com (seth b)
Date: Wed, 19 Nov 2008 17:57:24 -0800
Subject: [Backgroundrb-devel] How to obtain task progress for
	long-running methods
In-Reply-To: <fb283d5e0811071004q2232d264ra7d685dc25a6253b@mail.gmail.com>
References: <1225987982.25767.7.camel@jnick.skarven.net>
	<fc3b32cf0811061557x202eb952m28b10c7680ce71c5@mail.gmail.com>
	<1226073854.25767.21.camel@jnick.skarven.net>
	<fb283d5e0811071004q2232d264ra7d685dc25a6253b@mail.gmail.com>
Message-ID: <7aff9b4c0811191757l7b88ca8al5654e23eb09d259@mail.gmail.com>

It'd be nice if you had a comprehensive tutorial covering the latest
"up to date" way to handle these things.

I'm pulling my fucking hair out trying to figure out how to upgrade
from 0.2.1 and the doc site isn't doing much but confusing me more.

Used to be - you'd create one worker for each task you were running.
All work was run from my "do_work" methods, and I'd grab progress by
calling methods on the worker from Rails. Simple enough....

I'm trying to transition to the latest BDRB right now and I'm utterly
confused. Perhaps something has changed drastically and I'm missing a
huge concept shift. Right now I've moved my "do_work" method to the
"create" method, which seems to run fine - but I can't reference any
of my workers or get any progress back.

Can anyone point to a full tutorial or code example of running an
async process and grabbing multiple results WITHOUT memcached? Or is
memcached required now?

Has the paradigm shifted? Do you only create one worker and call jobs
against that worker and grab results using the cache now?

Someone please point me in the right direction. My emails to the list
and questions on the irc channel have gone unanswered for the last
couple of days.

--------------------
seth - subimage llc
-----
http://sublog.subimage.com
-----
Cashboard - Estimates, invoices, and time tracking software - for free!
http://www.getcashboard.com
-----
Substruct - Open source RoR e-commerce software.
http://code.google.com/p/substruct/



On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at gmail.com> wrote:
> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at skarven.net> wrote:
>> Hmm... I'm using the git version of backgroundrb, and currently up-to-
>> date with the repository; I don't have an "ask_worker" method; I have
>> "ask_result." Maybe I should be asking: where can I obtain the version
>> of Backgroundrb that's covered in "The Rails Way," because it seems the
>> API has changed significantly.
>>
>> Anyway, I'm creating new workers via something like
>>
>> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key => 'some key
>> ', :data => {:arguments => values})
>>
>> The worker is coded to such that the process logic is called inside the
>> create method, and it posts progress via cache[worker_key]
>>
>>  def create(args = nil)
>>        run_my_process(args)
>>  end
>>
>>  def run_my_process(args)
>>    cache[worker_key] = {}
>>    ... do stuff
>>    cache[worker_key][:progress] = 25
>>    ... do more stuff
>>    cache[worker_key][:progress] = 50
>>    ... etc
>>  end
>>
>> While this runs, I can see the worker chugging away (because it logs as
>> it runs,) but if I call
>>
>> MiddleMan.worker(:foobar_worker).ask_result('some key')
>>
>> while it is running, I get nil.
>>
>> Prior to this, I had a prior version of the worker that stayed in
>> menory. To use that one, I invoked the process via
>>
>> MiddleMan.worker(:foobar_worker).async_run_my_process('some key')
>>
>> The only difference in the coding of the worker for that version was
>> that the cache key was job_key rather than worker_key. Calling
>> ask_results('some key') produced nil until the task was completed, at
>> which point it contained my end-state results.
>
> We sort of grew very conservative about memory usage in 1.1 release
> and decided to handover this task to memcache. Earlier, bdrb master
> process used to cache all these results and hence they were readily
> available. But unless I maintain an LRU based access, I will have hard
> time keeping memory usage of master process under check.
>
> Hence this responsibility was shelved off to worker. But hey, while a
> worker is processing a task, it can't server cache result, and thus
> use Memcache, which is out of process. Now even if worker is
> processing task, you still get your cache result requests served
> alright.
>
> More details can be found here:
>
> http://backgroundrb.rubyforge.org/workers/#result_caching
>
> Lastly, compared to any book, that page is usually more up to date about stuff.
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>

From gethemant at gmail.com  Wed Nov 19 22:25:36 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 20 Nov 2008 08:55:36 +0530
Subject: [Backgroundrb-devel] How to obtain task progress for
	long-running methods
In-Reply-To: <7aff9b4c0811191757l7b88ca8al5654e23eb09d259@mail.gmail.com>
References: <1225987982.25767.7.camel@jnick.skarven.net>
	<fc3b32cf0811061557x202eb952m28b10c7680ce71c5@mail.gmail.com>
	<1226073854.25767.21.camel@jnick.skarven.net>
	<fb283d5e0811071004q2232d264ra7d685dc25a6253b@mail.gmail.com>
	<7aff9b4c0811191757l7b88ca8al5654e23eb09d259@mail.gmail.com>
Message-ID: <fb283d5e0811191925i4b4f4c6dx9e65af4e085a3391@mail.gmail.com>

You need memcache, if you want to use result storing and retrieval to
work, when worker is processing a task.

http://backgroundrb.rubyforge.org/workers/#result_caching

AFAIK, thats all, thats needed to get started with result caching.
All documentation of how/what/why is on backgroundrb.rubyforge.org ,
so read whats there on the website carefully.

At last, Do not swear on mailing lists. You can do that on your blog,
irc (some channels allow, others don't). But NEVER on mailing list, i
hate that.



On Thu, Nov 20, 2008 at 7:27 AM, seth b <subimage at gmail.com> wrote:
> It'd be nice if you had a comprehensive tutorial covering the latest
> "up to date" way to handle these things.
>
> I'm pulling my fucking hair out trying to figure out how to upgrade
> from 0.2.1 and the doc site isn't doing much but confusing me more.
>
> Used to be - you'd create one worker for each task you were running.
> All work was run from my "do_work" methods, and I'd grab progress by
> calling methods on the worker from Rails. Simple enough....
>
> I'm trying to transition to the latest BDRB right now and I'm utterly
> confused. Perhaps something has changed drastically and I'm missing a
> huge concept shift. Right now I've moved my "do_work" method to the
> "create" method, which seems to run fine - but I can't reference any
> of my workers or get any progress back.
>
> Can anyone point to a full tutorial or code example of running an
> async process and grabbing multiple results WITHOUT memcached? Or is
> memcached required now?
>
> Has the paradigm shifted? Do you only create one worker and call jobs
> against that worker and grab results using the cache now?
>
> Someone please point me in the right direction. My emails to the list
> and questions on the irc channel have gone unanswered for the last
> couple of days.
>
> --------------------
> seth - subimage llc
> -----
> http://sublog.subimage.com
> -----
> Cashboard - Estimates, invoices, and time tracking software - for free!
> http://www.getcashboard.com
> -----
> Substruct - Open source RoR e-commerce software.
> http://code.google.com/p/substruct/
>
>
>
> On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at gmail.com> wrote:
>> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at skarven.net> wrote:
>>> Hmm... I'm using the git version of backgroundrb, and currently up-to-
>>> date with the repository; I don't have an "ask_worker" method; I have
>>> "ask_result." Maybe I should be asking: where can I obtain the version
>>> of Backgroundrb that's covered in "The Rails Way," because it seems the
>>> API has changed significantly.
>>>
>>> Anyway, I'm creating new workers via something like
>>>
>>> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key => 'some key
>>> ', :data => {:arguments => values})
>>>
>>> The worker is coded to such that the process logic is called inside the
>>> create method, and it posts progress via cache[worker_key]
>>>
>>>  def create(args = nil)
>>>        run_my_process(args)
>>>  end
>>>
>>>  def run_my_process(args)
>>>    cache[worker_key] = {}
>>>    ... do stuff
>>>    cache[worker_key][:progress] = 25
>>>    ... do more stuff
>>>    cache[worker_key][:progress] = 50
>>>    ... etc
>>>  end
>>>
>>> While this runs, I can see the worker chugging away (because it logs as
>>> it runs,) but if I call
>>>
>>> MiddleMan.worker(:foobar_worker).ask_result('some key')
>>>
>>> while it is running, I get nil.
>>>
>>> Prior to this, I had a prior version of the worker that stayed in
>>> menory. To use that one, I invoked the process via
>>>
>>> MiddleMan.worker(:foobar_worker).async_run_my_process('some key')
>>>
>>> The only difference in the coding of the worker for that version was
>>> that the cache key was job_key rather than worker_key. Calling
>>> ask_results('some key') produced nil until the task was completed, at
>>> which point it contained my end-state results.
>>
>> We sort of grew very conservative about memory usage in 1.1 release
>> and decided to handover this task to memcache. Earlier, bdrb master
>> process used to cache all these results and hence they were readily
>> available. But unless I maintain an LRU based access, I will have hard
>> time keeping memory usage of master process under check.
>>
>> Hence this responsibility was shelved off to worker. But hey, while a
>> worker is processing a task, it can't server cache result, and thus
>> use Memcache, which is out of process. Now even if worker is
>> processing task, you still get your cache result requests served
>> alright.
>>
>> More details can be found here:
>>
>> http://backgroundrb.rubyforge.org/workers/#result_caching
>>
>> Lastly, compared to any book, that page is usually more up to date about stuff.
>> _______________________________________________
>> Backgroundrb-devel mailing list
>> Backgroundrb-devel at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From ramon.tayag at gmail.com  Wed Nov 19 22:32:17 2008
From: ramon.tayag at gmail.com (Ramon Tayag)
Date: Thu, 20 Nov 2008 11:32:17 +0800
Subject: [Backgroundrb-devel] Invalid worker with name [worker name] and
	key
In-Reply-To: <f25d5ad20811180013p78a634f3i55ef6fa28d72438e@mail.gmail.com>
References: <f25d5ad20811180013p78a634f3i55ef6fa28d72438e@mail.gmail.com>
Message-ID: <f25d5ad20811191932v58ed8d71n98e15545758cd434@mail.gmail.com>

Any suggestions regarding this problem?

Thanks :)
Ramon Tayag

On Tue, Nov 18, 2008 at 4:13 PM, Ramon Tayag <ramon.tayag at gmail.com> wrote:
> Hey everyone.  I started to wonder why no mail was being sent from my
> app and I checked the log and saw this:
>
> I haven't upgraded my backgroundrb so I can't/shouldn't be a version
> error.  I decided to restart my cluster (mongrels and backgroundrb
> server) and tried to send again.  It worked.  What causes this
> problem?  And what can I do so that I'll be notified when this problem
> arises?
>
> Invalid worker with name mail_worker and key
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_connection.rb:52:in
> `ask_worker'
> /home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_worker.rb:103:in
> `async_method_invoke'
> /home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_worker.rb:37:in
> `receive_data'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_parser.rb:44:in
> `extract'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_parser.rb:26:in
> `loop'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_parser.rb:26:in
> `extract'
> /home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_worker.rb:32:in
> `receive_data'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:232:in
> `read_external_socket'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:222:in
> `handle_external_messages'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:196:in
> `handle_read_event'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:192:in
> `each'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:192:in
> `handle_read_event'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:146:in
> `start_reactor'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:139:in
> `loop'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_core.rb:139:in
> `start_reactor'
> /usr/lib/ruby/gems/1.8/gems/packet-0.1.13/lib/packet/packet_master.rb:21:in
> `run'
> /home/ramon/my-app/current/vendor/plugins/backgroundrb/server/lib/master_proxy.rb:14:in
> `initialize'
> /home/ramon/my-app/current/script/backgroundrb:47:in `new'
> /home/ramon/my-app/current/script/backgroundrb:47
> Client disconected
>
> Thanks,
> Ramon Tayag
>

From subimage at gmail.com  Wed Nov 19 22:45:00 2008
From: subimage at gmail.com (seth b)
Date: Wed, 19 Nov 2008 19:45:00 -0800
Subject: [Backgroundrb-devel] How to obtain task progress for
	long-running methods
In-Reply-To: <fb283d5e0811191925i4b4f4c6dx9e65af4e085a3391@mail.gmail.com>
References: <1225987982.25767.7.camel@jnick.skarven.net>
	<fc3b32cf0811061557x202eb952m28b10c7680ce71c5@mail.gmail.com>
	<1226073854.25767.21.camel@jnick.skarven.net>
	<fb283d5e0811071004q2232d264ra7d685dc25a6253b@mail.gmail.com>
	<7aff9b4c0811191757l7b88ca8al5654e23eb09d259@mail.gmail.com>
	<fb283d5e0811191925i4b4f4c6dx9e65af4e085a3391@mail.gmail.com>
Message-ID: <7aff9b4c0811191945v3b2abcc6q8f7234a42e355fc1@mail.gmail.com>

Give me a fucking break. What are you the internet language police?

No wonder everyone I know is switching to workling / starling. Not
only does the BDRB documentation suck, the support on the mailing list
sucks even worse.

I'll go with Ezra's recommendation and move along. Thanks for the
non-help, dickhead.

--------------------
seth - subimage llc
-----
http://sublog.subimage.com
-----
Cashboard - Estimates, invoices, and time tracking software - for free!
http://www.getcashboard.com
-----
Substruct - Open source RoR e-commerce software.
http://code.google.com/p/substruct/



On Wed, Nov 19, 2008 at 7:25 PM, hemant <gethemant at gmail.com> wrote:
> You need memcache, if you want to use result storing and retrieval to
> work, when worker is processing a task.
>
> http://backgroundrb.rubyforge.org/workers/#result_caching
>
> AFAIK, thats all, thats needed to get started with result caching.
> All documentation of how/what/why is on backgroundrb.rubyforge.org ,
> so read whats there on the website carefully.
>
> At last, Do not swear on mailing lists. You can do that on your blog,
> irc (some channels allow, others don't). But NEVER on mailing list, i
> hate that.
>
>
>
> On Thu, Nov 20, 2008 at 7:27 AM, seth b <subimage at gmail.com> wrote:
>> It'd be nice if you had a comprehensive tutorial covering the latest
>> "up to date" way to handle these things.
>>
>> I'm pulling my fucking hair out trying to figure out how to upgrade
>> from 0.2.1 and the doc site isn't doing much but confusing me more.
>>
>> Used to be - you'd create one worker for each task you were running.
>> All work was run from my "do_work" methods, and I'd grab progress by
>> calling methods on the worker from Rails. Simple enough....
>>
>> I'm trying to transition to the latest BDRB right now and I'm utterly
>> confused. Perhaps something has changed drastically and I'm missing a
>> huge concept shift. Right now I've moved my "do_work" method to the
>> "create" method, which seems to run fine - but I can't reference any
>> of my workers or get any progress back.
>>
>> Can anyone point to a full tutorial or code example of running an
>> async process and grabbing multiple results WITHOUT memcached? Or is
>> memcached required now?
>>
>> Has the paradigm shifted? Do you only create one worker and call jobs
>> against that worker and grab results using the cache now?
>>
>> Someone please point me in the right direction. My emails to the list
>> and questions on the irc channel have gone unanswered for the last
>> couple of days.
>>
>> --------------------
>> seth - subimage llc
>> -----
>> http://sublog.subimage.com
>> -----
>> Cashboard - Estimates, invoices, and time tracking software - for free!
>> http://www.getcashboard.com
>> -----
>> Substruct - Open source RoR e-commerce software.
>> http://code.google.com/p/substruct/
>>
>>
>>
>> On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at gmail.com> wrote:
>>> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at skarven.net> wrote:
>>>> Hmm... I'm using the git version of backgroundrb, and currently up-to-
>>>> date with the repository; I don't have an "ask_worker" method; I have
>>>> "ask_result." Maybe I should be asking: where can I obtain the version
>>>> of Backgroundrb that's covered in "The Rails Way," because it seems the
>>>> API has changed significantly.
>>>>
>>>> Anyway, I'm creating new workers via something like
>>>>
>>>> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key => 'some key
>>>> ', :data => {:arguments => values})
>>>>
>>>> The worker is coded to such that the process logic is called inside the
>>>> create method, and it posts progress via cache[worker_key]
>>>>
>>>>  def create(args = nil)
>>>>        run_my_process(args)
>>>>  end
>>>>
>>>>  def run_my_process(args)
>>>>    cache[worker_key] = {}
>>>>    ... do stuff
>>>>    cache[worker_key][:progress] = 25
>>>>    ... do more stuff
>>>>    cache[worker_key][:progress] = 50
>>>>    ... etc
>>>>  end
>>>>
>>>> While this runs, I can see the worker chugging away (because it logs as
>>>> it runs,) but if I call
>>>>
>>>> MiddleMan.worker(:foobar_worker).ask_result('some key')
>>>>
>>>> while it is running, I get nil.
>>>>
>>>> Prior to this, I had a prior version of the worker that stayed in
>>>> menory. To use that one, I invoked the process via
>>>>
>>>> MiddleMan.worker(:foobar_worker).async_run_my_process('some key')
>>>>
>>>> The only difference in the coding of the worker for that version was
>>>> that the cache key was job_key rather than worker_key. Calling
>>>> ask_results('some key') produced nil until the task was completed, at
>>>> which point it contained my end-state results.
>>>
>>> We sort of grew very conservative about memory usage in 1.1 release
>>> and decided to handover this task to memcache. Earlier, bdrb master
>>> process used to cache all these results and hence they were readily
>>> available. But unless I maintain an LRU based access, I will have hard
>>> time keeping memory usage of master process under check.
>>>
>>> Hence this responsibility was shelved off to worker. But hey, while a
>>> worker is processing a task, it can't server cache result, and thus
>>> use Memcache, which is out of process. Now even if worker is
>>> processing task, you still get your cache result requests served
>>> alright.
>>>
>>> More details can be found here:
>>>
>>> http://backgroundrb.rubyforge.org/workers/#result_caching
>>>
>>> Lastly, compared to any book, that page is usually more up to date about stuff.
>>> _______________________________________________
>>> Backgroundrb-devel mailing list
>>> Backgroundrb-devel at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>>
>> _______________________________________________
>> Backgroundrb-devel mailing list
>> Backgroundrb-devel at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>

From enzodm at gmail.com  Thu Nov 20 01:31:05 2008
From: enzodm at gmail.com (Samer Masry)
Date: Wed, 19 Nov 2008 22:31:05 -0800
Subject: [Backgroundrb-devel] How to obtain task progress for
	long-running methods
In-Reply-To: <7aff9b4c0811191945v3b2abcc6q8f7234a42e355fc1@mail.gmail.com>
References: <1225987982.25767.7.camel@jnick.skarven.net>
	<fc3b32cf0811061557x202eb952m28b10c7680ce71c5@mail.gmail.com>
	<1226073854.25767.21.camel@jnick.skarven.net>
	<fb283d5e0811071004q2232d264ra7d685dc25a6253b@mail.gmail.com>
	<7aff9b4c0811191757l7b88ca8al5654e23eb09d259@mail.gmail.com>
	<fb283d5e0811191925i4b4f4c6dx9e65af4e085a3391@mail.gmail.com>
	<7aff9b4c0811191945v3b2abcc6q8f7234a42e355fc1@mail.gmail.com>
Message-ID: <567BC594-E847-4F95-9312-0A254786E977@gmail.com>

There is no language policy.  Just curtosy.  We are all  
hackers( reverse engineers ).  It's not always easy but we're  
persistent.  If you can help with documentation or code please do it's  
how apps evolve.

On Nov 19, 2008, at 7:45 PM, "seth b" <subimage at gmail.com> wrote:

> Give me a fucking break. What are you the internet language police?
>
> No wonder everyone I know is switching to workling / starling. Not
> only does the BDRB documentation suck, the support on the mailing list
> sucks even worse.
>
> I'll go with Ezra's recommendation and move along. Thanks for the
> non-help, dickhead.
>
> --------------------
> seth - subimage llc
> -----
> http://sublog.subimage.com
> -----
> Cashboard - Estimates, invoices, and time tracking software - for  
> free!
> http://www.getcashboard.com
> -----
> Substruct - Open source RoR e-commerce software.
> http://code.google.com/p/substruct/
>
>
>
> On Wed, Nov 19, 2008 at 7:25 PM, hemant <gethemant at gmail.com> wrote:
>> You need memcache, if you want to use result storing and retrieval to
>> work, when worker is processing a task.
>>
>> http://backgroundrb.rubyforge.org/workers/#result_caching
>>
>> AFAIK, thats all, thats needed to get started with result caching.
>> All documentation of how/what/why is on backgroundrb.rubyforge.org ,
>> so read whats there on the website carefully.
>>
>> At last, Do not swear on mailing lists. You can do that on your blog,
>> irc (some channels allow, others don't). But NEVER on mailing list, i
>> hate that.
>>
>>
>>
>> On Thu, Nov 20, 2008 at 7:27 AM, seth b <subimage at gmail.com> wrote:
>>> It'd be nice if you had a comprehensive tutorial covering the latest
>>> "up to date" way to handle these things.
>>>
>>> I'm pulling my fucking hair out trying to figure out how to upgrade
>>> from 0.2.1 and the doc site isn't doing much but confusing me more.
>>>
>>> Used to be - you'd create one worker for each task you were running.
>>> All work was run from my "do_work" methods, and I'd grab progress by
>>> calling methods on the worker from Rails. Simple enough....
>>>
>>> I'm trying to transition to the latest BDRB right now and I'm  
>>> utterly
>>> confused. Perhaps something has changed drastically and I'm  
>>> missing a
>>> huge concept shift. Right now I've moved my "do_work" method to the
>>> "create" method, which seems to run fine - but I can't reference any
>>> of my workers or get any progress back.
>>>
>>> Can anyone point to a full tutorial or code example of running an
>>> async process and grabbing multiple results WITHOUT memcached? Or is
>>> memcached required now?
>>>
>>> Has the paradigm shifted? Do you only create one worker and call  
>>> jobs
>>> against that worker and grab results using the cache now?
>>>
>>> Someone please point me in the right direction. My emails to the  
>>> list
>>> and questions on the irc channel have gone unanswered for the last
>>> couple of days.
>>>
>>> --------------------
>>> seth - subimage llc
>>> -----
>>> http://sublog.subimage.com
>>> -----
>>> Cashboard - Estimates, invoices, and time tracking software - for  
>>> free!
>>> http://www.getcashboard.com
>>> -----
>>> Substruct - Open source RoR e-commerce software.
>>> http://code.google.com/p/substruct/
>>>
>>>
>>>
>>> On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at gmail.com> wrote:
>>>> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at skarven.net 
>>>> > wrote:
>>>>> Hmm... I'm using the git version of backgroundrb, and currently  
>>>>> up-to-
>>>>> date with the repository; I don't have an "ask_worker" method; I  
>>>>> have
>>>>> "ask_result." Maybe I should be asking: where can I obtain the  
>>>>> version
>>>>> of Backgroundrb that's covered in "The Rails Way," because it  
>>>>> seems the
>>>>> API has changed significantly.
>>>>>
>>>>> Anyway, I'm creating new workers via something like
>>>>>
>>>>> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key =>  
>>>>> 'some key
>>>>> ', :data => {:arguments => values})
>>>>>
>>>>> The worker is coded to such that the process logic is called  
>>>>> inside the
>>>>> create method, and it posts progress via cache[worker_key]
>>>>>
>>>>> def create(args = nil)
>>>>>       run_my_process(args)
>>>>> end
>>>>>
>>>>> def run_my_process(args)
>>>>>   cache[worker_key] = {}
>>>>>   ... do stuff
>>>>>   cache[worker_key][:progress] = 25
>>>>>   ... do more stuff
>>>>>   cache[worker_key][:progress] = 50
>>>>>   ... etc
>>>>> end
>>>>>
>>>>> While this runs, I can see the worker chugging away (because it  
>>>>> logs as
>>>>> it runs,) but if I call
>>>>>
>>>>> MiddleMan.worker(:foobar_worker).ask_result('some key')
>>>>>
>>>>> while it is running, I get nil.
>>>>>
>>>>> Prior to this, I had a prior version of the worker that stayed in
>>>>> menory. To use that one, I invoked the process via
>>>>>
>>>>> MiddleMan.worker(:foobar_worker).async_run_my_process('some key')
>>>>>
>>>>> The only difference in the coding of the worker for that version  
>>>>> was
>>>>> that the cache key was job_key rather than worker_key. Calling
>>>>> ask_results('some key') produced nil until the task was  
>>>>> completed, at
>>>>> which point it contained my end-state results.
>>>>
>>>> We sort of grew very conservative about memory usage in 1.1 release
>>>> and decided to handover this task to memcache. Earlier, bdrb master
>>>> process used to cache all these results and hence they were readily
>>>> available. But unless I maintain an LRU based access, I will have  
>>>> hard
>>>> time keeping memory usage of master process under check.
>>>>
>>>> Hence this responsibility was shelved off to worker. But hey,  
>>>> while a
>>>> worker is processing a task, it can't server cache result, and thus
>>>> use Memcache, which is out of process. Now even if worker is
>>>> processing task, you still get your cache result requests served
>>>> alright.
>>>>
>>>> More details can be found here:
>>>>
>>>> http://backgroundrb.rubyforge.org/workers/#result_caching
>>>>
>>>> Lastly, compared to any book, that page is usually more up to  
>>>> date about stuff.
>>>> _______________________________________________
>>>> Backgroundrb-devel mailing list
>>>> Backgroundrb-devel at rubyforge.org
>>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>>>
>>> _______________________________________________
>>> Backgroundrb-devel mailing list
>>> Backgroundrb-devel at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>>
>>
>>
>>
>> --
>> Let them talk of their oriental summer climes of everlasting
>> conservatories; give me the privilege of making my own summer with my
>> own coals.
>>
>> http://gnufied.org
>>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

From f.schwach at uea.ac.uk  Thu Nov 20 03:56:49 2008
From: f.schwach at uea.ac.uk (Frank Schwach)
Date: Thu, 20 Nov 2008 08:56:49 +0000
Subject: [Backgroundrb-devel] How to obtain task progress
	for	long-running methods
In-Reply-To: <7aff9b4c0811191945v3b2abcc6q8f7234a42e355fc1@mail.gmail.com>
References: <1225987982.25767.7.camel@jnick.skarven.net>
	<fc3b32cf0811061557x202eb952m28b10c7680ce71c5@mail.gmail.com>
	<1226073854.25767.21.camel@jnick.skarven.net>
	<fb283d5e0811071004q2232d264ra7d685dc25a6253b@mail.gmail.com>
	<7aff9b4c0811191757l7b88ca8al5654e23eb09d259@mail.gmail.com>
	<fb283d5e0811191925i4b4f4c6dx9e65af4e085a3391@mail.gmail.com>
	<7aff9b4c0811191945v3b2abcc6q8f7234a42e355fc1@mail.gmail.com>
Message-ID: <1227171409.5055.642.camel@fschwachbiopc.bio.uea.ac.uk>

While I do agree that the documentation for BDRB can be quite confusing,
especially with major changes happening so frequently, it is quite
unfair to say that there is no support on the mailing list. Hemant and
others often write long and detailed replies to posts here and on many
other public forums but I guess it depends how you ask. 
Your use of abusive language is completely unacceptable and it throws a
rather bad light on your company (BTW: quite a lot of advertisement in
your posts)! If I was the admin on this mailing list, I'd ban you from
the list.

Frank


On Wed, 2008-11-19 at 19:45 -0800, seth b wrote:
> Give me a fucking break. What are you the internet language police?
> 
> No wonder everyone I know is switching to workling / starling. Not
> only does the BDRB documentation suck, the support on the mailing list
> sucks even worse.
> 
> I'll go with Ezra's recommendation and move along. Thanks for the
> non-help, dickhead.
> 
> --------------------
> seth - subimage llc
> -----
> http://sublog.subimage.com
> -----
> Cashboard - Estimates, invoices, and time tracking software - for free!
> http://www.getcashboard.com
> -----
> Substruct - Open source RoR e-commerce software.
> http://code.google.com/p/substruct/
> 
> 
> 
> On Wed, Nov 19, 2008 at 7:25 PM, hemant <gethemant at gmail.com> wrote:
> > You need memcache, if you want to use result storing and retrieval to
> > work, when worker is processing a task.
> >
> > http://backgroundrb.rubyforge.org/workers/#result_caching
> >
> > AFAIK, thats all, thats needed to get started with result caching.
> > All documentation of how/what/why is on backgroundrb.rubyforge.org ,
> > so read whats there on the website carefully.
> >
> > At last, Do not swear on mailing lists. You can do that on your blog,
> > irc (some channels allow, others don't). But NEVER on mailing list, i
> > hate that.
> >
> >
> >
> > On Thu, Nov 20, 2008 at 7:27 AM, seth b <subimage at gmail.com> wrote:
> >> It'd be nice if you had a comprehensive tutorial covering the latest
> >> "up to date" way to handle these things.
> >>
> >> I'm pulling my fucking hair out trying to figure out how to upgrade
> >> from 0.2.1 and the doc site isn't doing much but confusing me more.
> >>
> >> Used to be - you'd create one worker for each task you were running.
> >> All work was run from my "do_work" methods, and I'd grab progress by
> >> calling methods on the worker from Rails. Simple enough....
> >>
> >> I'm trying to transition to the latest BDRB right now and I'm utterly
> >> confused. Perhaps something has changed drastically and I'm missing a
> >> huge concept shift. Right now I've moved my "do_work" method to the
> >> "create" method, which seems to run fine - but I can't reference any
> >> of my workers or get any progress back.
> >>
> >> Can anyone point to a full tutorial or code example of running an
> >> async process and grabbing multiple results WITHOUT memcached? Or is
> >> memcached required now?
> >>
> >> Has the paradigm shifted? Do you only create one worker and call jobs
> >> against that worker and grab results using the cache now?
> >>
> >> Someone please point me in the right direction. My emails to the list
> >> and questions on the irc channel have gone unanswered for the last
> >> couple of days.
> >>
> >> --------------------
> >> seth - subimage llc
> >> -----
> >> http://sublog.subimage.com
> >> -----
> >> Cashboard - Estimates, invoices, and time tracking software - for free!
> >> http://www.getcashboard.com
> >> -----
> >> Substruct - Open source RoR e-commerce software.
> >> http://code.google.com/p/substruct/
> >>
> >>
> >>
> >> On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at gmail.com> wrote:
> >>> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at skarven.net> wrote:
> >>>> Hmm... I'm using the git version of backgroundrb, and currently up-to-
> >>>> date with the repository; I don't have an "ask_worker" method; I have
> >>>> "ask_result." Maybe I should be asking: where can I obtain the version
> >>>> of Backgroundrb that's covered in "The Rails Way," because it seems the
> >>>> API has changed significantly.
> >>>>
> >>>> Anyway, I'm creating new workers via something like
> >>>>
> >>>> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key => 'some key
> >>>> ', :data => {:arguments => values})
> >>>>
> >>>> The worker is coded to such that the process logic is called inside the
> >>>> create method, and it posts progress via cache[worker_key]
> >>>>
> >>>>  def create(args = nil)
> >>>>        run_my_process(args)
> >>>>  end
> >>>>
> >>>>  def run_my_process(args)
> >>>>    cache[worker_key] = {}
> >>>>    ... do stuff
> >>>>    cache[worker_key][:progress] = 25
> >>>>    ... do more stuff
> >>>>    cache[worker_key][:progress] = 50
> >>>>    ... etc
> >>>>  end
> >>>>
> >>>> While this runs, I can see the worker chugging away (because it logs as
> >>>> it runs,) but if I call
> >>>>
> >>>> MiddleMan.worker(:foobar_worker).ask_result('some key')
> >>>>
> >>>> while it is running, I get nil.
> >>>>
> >>>> Prior to this, I had a prior version of the worker that stayed in
> >>>> menory. To use that one, I invoked the process via
> >>>>
> >>>> MiddleMan.worker(:foobar_worker).async_run_my_process('some key')
> >>>>
> >>>> The only difference in the coding of the worker for that version was
> >>>> that the cache key was job_key rather than worker_key. Calling
> >>>> ask_results('some key') produced nil until the task was completed, at
> >>>> which point it contained my end-state results.
> >>>
> >>> We sort of grew very conservative about memory usage in 1.1 release
> >>> and decided to handover this task to memcache. Earlier, bdrb master
> >>> process used to cache all these results and hence they were readily
> >>> available. But unless I maintain an LRU based access, I will have hard
> >>> time keeping memory usage of master process under check.
> >>>
> >>> Hence this responsibility was shelved off to worker. But hey, while a
> >>> worker is processing a task, it can't server cache result, and thus
> >>> use Memcache, which is out of process. Now even if worker is
> >>> processing task, you still get your cache result requests served
> >>> alright.
> >>>
> >>> More details can be found here:
> >>>
> >>> http://backgroundrb.rubyforge.org/workers/#result_caching
> >>>
> >>> Lastly, compared to any book, that page is usually more up to date about stuff.
> >>> _______________________________________________
> >>> Backgroundrb-devel mailing list
> >>> Backgroundrb-devel at rubyforge.org
> >>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >>>
> >> _______________________________________________
> >> Backgroundrb-devel mailing list
> >> Backgroundrb-devel at rubyforge.org
> >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >>
> >
> >
> >
> > --
> > Let them talk of their oriental summer climes of everlasting
> > conservatories; give me the privilege of making my own summer with my
> > own coals.
> >
> > http://gnufied.org
> >
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel


From toastkid.williams at gmail.com  Thu Nov 20 05:29:01 2008
From: toastkid.williams at gmail.com (Max Williams)
Date: Thu, 20 Nov 2008 10:29:01 +0000
Subject: [Backgroundrb-devel] Get a unique job id back when enqueing a task
Message-ID: <f1b194900811200229m61798768r8ee299f7e9b0a41d@mail.gmail.com>

This is a simple question, but i can't find the answer on the internets.  Is
there a way, when putting a job into the job queue, to get back an object
for the BdrbJobQueue record that is created?  (or at least its id)

I need to pass a simple id around my app that will always relate to a
specific job.  At the moment i'm doing this by making a unique job key
string and using that to create and track the job, but it's a clumsy
approach and i'd rather just use the id from the db table, which i would
pull from the new job object.  Assuming i can get it back.

Thanks for the help, bdrbheads.  ;-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081120/01b927c3/attachment.html>

From theturingmachine at gmail.com  Thu Nov 20 05:48:09 2008
From: theturingmachine at gmail.com (Jorge Dias)
Date: Thu, 20 Nov 2008 11:48:09 +0100
Subject: [Backgroundrb-devel] Stopping bdrb from another process gets the
	process killed
Message-ID: <fc4cdd60811200248k5e3a1cd8id2b4adfb183864b2@mail.gmail.com>

Hi, I've been having some issues integrating bdrb on a cruisecontrolrb
server
It's related to the way we stop the bdrb server. The thing is, right now
when we stop the process we are killing the parent of the group of the
process. This is ok if you're running script/backgroundrb stop directly from
the console, but if I'm running this from another process, let's say a rake
task, the rake task gets killed because it gets the SIGTERM signal.

In the class StarStop we have:

  def kill_process arg_pid_file
    pid = nil
    pid = File.open(arg_pid_file, "r") { |pid_handle|
pid_handle.gets.strip.chomp.to_i }
    pgid =  Process.getpgid(pid)
    puts "Stopping BackgrounDRb with pid #{pid}...."
    Process.kill('-TERM', pgid)
    File.delete(arg_pid_file) if File.exists?(arg_pid_file)
    puts "Success!"
  end

What I did was, that instead of killing pgid, I'm doing a system call and
killing the pid, like this

  def kill_process arg_pid_file
    pid = nil
    pid = File.open(arg_pid_file, "r") { |pid_handle|
pid_handle.gets.strip.chomp.to_i }
    puts "Stopping BackgrounDRb with pid #{pid}...."
    system("kill -s TERM #{pid}") #This is what's different
    File.delete(arg_pid_file) if File.exists?(arg_pid_file)
    puts "Success!"
  end

I did the system call because Process, says that there's no process with the
pid we extract from the file.

It's working good, but I don't know if there are some hidden side effects?
any ideas?

here's a rake task so you can try what I'm saying http://pastie.org/318949
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081120/cc8da325/attachment.html>

From gethemant at gmail.com  Thu Nov 20 06:26:02 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 20 Nov 2008 16:56:02 +0530
Subject: [Backgroundrb-devel] Get a unique job id back when enqueing a
	task
In-Reply-To: <f1b194900811200229m61798768r8ee299f7e9b0a41d@mail.gmail.com>
References: <f1b194900811200229m61798768r8ee299f7e9b0a41d@mail.gmail.com>
Message-ID: <fb283d5e0811200326k7528cd0cr4b531019f1a275d4@mail.gmail.com>

Yes!

You can use job_key method in your worker to retrieve the job key of
current task. Its a thread local variable and hence thread safe, can
be also used inside threads without problems.

You can also use persistent_job method inside your worker to retrieve
currently executed task.

On Thu, Nov 20, 2008 at 3:59 PM, Max Williams
<toastkid.williams at gmail.com> wrote:
> This is a simple question, but i can't find the answer on the internets.  Is
> there a way, when putting a job into the job queue, to get back an object
> for the BdrbJobQueue record that is created?  (or at least its id)
>
> I need to pass a simple id around my app that will always relate to a
> specific job.  At the moment i'm doing this by making a unique job key
> string and using that to create and track the job, but it's a clumsy
> approach and i'd rather just use the id from the db table, which i would
> pull from the new job object.  Assuming i can get it back.
>
> Thanks for the help, bdrbheads.  ;-)
>
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Thu Nov 20 06:29:57 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 20 Nov 2008 16:59:57 +0530
Subject: [Backgroundrb-devel] Stopping bdrb from another process gets
	the process killed
In-Reply-To: <fc4cdd60811200248k5e3a1cd8id2b4adfb183864b2@mail.gmail.com>
References: <fc4cdd60811200248k5e3a1cd8id2b4adfb183864b2@mail.gmail.com>
Message-ID: <fb283d5e0811200329w28d7ed62v8361cfc3fa67dc72@mail.gmail.com>

On Thu, Nov 20, 2008 at 4:18 PM, Jorge Dias <theturingmachine at gmail.com> wrote:
> Hi, I've been having some issues integrating bdrb on a cruisecontrolrb
> server
> It's related to the way we stop the bdrb server. The thing is, right now
> when we stop the process we are killing the parent of the group of the
> process. This is ok if you're running script/backgroundrb stop directly from
> the console, but if I'm running this from another process, let's say a rake
> task, the rake task gets killed because it gets the SIGTERM signal.
>
> In the class StarStop we have:
>
>   def kill_process arg_pid_file
>     pid = nil
>     pid = File.open(arg_pid_file, "r") { |pid_handle|
> pid_handle.gets.strip.chomp.to_i }
>     pgid =  Process.getpgid(pid)
>     puts "Stopping BackgrounDRb with pid #{pid}...."
>     Process.kill('-TERM', pgid)
>     File.delete(arg_pid_file) if File.exists?(arg_pid_file)
>     puts "Success!"
>   end
>
> What I did was, that instead of killing pgid, I'm doing a system call and
> killing the pid, like this
>
>   def kill_process arg_pid_file
>     pid = nil
>     pid = File.open(arg_pid_file, "r") { |pid_handle|
> pid_handle.gets.strip.chomp.to_i }
>     puts "Stopping BackgrounDRb with pid #{pid}...."
>     system("kill -s TERM #{pid}") #This is what's different
>     File.delete(arg_pid_file) if File.exists?(arg_pid_file)
>     puts "Success!"
>   end
>
> I did the system call because Process, says that there's no process with the
> pid we extract from the file.
>
> It's working good, but I don't know if there are some hidden side effects?
> any ideas?
>
> here's a rake task so you can try what I'm saying http://pastie.org/318949

Hmm, I will play with this and let you know. Doesn't look like much of
problem, except that system call is opaque.

From toastkid.williams at gmail.com  Thu Nov 20 09:24:52 2008
From: toastkid.williams at gmail.com (Max Williams)
Date: Thu, 20 Nov 2008 14:24:52 +0000
Subject: [Backgroundrb-devel] Get a unique job id back when enqueing a
	task
In-Reply-To: <fb283d5e0811200326k7528cd0cr4b531019f1a275d4@mail.gmail.com>
References: <f1b194900811200229m61798768r8ee299f7e9b0a41d@mail.gmail.com>
	<fb283d5e0811200326k7528cd0cr4b531019f1a275d4@mail.gmail.com>
Message-ID: <f1b194900811200624w1c9dd62en1f060ea50e7d6e87@mail.gmail.com>

Thanks Hemant.  My problem is though that i need to know a unique id for a
job before it is picked up by a worker (so i can start tracking it).

What would be perfect is to get the id of the job queue record: at the
moment that's what i'm doing but in quite a clumsy way.  Is there a way to
just get the actual BdrbJobQueue object back?

max

2008/11/20 hemant <gethemant at gmail.com>

> Yes!
>
> You can use job_key method in your worker to retrieve the job key of
> current task. Its a thread local variable and hence thread safe, can
> be also used inside threads without problems.
>
> You can also use persistent_job method inside your worker to retrieve
> currently executed task.
>
> On Thu, Nov 20, 2008 at 3:59 PM, Max Williams
> <toastkid.williams at gmail.com> wrote:
> > This is a simple question, but i can't find the answer on the internets.
>  Is
> > there a way, when putting a job into the job queue, to get back an object
> > for the BdrbJobQueue record that is created?  (or at least its id)
> >
> > I need to pass a simple id around my app that will always relate to a
> > specific job.  At the moment i'm doing this by making a unique job key
> > string and using that to create and track the job, but it's a clumsy
> > approach and i'd rather just use the id from the db table, which i would
> > pull from the new job object.  Assuming i can get it back.
> >
> > Thanks for the help, bdrbheads.  ;-)
> >
> >
> >
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081120/c9317167/attachment-0001.html>

From toastkid.williams at gmail.com  Thu Nov 20 09:26:53 2008
From: toastkid.williams at gmail.com (Max Williams)
Date: Thu, 20 Nov 2008 14:26:53 +0000
Subject: [Backgroundrb-devel] Get a unique job id back when enqueing a
	task
In-Reply-To: <f1b194900811200624w1c9dd62en1f060ea50e7d6e87@mail.gmail.com>
References: <f1b194900811200229m61798768r8ee299f7e9b0a41d@mail.gmail.com>
	<fb283d5e0811200326k7528cd0cr4b531019f1a275d4@mail.gmail.com>
	<f1b194900811200624w1c9dd62en1f060ea50e7d6e87@mail.gmail.com>
Message-ID: <f1b194900811200626n7c1d2fe9wfe946f9ab24002d1@mail.gmail.com>

Another dumb question...

Before i enqueue a job, I want to raise an exception if the Bdrb server
isn't running.  Is there  a quick and easy way to test for this?  I thought
i'd seen a method somewhere but i've been hunting round various docs and the
code and can't find a way.  I might just be being blind though.

thanks, max
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081120/572cc8d9/attachment.html>

From toastkid.williams at gmail.com  Thu Nov 20 09:54:15 2008
From: toastkid.williams at gmail.com (Max Williams)
Date: Thu, 20 Nov 2008 14:54:15 +0000
Subject: [Backgroundrb-devel] Get a unique job id back when enqueing a
	task
In-Reply-To: <f1b194900811200626n7c1d2fe9wfe946f9ab24002d1@mail.gmail.com>
References: <f1b194900811200229m61798768r8ee299f7e9b0a41d@mail.gmail.com>
	<fb283d5e0811200326k7528cd0cr4b531019f1a275d4@mail.gmail.com>
	<f1b194900811200624w1c9dd62en1f060ea50e7d6e87@mail.gmail.com>
	<f1b194900811200626n7c1d2fe9wfe946f9ab24002d1@mail.gmail.com>
Message-ID: <f1b194900811200654j72c1e988o12389fec0beae9d3@mail.gmail.com>

Never mind, i fixed server-testing problem, i was being a dumbass.  I'd
still like to know about the job ids though...

2008/11/20 Max Williams <toastkid.williams at gmail.com>

> Another dumb question...
>
> Before i enqueue a job, I want to raise an exception if the Bdrb server
> isn't running.  Is there  a quick and easy way to test for this?  I thought
> i'd seen a method somewhere but i've been hunting round various docs and the
> code and can't find a way.  I might just be being blind though.
>
> thanks, max
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081120/4caaba3d/attachment.html>

From gethemant at gmail.com  Thu Nov 20 10:59:22 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 20 Nov 2008 21:29:22 +0530
Subject: [Backgroundrb-devel] Get a unique job id back when enqueing a
	task
In-Reply-To: <f1b194900811200626n7c1d2fe9wfe946f9ab24002d1@mail.gmail.com>
References: <f1b194900811200229m61798768r8ee299f7e9b0a41d@mail.gmail.com>
	<fb283d5e0811200326k7528cd0cr4b531019f1a275d4@mail.gmail.com>
	<f1b194900811200624w1c9dd62en1f060ea50e7d6e87@mail.gmail.com>
	<f1b194900811200626n7c1d2fe9wfe946f9ab24002d1@mail.gmail.com>
Message-ID: <fb283d5e0811200759k60ad1cb9n898bd809a0eb2a7@mail.gmail.com>

The object returned by "persistent_job" method is an instance
BdrbJobQueue class.
Also, currently there is no API method to check if server is running.
I am afraid, you will have to hack your own or wait for next release.




On Thu, Nov 20, 2008 at 7:56 PM, Max Williams
<toastkid.williams at gmail.com> wrote:
> Another dumb question...
>
> Before i enqueue a job, I want to raise an exception if the Bdrb server
> isn't running.  Is there  a quick and easy way to test for this?  I thought
> i'd seen a method somewhere but i've been hunting round various docs and the
> code and can't find a way.  I might just be being blind though.
>
> thanks, max
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From toastkid.williams at gmail.com  Thu Nov 20 11:22:39 2008
From: toastkid.williams at gmail.com (Max Williams)
Date: Thu, 20 Nov 2008 16:22:39 +0000
Subject: [Backgroundrb-devel] Get a unique job id back when enqueing a
	task
In-Reply-To: <fb283d5e0811200759k60ad1cb9n898bd809a0eb2a7@mail.gmail.com>
References: <f1b194900811200229m61798768r8ee299f7e9b0a41d@mail.gmail.com>
	<fb283d5e0811200326k7528cd0cr4b531019f1a275d4@mail.gmail.com>
	<f1b194900811200624w1c9dd62en1f060ea50e7d6e87@mail.gmail.com>
	<f1b194900811200626n7c1d2fe9wfe946f9ab24002d1@mail.gmail.com>
	<fb283d5e0811200759k60ad1cb9n898bd809a0eb2a7@mail.gmail.com>
Message-ID: <f1b194900811200822s481c7672w994597a645cb7462@mail.gmail.com>

Don't i have the same problem with persistent_job though?  If persistent_job
is the currently running queued task, then when i put the job into the
queue, the worker hasn't started it and so the job doesn't exist - right?
Or, can MiddleMan get it back somehow?  I guess i'm talking about a job
request (which is what the db table holds) rather than the actual job.

As for testing if the server is connected properly, I monkey_patched my own
method :)  Quick and dirty, probably doesn't work in all situations but
suits my needs:

module BackgrounDRb

  class Config
    #returns socket for current environment eg "0.0.0.0:11006"
    def self.socket_string

"#{BDRB_CONFIG[:backgroundrb][:ip]}:#{BDRB_CONFIG[RAILS_ENV.to_sym][:backgroundrb][:port]}"
    end
  end

  class ClusterConnection
    #call on MiddleMan
    def connected_to_server?
      self.all_worker_info[BackgrounDRb::Config.socket_string] != nil
    end
  end

end

thanks, max

2008/11/20 hemant <gethemant at gmail.com>

> The object returned by "persistent_job" method is an instance
> BdrbJobQueue class.
> Also, currently there is no API method to check if server is running.
> I am afraid, you will have to hack your own or wait for next release.
>
>
>
>
> On Thu, Nov 20, 2008 at 7:56 PM, Max Williams
> <toastkid.williams at gmail.com> wrote:
> > Another dumb question...
> >
> > Before i enqueue a job, I want to raise an exception if the Bdrb server
> > isn't running.  Is there  a quick and easy way to test for this?  I
> thought
> > i'd seen a method somewhere but i've been hunting round various docs and
> the
> > code and can't find a way.  I might just be being blind though.
> >
> > thanks, max
> >
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081120/537f6c9f/attachment.html>

From raghu.srinivasan at gmail.com  Thu Nov 20 16:12:13 2008
From: raghu.srinivasan at gmail.com (Raghu Srinivasan)
Date: Thu, 20 Nov 2008 13:12:13 -0800
Subject: [Backgroundrb-devel] How do I get a value back from a worker in
	BDRB v 1.0.3
Message-ID: <6482c06a0811201312i3e30ac0fqab24652e9ed20bd7@mail.gmail.com>

I am currently on BDRB 1.0.3. What's the best way to set a
particular variable in the worker and then access it from the Rails
controller. I googled around but the info spans many different versions and
I thought I'd just ask the ML.
In the worker code, I am making an API call and getting some XML back and I
am parsing it with the idea of setting a cookie with one of the returned
values. So I want to register_status (or equivalent) that from within the
worker and then access it in my controller so that I can set the cookie.

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081120/b62a2407/attachment.html>

From toastkid.williams at gmail.com  Fri Nov 21 04:25:07 2008
From: toastkid.williams at gmail.com (Max Williams)
Date: Fri, 21 Nov 2008 09:25:07 +0000
Subject: [Backgroundrb-devel] Get a unique job id back when enqueing a
	task
In-Reply-To: <fb283d5e0811210025p456a2d9ak4a0b564150f96edf@mail.gmail.com>
References: <f1b194900811200229m61798768r8ee299f7e9b0a41d@mail.gmail.com>
	<fb283d5e0811200326k7528cd0cr4b531019f1a275d4@mail.gmail.com>
	<f1b194900811200624w1c9dd62en1f060ea50e7d6e87@mail.gmail.com>
	<f1b194900811200626n7c1d2fe9wfe946f9ab24002d1@mail.gmail.com>
	<fb283d5e0811200759k60ad1cb9n898bd809a0eb2a7@mail.gmail.com>
	<f1b194900811200822s481c7672w994597a645cb7462@mail.gmail.com>
	<fb283d5e0811210025p456a2d9ak4a0b564150f96edf@mail.gmail.com>
Message-ID: <f1b194900811210125u55aff5e9w24941a3fa5327041@mail.gmail.com>

I have this situation, maybe i'm not dealing with it very cleverly, but here
goes.

We have lessons that can be zipped up and downloaded by the user.  We use
backgroundrb to do the zipping up since it's quite cpu intensive and the
queueing system means that only one can be done at once, and our mongrels
don't get tied up with it (which they didn't like very much).

So, i shove a job into the zipping up queue, then open a lightbox style
window that checks with ajax every five seconds to see if the job has been
finished yet.  If it has then it triggers the download of the zip file.

The same user could download the same lesson many times, so i need some way
to tell these jobs apart.  At the moment i'm doing something horrible with
the current time, lesson id and user id to generate a unique key, and using
that key to search for the BdrbJobQueue object, getting the id and then
passing that through to my view page for the ajax method to use to see if
it's finished.  I can't rely on getting anything from the worker, because i
need to start tracking the job request as soon as it goes into the queue.
So, i can't wait for a worker to do anything with persistent job.

Do you see what i mean?  Am i going about this in a dumb way?

thanks for your time :)
max

2008/11/21 hemant <gethemant at gmail.com>

> I am afraid, I do not follow you. You said you wanted actual
> BdrbJobQueue object back, persistent_job returns exactly the same, but
> from within the worker. It will always return the job that has been
> just started. If you want to know the status of a task from outside,
> using MiddleMan proxy, then you can use job_key for that purpose. But,
> I am assuming, since user is setting the job_key himself, when he
> submits a task, he does know about the job key that he submitted,
> right?
>
>
>
> On Thu, Nov 20, 2008 at 9:52 PM, Max Williams
> <toastkid.williams at gmail.com> wrote:
> > Don't i have the same problem with persistent_job though?  If
> persistent_job
> > is the currently running queued task, then when i put the job into the
> > queue, the worker hasn't started it and so the job doesn't exist - right?
> > Or, can MiddleMan get it back somehow?  I guess i'm talking about a job
> > request (which is what the db table holds) rather than the actual job.
> >
> > As for testing if the server is connected properly, I monkey_patched my
> own
> > method :)  Quick and dirty, probably doesn't work in all situations but
> > suits my needs:
> >
> > module BackgrounDRb
> >
> >   class Config
> >     #returns socket for current environment eg "0.0.0.0:11006"
> >     def self.socket_string
> >
> >
> "#{BDRB_CONFIG[:backgroundrb][:ip]}:#{BDRB_CONFIG[RAILS_ENV.to_sym][:backgroundrb][:port]}"
> >     end
> >   end
> >
> >   class ClusterConnection
> >     #call on MiddleMan
> >     def connected_to_server?
> >       self.all_worker_info[BackgrounDRb::Config.socket_string] != nil
> >     end
> >   end
> >
> > end
> >
> > thanks, max
> >
> > 2008/11/20 hemant <gethemant at gmail.com>
> >>
> >> The object returned by "persistent_job" method is an instance
> >> BdrbJobQueue class.
> >> Also, currently there is no API method to check if server is running.
> >> I am afraid, you will have to hack your own or wait for next release.
> >>
> >>
> >>
> >>
> >> On Thu, Nov 20, 2008 at 7:56 PM, Max Williams
> >> <toastkid.williams at gmail.com> wrote:
> >> > Another dumb question...
> >> >
> >> > Before i enqueue a job, I want to raise an exception if the Bdrb
> server
> >> > isn't running.  Is there  a quick and easy way to test for this?  I
> >> > thought
> >> > i'd seen a method somewhere but i've been hunting round various docs
> and
> >> > the
> >> > code and can't find a way.  I might just be being blind though.
> >> >
> >> > thanks, max
> >> >
> >> > _______________________________________________
> >> > Backgroundrb-devel mailing list
> >> > Backgroundrb-devel at rubyforge.org
> >> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >> >
> >>
> >>
> >>
> >> --
> >> Let them talk of their oriental summer climes of everlasting
> >> conservatories; give me the privilege of making my own summer with my
> >> own coals.
> >>
> >> http://gnufied.org
> >
> >
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081121/e614bc37/attachment-0001.html>

From gethemant at gmail.com  Fri Nov 21 03:25:09 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 21 Nov 2008 13:55:09 +0530
Subject: [Backgroundrb-devel] Get a unique job id back when enqueing a
	task
In-Reply-To: <f1b194900811200822s481c7672w994597a645cb7462@mail.gmail.com>
References: <f1b194900811200229m61798768r8ee299f7e9b0a41d@mail.gmail.com>
	<fb283d5e0811200326k7528cd0cr4b531019f1a275d4@mail.gmail.com>
	<f1b194900811200624w1c9dd62en1f060ea50e7d6e87@mail.gmail.com>
	<f1b194900811200626n7c1d2fe9wfe946f9ab24002d1@mail.gmail.com>
	<fb283d5e0811200759k60ad1cb9n898bd809a0eb2a7@mail.gmail.com>
	<f1b194900811200822s481c7672w994597a645cb7462@mail.gmail.com>
Message-ID: <fb283d5e0811210025p456a2d9ak4a0b564150f96edf@mail.gmail.com>

I am afraid, I do not follow you. You said you wanted actual
BdrbJobQueue object back, persistent_job returns exactly the same, but
from within the worker. It will always return the job that has been
just started. If you want to know the status of a task from outside,
using MiddleMan proxy, then you can use job_key for that purpose. But,
I am assuming, since user is setting the job_key himself, when he
submits a task, he does know about the job key that he submitted,
right?



On Thu, Nov 20, 2008 at 9:52 PM, Max Williams
<toastkid.williams at gmail.com> wrote:
> Don't i have the same problem with persistent_job though?  If persistent_job
> is the currently running queued task, then when i put the job into the
> queue, the worker hasn't started it and so the job doesn't exist - right?
> Or, can MiddleMan get it back somehow?  I guess i'm talking about a job
> request (which is what the db table holds) rather than the actual job.
>
> As for testing if the server is connected properly, I monkey_patched my own
> method :)  Quick and dirty, probably doesn't work in all situations but
> suits my needs:
>
> module BackgrounDRb
>
>   class Config
>     #returns socket for current environment eg "0.0.0.0:11006"
>     def self.socket_string
>
> "#{BDRB_CONFIG[:backgroundrb][:ip]}:#{BDRB_CONFIG[RAILS_ENV.to_sym][:backgroundrb][:port]}"
>     end
>   end
>
>   class ClusterConnection
>     #call on MiddleMan
>     def connected_to_server?
>       self.all_worker_info[BackgrounDRb::Config.socket_string] != nil
>     end
>   end
>
> end
>
> thanks, max
>
> 2008/11/20 hemant <gethemant at gmail.com>
>>
>> The object returned by "persistent_job" method is an instance
>> BdrbJobQueue class.
>> Also, currently there is no API method to check if server is running.
>> I am afraid, you will have to hack your own or wait for next release.
>>
>>
>>
>>
>> On Thu, Nov 20, 2008 at 7:56 PM, Max Williams
>> <toastkid.williams at gmail.com> wrote:
>> > Another dumb question...
>> >
>> > Before i enqueue a job, I want to raise an exception if the Bdrb server
>> > isn't running.  Is there  a quick and easy way to test for this?  I
>> > thought
>> > i'd seen a method somewhere but i've been hunting round various docs and
>> > the
>> > code and can't find a way.  I might just be being blind though.
>> >
>> > thanks, max
>> >
>> > _______________________________________________
>> > Backgroundrb-devel mailing list
>> > Backgroundrb-devel at rubyforge.org
>> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>> >
>>
>>
>>
>> --
>> Let them talk of their oriental summer climes of everlasting
>> conservatories; give me the privilege of making my own summer with my
>> own coals.
>>
>> http://gnufied.org
>
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From nicpon at nicpon.net  Wed Nov 26 13:44:52 2008
From: nicpon at nicpon.net (Michal Szapiel)
Date: Wed, 26 Nov 2008 13:44:52 -0500
Subject: [Backgroundrb-devel] Problem while starting the server
Message-ID: <8d02450f0811261044v6b702093n498b985a7a16dbb8@mail.gmail.com>

Hello,
I have followed installation instructions, but I can't get the server to
start. I get following message in the log:
c:/ruby/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:477:in
`const_missing': uninitialized constant Packet::Reactor::UNIXSocket
(NameError)
    from
c:/ruby/lib/ruby/gems/1.8/gems/packet-0.1.14/lib/packet/packet_master.rb:98:in
`fork_and_load'
    from
c:/ruby/lib/ruby/gems/1.8/gems/packet-0.1.14/lib/packet/packet_master.rb:80:in
`start_worker'
    from
C:/AOL/branch/workers/AlliedTesting/vendor/plugins/backgroundrb/server/lib/master_proxy.rb:16:in
`initialize'
    from
c:/ruby/lib/ruby/gems/1.8/gems/packet-0.1.14/lib/packet/packet_master.rb:19:in
`run'
    from
C:/AOL/branch/workers/AlliedTesting/vendor/plugins/backgroundrb/server/lib/master_proxy.rb:14:in
`initialize'
    from
C:/AOL/branch/workers/AlliedTesting/config/../vendor/plugins/backgroundrb/lib/backgroundrb/bdrb_start_stop.rb:50:in
`new'
    from
C:/AOL/branch/workers/AlliedTesting/config/../vendor/plugins/backgroundrb/lib/backgroundrb/bdrb_start_stop.rb:50:in
`start'
    from script/backgroundrb:35

Have anyone had this problem? I tried looking for all the dependencies and
it seems to me that they are there, yet I still get this error.
Thanks,
Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081126/f1d059d1/attachment.html>

From gethemant at gmail.com  Wed Nov 26 14:12:47 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 27 Nov 2008 00:42:47 +0530
Subject: [Backgroundrb-devel] Problem while starting the server
In-Reply-To: <8d02450f0811261044v6b702093n498b985a7a16dbb8@mail.gmail.com>
References: <8d02450f0811261044v6b702093n498b985a7a16dbb8@mail.gmail.com>
Message-ID: <fb283d5e0811261112y2e321edej64731aeb3fd5690d@mail.gmail.com>

Bdrb, doesn't work on Windows.

On Thu, Nov 27, 2008 at 12:14 AM, Michal Szapiel <nicpon at nicpon.net> wrote:
> Hello,
> I have followed installation instructions, but I can't get the server to
> start. I get following message in the log:
> c:/ruby/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:477:in
> `const_missing': uninitialized constant Packet::Reactor::UNIXSocket
> (NameError)
>     from
> c:/ruby/lib/ruby/gems/1.8/gems/packet-0.1.14/lib/packet/packet_master.rb:98:in
> `fork_and_load'
>     from
> c:/ruby/lib/ruby/gems/1.8/gems/packet-0.1.14/lib/packet/packet_master.rb:80:in
> `start_worker'
>     from
> C:/AOL/branch/workers/AlliedTesting/vendor/plugins/backgroundrb/server/lib/master_proxy.rb:16:in
> `initialize'
>     from
> c:/ruby/lib/ruby/gems/1.8/gems/packet-0.1.14/lib/packet/packet_master.rb:19:in
> `run'
>     from
> C:/AOL/branch/workers/AlliedTesting/vendor/plugins/backgroundrb/server/lib/master_proxy.rb:14:in
> `initialize'
>     from
> C:/AOL/branch/workers/AlliedTesting/config/../vendor/plugins/backgroundrb/lib/backgroundrb/bdrb_start_stop.rb:50:in
> `new'
>     from
> C:/AOL/branch/workers/AlliedTesting/config/../vendor/plugins/backgroundrb/lib/backgroundrb/bdrb_start_stop.rb:50:in
> `start'
>     from script/backgroundrb:35
>
> Have anyone had this problem? I tried looking for all the dependencies and
> it seems to me that they are there, yet I still get this error.
> Thanks,
> Mike
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From william.yeung at gearboxsoft.com  Sat Nov 29 04:37:18 2008
From: william.yeung at gearboxsoft.com (Yeung William)
Date: Sat, 29 Nov 2008 17:37:18 +0800
Subject: [Backgroundrb-devel] Dynamic start/stop test environment
	backgroundrb during test execution?
Message-ID: <0E30C954-55D5-4717-A272-7708D3CEAFC3@gearboxsoft.com>

I would like to see if there is any chance I could start/stop the test  
environment backgroundrb automatically before and after test. Forget  
to start it or forget to stop it after test both cause quite a bit  
trouble.




From gissmog at gissmog.de  Sun Nov 30 14:05:26 2008
From: gissmog at gissmog.de (Jochen Kaechelin)
Date: Sun, 30 Nov 2008 20:05:26 +0100
Subject: [Backgroundrb-devel] Problem with scheduled Task - gets executed
	several times
Message-ID: <AF3E6A53-FB97-4A54-9886-F1F64C9E32E4@gissmog.de>

I have the following:

---
:backgroundrb:
   :port: 11006
   :ip: 0.0.0.0
   :environment: development
   :persistent_disabled: true
:schedules:
     :inventory_worker:
       :dummy:
         :trigger_args: * */10 * * * *


class InventoryWorker < BackgrounDRb::MetaWorker
   set_worker_name :inventory_worker
   # reload_on_schedule true

   def create(args = nil)
   end

   def dummy(args = nil)
     Inventory.create(:kadisnr => "10000", :inventory => 1)
   end
end


 >> MiddleMan.worker(:inventory_worker).worker_info
=> {:status=>:running, :worker=>:inventory_worker, :worker_key=>nil}


The Task getes executed after 10 minutes but the logfile will show me:

Inventory Create (0.2ms)   INSERT INTO `inventories` (`updated_at`,  
`kadisnr`, `inventory`, `created_at`) VALUES('2008-11-30 19:00:41',  
'10000', 1, '2008-11-30 19:00:41')
   SQL (0.4ms)   COMMIT
   SQL (0.1ms)   BEGIN
   Inventory Create (0.2ms)   INSERT INTO `inventories` (`updated_at`,  
`kadisnr`, `inventory`, `created_at`) VALUES('2008-11-30 19:00:42',  
'10000', 1, '2008-11-30 19:00:42')
   SQL (0.4ms)   COMMIT
   SQL (0.1ms)   BEGIN


so I get hundreds of new database entries.

Why is the dummy method executed several times? And why will it never  
stop again?

--
Jochen