From reynard.list at gmail.com  Wed Jan  2 17:38:58 2008
From: reynard.list at gmail.com (Reynard)
Date: Wed, 2 Jan 2008 17:38:58 -0500
Subject: [Backgroundrb-devel] disabling backgroundrb_debug.log
Message-ID: <26e148920801021438p6f47c81q229398a99d663a09@mail.gmail.com>

is there an easy way to disable logging to backgroundrb_debug.log?

Regards
- reynard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080102/195dee18/attachment.html 

From gethemant at gmail.com  Wed Jan  2 19:09:48 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 3 Jan 2008 05:39:48 +0530
Subject: [Backgroundrb-devel] disabling backgroundrb_debug.log
In-Reply-To: <26e148920801021438p6f47c81q229398a99d663a09@mail.gmail.com>
References: <26e148920801021438p6f47c81q229398a99d663a09@mail.gmail.com>
Message-ID: <fb283d5e0801021609l97a9261kb87849738e4d1f86@mail.gmail.com>

On Jan 3, 2008 4:08 AM, Reynard <reynard.list at gmail.com> wrote:
>
> is there an easy way to disable logging to backgroundrb_debug.log?
>

If you mean, you rather want it on console for debugging purpose, may
be you can use :log option, but perhaps you already know that.
Otherwise, there is no way to disable backgroundrb_debug.log. I am
putting 1.0.1 release pretty soon and I will add this.
If you have a patch ready that will be awesome.

I am waiting from Jason about his problems with mysql connections and
if they are solved.


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From emarceta at gmail.com  Wed Jan  2 20:08:50 2008
From: emarceta at gmail.com (Emil Marceta)
Date: Thu, 3 Jan 2008 02:08:50 +0100
Subject: [Backgroundrb-devel] models loading
Message-ID: <74b000990801021708p7864751fxcc25cb7e3fd775f9@mail.gmail.com>

Hi there,

In the latest releases bdrb attempts to load all models during startup
routine. We have models that depend on the particular RUBY_PLATFORM
but they live in the same models directory. It seems that the
inability to load them stops bdrb. Could the models requiring /
loading be done one demand, the same as Rails does?

thanks,
emil

From reynard.list at gmail.com  Thu Jan  3 00:35:07 2008
From: reynard.list at gmail.com (Reynard)
Date: Thu, 3 Jan 2008 00:35:07 -0500
Subject: [Backgroundrb-devel] disabling backgroundrb_debug.log
In-Reply-To: <fb283d5e0801021609l97a9261kb87849738e4d1f86@mail.gmail.com>
References: <26e148920801021438p6f47c81q229398a99d663a09@mail.gmail.com>
	<fb283d5e0801021609l97a9261kb87849738e4d1f86@mail.gmail.com>
Message-ID: <26e148920801022135p6bd80072s1f4173ff4a8f6ea1@mail.gmail.com>

Hi Hemant,

I just want it not to log anything on production use, since there might be
sensitive data.  Currently I can see all the data passed to the worker in
that file.
I don't have any patch and since I'm not familiar with the code I doubt if I
have time to implement it quickly, but if that's something easy for you to
do, that would be great :)

Thanks,
reynard


On Jan 2, 2008 7:09 PM, hemant <gethemant at gmail.com> wrote:

> On Jan 3, 2008 4:08 AM, Reynard <reynard.list at gmail.com> wrote:
> >
> > is there an easy way to disable logging to backgroundrb_debug.log?
> >
>
> If you mean, you rather want it on console for debugging purpose, may
> be you can use :log option, but perhaps you already know that.
> Otherwise, there is no way to disable backgroundrb_debug.log. I am
> putting 1.0.1 release pretty soon and I will add this.
> If you have a patch ready that will be awesome.
>
> I am waiting from Jason about his problems with mysql connections and
> if they are solved.
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080103/98154991/attachment.html 

From maku at makuchaku.info  Thu Jan  3 01:09:08 2008
From: maku at makuchaku.info (=?UTF-8?Q?=E0=A4=AE=E0=A4=AF=E0=A4=82=E0=A4=95_?=
	=?UTF-8?Q?=E0=A4=9C=E0=A5=88=E0=A4=A8_(?=
	=?UTF-8?Q?makuchaku)?=)
Date: Thu, 3 Jan 2008 11:39:08 +0530
Subject: [Backgroundrb-devel] A newbie question regarding BackgroundRb
In-Reply-To: <1199018373.13052.12.camel@shire>
References: <185269960712300340v5337f362j7994a6f7b5ae203b@mail.gmail.com>
	<1199018373.13052.12.camel@shire>
Message-ID: <185269960801022209t5b5f0486v431d565fa391b9c9@mail.gmail.com>

Hi Hemat, Geoff,

Thanks for the tip - I'll make sure that I am on the latest version :)

--
Maku


On 12/30/07, hemant kumar <gethemant at gmail.com> wrote:
>
> On Sun, 2007-12-30 at 17:10 +0530, ???? ??? (makuchaku) wrote:
> > Hi all,
> >
> > I just read Ezra's article on BackgroundRb and decided to use it one
> > of my projects. I installed the plugin into my rails project using
> > script/plugin and then ran rake backgroundrb:setup to copy the yml
> > scripts.
> >
> > Now my backgroundrb.yml looks like
> > =======
> > ---
> > port: "22222"
> > timer_sleep: 60
> > load_rails: true
> > environment: development
> > host: localhost
> > database_yml: config/database.yml
> > acl:
> >   deny: all
> >   allow: localhost 127.0.0.1
> >   order: deny,allow
> >
> > scheduled_task:
> >   :class: :tx_reaper_worker
> >   :worker_method: :do_work
> >   :trigger_args:
> >     :start: <%= Time.now %>
> >     :repeat_interval: <%= 1.second %
> > =======
> >
> > TxReaperWorker is my class which I generated.
> >
> > =======
> > class TxReaperWorker < BackgrounDRb::Rails
> >   def do_work(args)
> >     puts "Hello BackgroundRb"
> >   end
> > end
> > =======
> >
> > I started my rails app using
> > rake backgroundrb:start
> > script/server
> >
> > However, I am unable to view the output of "puts" anywhere. I can
> > connect successfully to localhost:22222 but cannot see anything
> > happening. Neither anything shown up on the app's output, nor on the
> > log/backgroundrb.log file.
> > However, the logfile does shows that "Log file created...".
> >
> > Am I missing any step? Doing anything wrong?
> >
> > Thanks in advance :)
> >
>
> You are using VERY old version of plugin, can you please checkout
> details regarding new version from:
>
> http://backgroundrb.rubyforge.org
>
>
>
>

From gethemant at gmail.com  Thu Jan  3 01:36:09 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 3 Jan 2008 12:06:09 +0530
Subject: [Backgroundrb-devel] disabling backgroundrb_debug.log
In-Reply-To: <26e148920801022135p6bd80072s1f4173ff4a8f6ea1@mail.gmail.com>
References: <26e148920801021438p6f47c81q229398a99d663a09@mail.gmail.com>
	<fb283d5e0801021609l97a9261kb87849738e4d1f86@mail.gmail.com>
	<26e148920801022135p6bd80072s1f4173ff4a8f6ea1@mail.gmail.com>
Message-ID: <fb283d5e0801022236m17866b51vdda35fed296bae2e@mail.gmail.com>

On Jan 3, 2008 11:05 AM, Reynard <reynard.list at gmail.com> wrote:
> Hi Hemant,
>
> I just want it not to log anything on production use, since there might be
> sensitive data.  Currently I can see all the data passed to the worker in
> that file.
> I don't have any patch and since I'm not familiar with the code I doubt if I
> have time to implement it quickly, but if that's something easy for you to
> do, that would be great :)

Yes, I will provide an option to disable logging in 1.0.1, no problems.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Thu Jan  3 01:38:48 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 3 Jan 2008 12:08:48 +0530
Subject: [Backgroundrb-devel] models loading
In-Reply-To: <74b000990801021708p7864751fxcc25cb7e3fd775f9@mail.gmail.com>
References: <74b000990801021708p7864751fxcc25cb7e3fd775f9@mail.gmail.com>
Message-ID: <fb283d5e0801022238m19762de6pd7d5c514057679f9@mail.gmail.com>

Hi,

On Jan 3, 2008 6:38 AM, Emil Marceta <emarceta at gmail.com> wrote:
> Hi there,
>
> In the latest releases bdrb attempts to load all models during startup
> routine. We have models that depend on the particular RUBY_PLATFORM
> but they live in the same models directory. It seems that the
> inability to load them stops bdrb. Could the models requiring /
> loading be done one demand, the same as Rails does?
>

The problem is, rails way of lazy loading doesn't work, when we are
creating ActiveRecord objects that were dumped at one end of network.
What I can do is, make this configurable, so as for people who don't
pass actual model objects across the wire, they can have this option
disabled.

From chuyeow at gmail.com  Thu Jan  3 01:51:27 2008
From: chuyeow at gmail.com (Cheah Chu Yeow)
Date: Thu, 3 Jan 2008 14:51:27 +0800
Subject: [Backgroundrb-devel] Trac not for general use?
Message-ID: <74ce3950801022251q52f71484v2463ce65756c52c7@mail.gmail.com>

Hi devs,

I was trying to open a new ticket with a minor doc patch at the
BackgroundDrb Trac (http://backgroundrb.devjavu.com/) and it seems
like only admins have permission to do so. Is this intended?

I read in the readme that patches should be mailed to this list, but I
much prefer to use Trac since there's already one set up. Any reason
Trac is not open to registered Devjavu users? Spam perhaps?

Cheers,
Chu Yeow

From gethemant at gmail.com  Thu Jan  3 03:39:24 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 3 Jan 2008 14:09:24 +0530
Subject: [Backgroundrb-devel] Trac not for general use?
In-Reply-To: <74ce3950801022251q52f71484v2463ce65756c52c7@mail.gmail.com>
References: <74ce3950801022251q52f71484v2463ce65756c52c7@mail.gmail.com>
Message-ID: <fb283d5e0801030039k72ebc55dr57163b84563bb77f@mail.gmail.com>

Thats weird, Trac is open for patches and tickets. However, you need
to be registered with Devjavu for creating a ticket.
I just tried posting a sample ticket it worked.

You may have better luck trying with Devjavu support or something.


On Jan 3, 2008 12:21 PM, Cheah Chu Yeow <chuyeow at gmail.com> wrote:
> Hi devs,
>
> I was trying to open a new ticket with a minor doc patch at the
> BackgroundDrb Trac (http://backgroundrb.devjavu.com/) and it seems
> like only admins have permission to do so. Is this intended?
>
> I read in the readme that patches should be mailed to this list, but I
> much prefer to use Trac since there's already one set up. Any reason
> Trac is not open to registered Devjavu users? Spam perhaps?
>
> Cheers,
> Chu Yeow
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From joost at yelloyello.com  Thu Jan  3 09:22:40 2008
From: joost at yelloyello.com (Joost Hietbrink (YelloYello))
Date: Thu, 03 Jan 2008 15:22:40 +0100
Subject: [Backgroundrb-devel] lost connection 2.0
In-Reply-To: <fb283d5e0712291832n43084814m6162c15b1dab4b06@mail.gmail.com>
References: <4774AC6F.3030302@yelloyello.com>	
	<fb283d5e0712280022h68b9380ancdb91f2a86842bea@mail.gmail.com>	
	<4774B3E1.4010908@yelloyello.com>	
	<fb283d5e0712280033ga167e64mb0f25175cde00c6f@mail.gmail.com>	
	<4774E426.4030901@yelloyello.com>	
	<fb283d5e0712290809x3824fc3en3420272f88126f98@mail.gmail.com>	
	<4776B42B.4020901@yelloyello.com>
	<fb283d5e0712291832n43084814m6162c15b1dab4b06@mail.gmail.com>
Message-ID: <477CEFB0.5010703@yelloyello.com>

The problem has occurred again... :( (with the 'thread_fix.tar.gz' version).
Should we set 'allow_concurrency' in the environment? Because we didn't 
do that yet..

Joost

hemant wrote:
> Hi Joost,
>
> I am attaching a version of plugin that fixes issues with mysql lost
> connections. Can you try and let me know, how were the results.
>
>
> On Dec 30, 2007 2:25 AM, Joost Hietbrink <joost at yelloyello.com> wrote:
>   
>>  This is the 'main' worker we created.. The problem is still occurring.. I
>> did not yet put the config things in environment.rb (like I said on:
>> http://rubyforge.org/pipermail/backgroundrb-devel/2007-December/001233.html)..
>> so maybe that fixes it?
>>
>>  Hope you can help..
>>
>>
>>  hemant wrote:
>>  Hi Joost,
>>
>> On Dec 28, 2007 5:25 PM, Joost Hietbrink (YelloYello)
>> <joost at yelloyello.com> wrote:
>>
>>
>>  The rails_model_worker.rb contains a module that fetches an object from the
>> database.
>>  We pass parameters to the worker (like classname & id (eg. Place and
>> 12312)) and fetch the object again in the worker (using this module method).
>>
>>  The lost connection occurs when we try to fetch the object (record) again
>> (in the worker..using the included module method).
>>
>>  The old BackgroundRb did not support sending ActiveRecord objects to the
>> worker (thats why we refetch it)..does the new version??
>>
>>  We now also installed libmysql-ruby1.8 (see:
>> http://wiki.rubyonrails.org/rails/pages/Mysql%2BConnection%2BProblems) so
>> maybe this fixes the problem..but we have to wait some time to see.. :)
>>
>>
>>  Sorry for late reply. But If you can send me the worker code, I will
>> try to fix it.
>> There seems to be some problems with AR in threaded environment. I
>> just want to make sure, none remain unfixed.
>>
>>
>>
>>
>> module RailsModelWorker
>> # Methods to be implemented in subclass
>>
>>   # execute_job should implement the work you want to be done, it is called
>> with the instance of the
>>   # rails model object, a boolean stating whether more jobs are already
>> waiting
>>   # (which can be used to postpone some general work to the last job in the
>> queue),
>>   # and a hash containing any extra arguments (which is validated to be a
>> hash before the call)
>>   # obj is not nil when entering this call
>> #  def execute_job(obj, pending_jobs, args)
>> #    raise NotImplementedError, "execute_job is an abstract method"
>> #  end
>>
>>   # parse_args can be optionally implemented, use it to set @var's which you
>> use in execute_job
>>   # it should accept an hash of arguments (which is validated to be a hash
>> before the call)
>> #  def parse_args(args)
>> #    raise NotImplementedError, "parse_args is an abstract method"
>> #  end
>>
>> # General methods
>>
>>   def create(args)
>>     log("new worker created")
>>     register_status(:booting)
>>     @job_queue = Array.new
>>
>>     if args.nil?
>>       errlog("weird: args argument of do_work is nil!")
>>     elsif !args.is_a?(Hash)
>>       errlog("weird: args argument of do_work is not a Hash")
>>     elsif !args.has_key?(:record_type)
>>       errlog("weird: args argument of do_work does not contain
>> :record_type")
>>     else
>>       @record_type = args[:record_type]
>>     end
>>
>>     if args.has_key?(:args) && args[:args].is_a?(Hash)
>>       parse_args(args[:args]) if defined?(parse_args)
>>     end
>>
>>     # This method is called in it's own new thread when you
>>     # call new worker. args is set to :args
>>     register_status(:running)
>>     @timer = add_periodic_timer(5) do
>>       if @job_queue.nil? || !@job_queue.is_a?(Array)
>>         register_status(:dead)
>>         cancel_timer(@timer) if !@timer.nil?
>>         errlog("weird: our job_queue object is nil or not Array
>> #{@job_queue}")
>>       else
>>         start_work() if @job_queue.size > 0
>>       end
>>     end
>>   end
>>
>>   def request_work(params)
>>     log("work requested")
>>     id = params[:id]
>>     args = params[:args]
>>     if id.blank?
>>       errlog("weird: id argument of request_indexing is nil")
>>     elsif @job_queue.nil? || !@job_queue.is_a?(Array)
>>       errlog("weird: our job_queue object is nil or not Array
>> #{@job_queue}")
>>     elsif args.nil? || !args.is_a?(Hash)
>>       errlog("weird: our args argument is nil or not Hash #{args}")
>>     else
>>       @job_queue.each do |job|
>>         job[:pending_jobs] = true
>>       end
>>       @request_time = Time.now
>>       @job_queue << { :object_id => id, :request_time => @request_time,
>> :args => args}
>> # TODO: map this to the new backgroundrb
>> #      results[:last_request_at] = @request_time
>>     end
>>   end
>>
>>   def start_work()
>>     result = false
>>     begin
>>       if @job_queue.nil? || !@job_queue.is_a?(Array)
>>         raise "weird: our job_queue object is nil or not Array
>> #{@job_queue}"
>>       end
>>       job = @job_queue.shift
>>       if job.nil? || !job.is_a?(Hash)
>>         raise "weird: our current job is not a Hash or nil: #{job}"
>>       elsif job[:object_id].nil?
>>         raise "weird: we got a nil object_id"
>>       elsif job[:args].nil?
>>         raise "weird: we got a nil args"
>>       elsif @record_type.nil?
>>         raise "weird: we can't do work because @record_type is nil"
>>       end
>>       log("Fetch rails object..")
>>       for i in 1..10 do
>>         obj = refetch_record(job[:object_id])
>>         if obj.nil?
>>           raise "Failed to retrieve object" if i == 10
>>           sleep(1)
>>           next
>>         end
>>         break
>>       end
>>       log("Start work..")
>>       execute_job(obj, job[:pending_jobs], job[:args]) if
>> defined?(execute_job)
>>       log("Work done..")
>>       result = true
>>     rescue StandardError => e
>>       errlog("Work failed.. Exception raised: #{e}")
>>       #TODO: email about any exception thrown by execute_job. Something like
>> this but with correct stubs
>>       # if RAILS_ENV == 'production'
>>         # request_stub = ActionController::AbstractRequest.new
>>         # request_stub.instance_eval('@env={}')
>>         # controller_stub = ApplicationController.new
>>         # ExceptionNotifier.deliver_exception_notification(exception,
>> controller_stub, request_stub, {})
>>       # end
>>     end
>>     # Sets the global results hash to contain last result for this index
>>     # Access it using: worker.results.to_hash[:success]
>> # TODO: map this to the new backgroundrb
>> #        results[:last_success_at] = job[:request_time] if result
>> #        results[:success] = result
>>     return result
>>   end
>>
>>   def refetch_record(id)
>>     result = nil
>>     begin
>>       result = @record_type.constantize.find(id)
>>     rescue NameError => e
>>       raise "Cannot get object of non existing model #{@record_type},
>> exception #{e.to_s}"
>>     end
>>     return result
>>   end
>>
>>   # Write to log/backgroundrb.log with class-name as prefix
>>   def log(s, type='info')
>>     logger.info("#{Time.now.strftime("%Y%m%d-%H:%M:%S")} (#{type})
>> #{self.class}: #{s}")
>>   end
>>   def errlog(s)
>>     log(s, 'error')
>>   end
>>   def debuglog(s)
>>     log(s, 'debug')
>>   end
>>   def warnlog(s)
>>     log(s, 'warning')
>>   end
>> end
>>
>>     
>
>
>
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080103/c22f937a/attachment.html 

From list at lylo.co.uk  Thu Jan  3 10:03:49 2008
From: list at lylo.co.uk (Olly Lylo)
Date: Thu, 3 Jan 2008 15:03:49 +0000
Subject: [Backgroundrb-devel] Start script not working when run via Rake
Message-ID: <f317a9f40801030703p49091be9x91f7f5f3a1b0161e@mail.gmail.com>

Hi there
I'm using the latest release (r300) and 'script/backgroundrb start' works
perfectly when executed on the server, but when put inside a Rake task, it
doesn't launch the worker processes.

The Rake task is as follows:


task :start_backgroundrb do
  run "cd #{current_path} && ./script/backgroundrb start"
end


When I run this task, the following files are created in the log directory:

backgroundrb.pid
backgroundrb_debug.log
backgroundrb_server.log


Observations: The process ID found in backgroundrb.pid doesn't exist,
backgroundrb_server.log is zero bytes long and backgroundrb_debug.log
contains the following:

# Logfile created on Thu Jan 03 14:43:24 +0000 2008 by /


It looks like the process is being started by Rake, but terminates before
the worker processes are started.   Could anyone shed any light on why this
might be happening and how I can resolve the issue?

I have also noticed that calling './script/backgroundrb stop' when
BackgroundRB isn't running throws an exception, which causes problems when
executing 'backgroundrb stop' in a rake task.  As a workaround I have
modified by 'backgroundrb' script as follows:

when 'stop'
  path = "#{RAILS_HOME}/log/backgroundrb.pid"
  pid = nil
  begin
    File.open(path, "r") { |pid_handle| pid =
pid_handle.gets.strip.chomp.to_i }
    pgid =  Process.getpgid(pid)
    Process.kill('TERM', pid)
    Process.kill('-TERM', pgid)
    Process.kill('KILL', pid)
  rescue
    puts $!
  ensure
    puts "Deleting pid file"
    File.delete(path) if File.exists?(path)
  end




Cheers, Olly


---
FreeAgent: Streamlined Money Management for UK Freelancers, Contractors and
Consultants
www.freeagentcentral.co.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080103/5492d04a/attachment.html 

From gethemant at gmail.com  Thu Jan  3 10:48:38 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 3 Jan 2008 21:18:38 +0530
Subject: [Backgroundrb-devel] lost connection 2.0
In-Reply-To: <477CEFB0.5010703@yelloyello.com>
References: <4774AC6F.3030302@yelloyello.com>
	<fb283d5e0712280022h68b9380ancdb91f2a86842bea@mail.gmail.com>
	<4774B3E1.4010908@yelloyello.com>
	<fb283d5e0712280033ga167e64mb0f25175cde00c6f@mail.gmail.com>
	<4774E426.4030901@yelloyello.com>
	<fb283d5e0712290809x3824fc3en3420272f88126f98@mail.gmail.com>
	<4776B42B.4020901@yelloyello.com>
	<fb283d5e0712291832n43084814m6162c15b1dab4b06@mail.gmail.com>
	<477CEFB0.5010703@yelloyello.com>
Message-ID: <fb283d5e0801030748o3a46e8c8te40171c2601a5ed8@mail.gmail.com>

Just sync with trunk, I have merged all the fixes and its fixed now.


On Jan 3, 2008 7:52 PM, Joost Hietbrink (YelloYello)
<joost at yelloyello.com> wrote:
>
>  The problem has occurred again... :( (with the 'thread_fix.tar.gz'
> version).
>  Should we set 'allow_concurrency' in the environment? Because we didn't do
> that yet..
>
>  Joost
>
>
>
>  hemant wrote:
>  Hi Joost,
>
> I am attaching a version of plugin that fixes issues with mysql lost
> connections. Can you try and let me know, how were the results.
>
>
> On Dec 30, 2007 2:25 AM, Joost Hietbrink <joost at yelloyello.com> wrote:
>
>
>  This is the 'main' worker we created.. The problem is still occurring.. I
> did not yet put the config things in environment.rb (like I said on:
> http://rubyforge.org/pipermail/backgroundrb-devel/2007-December/001233.html)..
> so maybe that fixes it?
>
>  Hope you can help..
>
>
>  hemant wrote:
>  Hi Joost,
>
> On Dec 28, 2007 5:25 PM, Joost Hietbrink (YelloYello)
> <joost at yelloyello.com> wrote:
>
>
>  The rails_model_worker.rb contains a module that fetches an object from the
> database.
>  We pass parameters to the worker (like classname & id (eg. Place and
> 12312)) and fetch the object again in the worker (using this module method).
>
>  The lost connection occurs when we try to fetch the object (record) again
> (in the worker..using the included module method).
>
>  The old BackgroundRb did not support sending ActiveRecord objects to the
> worker (thats why we refetch it)..does the new version??
>
>  We now also installed libmysql-ruby1.8 (see:
> http://wiki.rubyonrails.org/rails/pages/Mysql%2BConnection%2BProblems) so
> maybe this fixes the problem..but we have to wait some time to see.. :)
>
>
>  Sorry for late reply. But If you can send me the worker code, I will
> try to fix it.
> There seems to be some problems with AR in threaded environment. I
> just want to make sure, none remain unfixed.
>
>
>
>
> module RailsModelWorker
> # Methods to be implemented in subclass
>
>  # execute_job should implement the work you want to be done, it is called
> with the instance of the
>  # rails model object, a boolean stating whether more jobs are already
> waiting
>  # (which can be used to postpone some general work to the last job in the
> queue),
>  # and a hash containing any extra arguments (which is validated to be a
> hash before the call)
>  # obj is not nil when entering this call
> # def execute_job(obj, pending_jobs, args)
> # raise NotImplementedError, "execute_job is an abstract method"
> # end
>
>  # parse_args can be optionally implemented, use it to set @var's which you
> use in execute_job
>  # it should accept an hash of arguments (which is validated to be a hash
> before the call)
> # def parse_args(args)
> # raise NotImplementedError, "parse_args is an abstract method"
> # end
>
> # General methods
>
>  def create(args)
>  log("new worker created")
>  register_status(:booting)
>  @job_queue = Array.new
>
>  if args.nil?
>  errlog("weird: args argument of do_work is nil!")
>  elsif !args.is_a?(Hash)
>  errlog("weird: args argument of do_work is not a Hash")
>  elsif !args.has_key?(:record_type)
>  errlog("weird: args argument of do_work does not contain
> :record_type")
>  else
>  @record_type = args[:record_type]
>  end
>
>  if args.has_key?(:args) && args[:args].is_a?(Hash)
>  parse_args(args[:args]) if defined?(parse_args)
>  end
>
>  # This method is called in it's own new thread when you
>  # call new worker. args is set to :args
>  register_status(:running)
>  @timer = add_periodic_timer(5) do
>  if @job_queue.nil? || !@job_queue.is_a?(Array)
>  register_status(:dead)
>  cancel_timer(@timer) if !@timer.nil?
>  errlog("weird: our job_queue object is nil or not Array
> #{@job_queue}")
>  else
>  start_work() if @job_queue.size > 0
>  end
>  end
>  end
>
>  def request_work(params)
>  log("work requested")
>  id = params[:id]
>  args = params[:args]
>  if id.blank?
>  errlog("weird: id argument of request_indexing is nil")
>  elsif @job_queue.nil? || !@job_queue.is_a?(Array)
>  errlog("weird: our job_queue object is nil or not Array
> #{@job_queue}")
>  elsif args.nil? || !args.is_a?(Hash)
>  errlog("weird: our args argument is nil or not Hash #{args}")
>  else
>  @job_queue.each do |job|
>  job[:pending_jobs] = true
>  end
>  @request_time = Time.now
>  @job_queue << { :object_id => id, :request_time => @request_time,
> :args => args}
> # TODO: map this to the new backgroundrb
> # results[:last_request_at] = @request_time
>  end
>  end
>
>  def start_work()
>  result = false
>  begin
>  if @job_queue.nil? || !@job_queue.is_a?(Array)
>  raise "weird: our job_queue object is nil or not Array
> #{@job_queue}"
>  end
>  job = @job_queue.shift
>  if job.nil? || !job.is_a?(Hash)
>  raise "weird: our current job is not a Hash or nil: #{job}"
>  elsif job[:object_id].nil?
>  raise "weird: we got a nil object_id"
>  elsif job[:args].nil?
>  raise "weird: we got a nil args"
>  elsif @record_type.nil?
>  raise "weird: we can't do work because @record_type is nil"
>  end
>  log("Fetch rails object..")
>  for i in 1..10 do
>  obj = refetch_record(job[:object_id])
>  if obj.nil?
>  raise "Failed to retrieve object" if i == 10
>  sleep(1)
>  next
>  end
>  break
>  end
>  log("Start work..")
>  execute_job(obj, job[:pending_jobs], job[:args]) if
> defined?(execute_job)
>  log("Work done..")
>  result = true
>  rescue StandardError => e
>  errlog("Work failed.. Exception raised: #{e}")
>  #TODO: email about any exception thrown by execute_job. Something like
> this but with correct stubs
>  # if RAILS_ENV == 'production'
>  # request_stub = ActionController::AbstractRequest.new
>  # request_stub.instance_eval('@env={}')
>  # controller_stub = ApplicationController.new
>  # ExceptionNotifier.deliver_exception_notification(exception,
> controller_stub, request_stub, {})
>  # end
>  end
>  # Sets the global results hash to contain last result for this index
>  # Access it using: worker.results.to_hash[:success]
> # TODO: map this to the new backgroundrb
> # results[:last_success_at] = job[:request_time] if result
> # results[:success] = result
>  return result
>  end
>
>  def refetch_record(id)
>  result = nil
>  begin
>  result = @record_type.constantize.find(id)
>  rescue NameError => e
>  raise "Cannot get object of non existing model #{@record_type},
> exception #{e.to_s}"
>  end
>  return result
>  end
>
>  # Write to log/backgroundrb.log with class-name as prefix
>  def log(s, type='info')
>  logger.info("#{Time.now.strftime("%Y%m%d-%H:%M:%S")} (#{type})
> #{self.class}: #{s}")
>  end
>  def errlog(s)
>  log(s, 'error')
>  end
>  def debuglog(s)
>  log(s, 'debug')
>  end
>  def warnlog(s)
>  log(s, 'warning')
>  end
> end
>
>
>
>
>
>
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From voodoorai2000 at gmail.com  Thu Jan  3 11:04:37 2008
From: voodoorai2000 at gmail.com (Raimond Garcia)
Date: Thu, 3 Jan 2008 17:04:37 +0100
Subject: [Backgroundrb-devel] backgroundrb and rspec Stories
Message-ID: <29884B1F-1C5F-4257-9894-6C661672DAA0@gmail.com>

Hi,

I would like to use backgroundrb inside a story, been trying to set  
this up for a while, but I think the only solution is to use fixture  
files and unit testing.  The problem is that every time I call a new  
worker, the test database gets wiped out, and only has available the  
records in fixture files. Instead, it would be great if the test  
database maintains its state throughout the rspec Story and so can use  
and update the records of previous steps.

Thoughts?


Thank you


Raimond Garcia

From jason.lapier at gmail.com  Thu Jan  3 11:29:51 2008
From: jason.lapier at gmail.com (Jason LaPier)
Date: Thu, 3 Jan 2008 08:29:51 -0800
Subject: [Backgroundrb-devel] Start script not working when run via Rake
In-Reply-To: <f317a9f40801030703p49091be9x91f7f5f3a1b0161e@mail.gmail.com>
References: <f317a9f40801030703p49091be9x91f7f5f3a1b0161e@mail.gmail.com>
Message-ID: <93b44c1f0801030829nada26e4n270ad9c70c019146@mail.gmail.com>

On Jan 3, 2008 7:03 AM, Olly Lylo <list at lylo.co.uk> wrote:
> Hi there
>
> I'm using the latest release (r300) and 'script/backgroundrb start' works
> perfectly when executed on the server, but when put inside a Rake task, it
> doesn't launch the worker processes.
>
> The Rake task is as follows:
>
> task :start_backgroundrb do
>   run "cd #{current_path} && ./script/backgroundrb start"
> end
>

I don't know much about rake, but I think you want to use 'sh' instead of 'run'.
http://rake.rubyforge.org/classes/FileUtils.html#M000018

- Jason L.

-- 
My Rails and Linux Blog: http://offtheline.net

From gobigdave at gmail.com  Thu Jan  3 12:13:11 2008
From: gobigdave at gmail.com (Dave Dupre)
Date: Thu, 3 Jan 2008 12:13:11 -0500
Subject: [Backgroundrb-devel] Strange problem
Message-ID: <258cc3f50801030913n86dfe90ha952e8d05888b6c7@mail.gmail.com>

I use backgroundrb for many long tasks in my system, but I'm having issues
with one in particular.  Two large tasks for me are importing people and
updating companies.

  def import_contacts(args = nil)
    thread_pool.defer(args) do |job_id|
      begin
        job = ImportJob.find(job_id)
        job.process_job
      rescue => err
        logger.error "MscWorker#import_contacts failed! #{err.class}:
#{err}"
      end
    end
  end

  def update_company_from_vendor(args = nil)
    thread_pool.defer(args) do |company_id|
      begin
        company = Company.find(company_id)
        info = company.firm_info_from_vendor  # webservice call to vendor
        if info && info.size == 1
          company.update_from_vendor!(
Company.find_firm_info_details_from_vendor(info[0]))  # webservice call to
vendor
        end
      rescue => err
        logger.error "MscWorker#update_company_from_vendor failed! #{
err.class}: #{err}"
      end
    end
  end

Part of import_contacts will result in many ask_work calls to
update_company_from_vendor while it is processing.  Importing contacts is
heavily db dependent, but not very code intensive.  If I upload two files
with > 1000 contacts each (two ask_work calls to import_contacts), things
will progress along and then pause for 20-40 seconds.  There is no DB
activity during the pause, but the backgroundrb process is using most of CPU
(98-99%).  There are no deadlock errors when things startup again, but it
really slows things down.  Are you using polling somewhere?

Also, on my Mac, Activity Monitor is only showing 1 thread and 1.2 Gig(!!)
of memory used.  I expected to see many threads due to my use of
thread_pool.

Thoughts?

-- 
Dave Dupre
(m) 617-921-1684
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080103/ca4cb781/attachment.html 

From list at lylo.co.uk  Thu Jan  3 12:23:58 2008
From: list at lylo.co.uk (Olly Lylo)
Date: Thu, 3 Jan 2008 17:23:58 +0000
Subject: [Backgroundrb-devel] Start script not working when run via Rake
In-Reply-To: <93b44c1f0801030829nada26e4n270ad9c70c019146@mail.gmail.com>
References: <f317a9f40801030703p49091be9x91f7f5f3a1b0161e@mail.gmail.com>
	<93b44c1f0801030829nada26e4n270ad9c70c019146@mail.gmail.com>
Message-ID: <f317a9f40801030923n23dfd16cn71abd489fe9d1e6f@mail.gmail.com>

Sorry, I'm saying Rake when actually I mean Capistrano (1.4.1).  The tasks I
mention are Capistrano tasks and run is a Cap method:
http://manuals.rubyonrails.com/read/chapter/104#page290

On 1/3/08, Jason LaPier <jason.lapier at gmail.com> wrote:
>
> I don't know much about rake, but I think you want to use 'sh' instead of
> 'run'.
> http://rake.rubyforge.org/classes/FileUtils.html#M000018
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080103/b1890413/attachment.html 

From zach at plugthegap.co.uk  Thu Jan  3 14:17:47 2008
From: zach at plugthegap.co.uk (Zachary Powell)
Date: Thu, 3 Jan 2008 14:17:47 -0500
Subject: [Backgroundrb-devel] Delete a busy worker
Message-ID: <ac243e420801031117t737affd7gd3cc17fc529f081c@mail.gmail.com>

I thought I could delete a busy worker by calling MiddleMan.delete_worker,
but doesn't seem to end. I believe this was possible in 0.2, is it not in
1.0?
thanks,
Zach
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080103/0c701f29/attachment.html 

From voodoorai2000 at gmail.com  Thu Jan  3 14:36:45 2008
From: voodoorai2000 at gmail.com (Raimond Garcia)
Date: Thu, 3 Jan 2008 20:36:45 +0100
Subject: [Backgroundrb-devel] Delete a busy worker
In-Reply-To: <ac243e420801031117t737affd7gd3cc17fc529f081c@mail.gmail.com>
References: <ac243e420801031117t737affd7gd3cc17fc529f081c@mail.gmail.com>
Message-ID: <7CC5ABD6-E14E-4C06-9A62-F87C7FAAC975@gmail.com>

Hi, I think there has been some changes in the arguments, the  
following is working fine for me.

MiddleMan.delete_worker(:worker => :example_worker, :job_key  
=> :that_long_process)

Rai

On Jan 3, 2008, at 8:17 PM, Zachary Powell wrote:

> I thought I could delete a busy worker by calling  
> MiddleMan.delete_worker, but doesn't seem to end. I believe this was  
> possible in 0.2, is it not in 1.0?
>
> thanks,
> Zach
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel


From zach at plugthegap.co.uk  Thu Jan  3 15:03:29 2008
From: zach at plugthegap.co.uk (Zachary Powell)
Date: Thu, 3 Jan 2008 15:03:29 -0500
Subject: [Backgroundrb-devel] Delete a busy worker
In-Reply-To: <7CC5ABD6-E14E-4C06-9A62-F87C7FAAC975@gmail.com>
References: <ac243e420801031117t737affd7gd3cc17fc529f081c@mail.gmail.com>
	<7CC5ABD6-E14E-4C06-9A62-F87C7FAAC975@gmail.com>
Message-ID: <ac243e420801031203t27ae087bq6a0335fe5a1bd456@mail.gmail.com>

This worker doesn't have a key, so that makes no difference... I've tried
making a test method which just does sleep(60) and I can't kill that either
(with or without key).. Its strange, I thought it worked when I first
migrated over, maybe it doesn't work since 299 (I noticed it now saves
statuses after the worker is deleted, so it could be related to that perhaps
- if it is a bug).

On 1/3/08, Raimond Garcia <voodoorai2000 at gmail.com> wrote:
>
> Hi, I think there has been some changes in the arguments, the
> following is working fine for me.
>
> MiddleMan.delete_worker(:worker => :example_worker, :job_key
> => :that_long_process)
>
> Rai
>
> On Jan 3, 2008, at 8:17 PM, Zachary Powell wrote:
>
> > I thought I could delete a busy worker by calling
> > MiddleMan.delete_worker, but doesn't seem to end. I believe this was
> > possible in 0.2, is it not in 1.0?
> >
> > thanks,
> > Zach
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080103/00b3e7aa/attachment.html 

From mike at csa.net  Thu Jan  3 14:57:06 2008
From: mike at csa.net (Mike Dalessio)
Date: Thu, 3 Jan 2008 14:57:06 -0500
Subject: [Backgroundrb-devel] daemon use-case questions
Message-ID: <618c07250801031157g3ee63a3hc446a335d6b2deb@mail.gmail.com>

Hi all,

I'm trying to replace an OpenWFEru/daemons-based infrastructure, with the
goal being to be able to communicate with the workers to a) monitor current
state and to b) initiate new actions.

The short version on why these are daemons and not transient workers (that
start up and shut down when they're done with a particular task) is because
they're subscribing to an external pub/sub infrastructure, and are
aggregating data or monitoring the data around the clock.

I've been struggling for a few days now, and I'm hoping someone can point me
in the right direction, or else tell me that Backgroundrb isn't the right
tool for the job. My set of requirements that I'm having trouble with are
pretty straightforward, so I would think maybe someone else has run into
them. Here goes ...

1) I want to have separate log files for each daemon.

I've managed to accomplish this by setting the MetaWorker's @logger to an
instance of a proper Logger. I've also redirected STDOUT/STDERR by using
reopen(). Pow!

I'm worried, though -- isn't this going to break something? I haven't seen
anything that's obviously broken, but I'm only two days into the effort, and
only ankle-deep in the code. I need enlightenment.

2) I want to know if it dies, so that I can restart it and send myself an
email.

With OpenWFEru I have a wrapper script which invokes the daemon via system()
and watches for unsuccessful exit codes. Is there an equivalent
best-practice for this in the BDRB universe?

3) I want to be able to shut down cleanly

Workers don't seem to be given the opportunity to shut down cleanly. I've
tried defining a SIGTERM handler, as well as redefining the terminate_me()
and shutdown() methods, but I can't get this to work. Anybody have any
guidance here?

4) I want to be able to monitor current status

After the pattern of Linux daemons, I'd like to be able to run
"./script/backgroundrb status" and get my status back. I can submit a patch
for this if anyone else is interested.

However, I noticed while writing that patch, that query_all_workers() seems
to be broken in the trunk r302 (status for all workers is always nil). So my
workaround for now is to use ask_status():

     t_response = MiddleMan.query_all_workers
     t_response.each {|k,v| puts "#{k}: #{MiddleMan.ask_status(:worker =>
k)}" }


Anyhoo, that's it. That really rambled on longer than I thought it would.
Please let me know if I can clarify any of my questions, and thanks to
everyone for their patience with my long-windedness.

Thanks,
-mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080103/b7e6df2c/attachment-0001.html 

From gobigdave at gmail.com  Thu Jan  3 15:10:10 2008
From: gobigdave at gmail.com (Dave Dupre)
Date: Thu, 3 Jan 2008 15:10:10 -0500
Subject: [Backgroundrb-devel] Memory leak and long process problem
Message-ID: <258cc3f50801031210r723f28denb288a41e0d6ccc03@mail.gmail.com>

I use backgroundrb for many long tasks in my system, but I'm having issues
with one in particular.  Two large tasks for me are importing people and
updating companies.

  def import_contacts(args = nil)
    thread_pool.defer(args) do |job_id|
      begin
        job = ImportJob.find(job_id)
        job.process_job
      rescue => err
        logger.error "MscWorker#import_contacts failed! #{err.class}:
#{err}"
      end
    end
  end

  def update_company_from_vendor(args = nil)
    thread_pool.defer(args) do |company_id|
      begin
        company = Company.find(company_id)
        info = company.firm_info_from_vendor  # webservice call to vendor
        if info && info.size == 1
          company.update_from_vendor!(Company.find_firm_info_details_from_vendor(info[0]))
# webservice call to vendor
        end
      rescue => err
        logger.error "MscWorker#update_company_from_vendor failed! #{
err.class}: #{err}"
      end
    end
  end

Part of import_contacts will result in many ask_work calls to
update_company_from_vendor while it is processing.  Importing contacts is
heavily db dependent, but not very code intensive.  If I upload two files
with > 1000 contacts each (two ask_work calls to import_contacts), things
will progress along and then pause for 20-40 seconds.  There is no DB
activity during the pause, but the backgroundrb process is using most of CPU
(98-99%).  There are no deadlock errors when things startup again, but it
really slows things down.  Are you using polling somewhere?

Also, on my Mac, Activity Monitor is only showing 1 thread and 1.2 Gig(!!)
of memory used.  I expected to see many threads due to my use of
thread_pool.

Since all of my processing code is in models, it is very easy to switch to
synchronous execution.  When I execute job.process_job (see
import_contacts), things never pause, and the ruby process never gets over
120Meg in size.

This all leaves me with two questions:

1. Sure looks like there is a serious memory leak someplace, but I don't
think it is my code.

2. What is the recommended method for this processing.  Currently, I have a
single worker for my web app to call for background tasks -- each task is
implemented as a thread pool.  I don't have much need for status since I can
get the status from the database.  Should I change things to dynamically
create workers?

3. I should repeat that I never saw multiple threads being created even
though update_company_from_vendor was called 1500 times during one call to
import_contacts.  update_company_from_vendor takes several seconds to
execute so I know calls should have queued up.

Thoughts?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080103/912d7699/attachment.html 

From voodoorai2000 at gmail.com  Thu Jan  3 15:10:45 2008
From: voodoorai2000 at gmail.com (Raimond Garcia)
Date: Thu, 3 Jan 2008 21:10:45 +0100
Subject: [Backgroundrb-devel] Delete a busy worker
In-Reply-To: <ac243e420801031203t27ae087bq6a0335fe5a1bd456@mail.gmail.com>
References: <ac243e420801031117t737affd7gd3cc17fc529f081c@mail.gmail.com>
	<7CC5ABD6-E14E-4C06-9A62-F87C7FAAC975@gmail.com>
	<ac243e420801031203t27ae087bq6a0335fe5a1bd456@mail.gmail.com>
Message-ID: <C71AE60D-4ABB-456B-9301-59A0B3287059@gmail.com>

Hmmm, check out the examples folder inside vendor/plugins/ 
backgroundrb, that helped me get everything up and running with the  
new version.

Rai

On Jan 3, 2008, at 9:03 PM, Zachary Powell wrote:

> This worker doesn't have a key, so that makes no difference... I've  
> tried making a test method which just does sleep(60) and I can't  
> kill that either (with or without key).. Its strange, I thought it  
> worked when I first migrated over, maybe it doesn't work since 299  
> (I noticed it now saves statuses after the worker is deleted, so it  
> could be related to that perhaps - if it is a bug).
>
> On 1/3/08, Raimond Garcia <voodoorai2000 at gmail.com> wrote:
> Hi, I think there has been some changes in the arguments, the
> following is working fine for me.
>
> MiddleMan.delete_worker(:worker => :example_worker, :job_key
> => :that_long_process)
>
> Rai
>
> On Jan 3, 2008, at 8:17 PM, Zachary Powell wrote:
>
> > I thought I could delete a busy worker by calling
> > MiddleMan.delete_worker, but doesn't seem to end. I believe this was
> > possible in 0.2, is it not in 1.0?
> >
> > thanks,
> > Zach
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080103/c525cc7f/attachment.html 

From walter at katipo.co.nz  Thu Jan  3 15:40:34 2008
From: walter at katipo.co.nz (Walter McGinnis)
Date: Fri, 4 Jan 2008 09:40:34 +1300
Subject: [Backgroundrb-devel] Delete a busy worker
In-Reply-To: <C71AE60D-4ABB-456B-9301-59A0B3287059@gmail.com>
References: <ac243e420801031117t737affd7gd3cc17fc529f081c@mail.gmail.com>
	<7CC5ABD6-E14E-4C06-9A62-F87C7FAAC975@gmail.com>
	<ac243e420801031203t27ae087bq6a0335fe5a1bd456@mail.gmail.com>
	<C71AE60D-4ABB-456B-9301-59A0B3287059@gmail.com>
Message-ID: <F827741C-89B0-4B6F-BA59-D6138B0E7309@katipo.co.nz>

I'm experiencing this, too.  I can delete a worker after it's finished  
it's finished executing a particular ask_work with :worker_method  
call, but I can't "abort" the worker early if I want to interrupt  
execution of ask_work.

I do call delete_worker with both the :worker and :job_key specified.

So I second Zach's problem.

Cheers,
Walter

On Jan 4, 2008, at 9:10 AM, Raimond Garcia wrote:

> Hmmm, check out the examples folder inside vendor/plugins/ 
> backgroundrb, that helped me get everything up and running with the  
> new version.
>
> Rai
>
> On Jan 3, 2008, at 9:03 PM, Zachary Powell wrote:
>
>> This worker doesn't have a key, so that makes no difference... I've  
>> tried making a test method which just does sleep(60) and I can't  
>> kill that either (with or without key).. Its strange, I thought it  
>> worked when I first migrated over, maybe it doesn't work since 299  
>> (I noticed it now saves statuses after the worker is deleted, so it  
>> could be related to that perhaps - if it is a bug).
>>
>> On 1/3/08, Raimond Garcia <voodoorai2000 at gmail.com> wrote:
>> Hi, I think there has been some changes in the arguments, the
>> following is working fine for me.
>>
>> MiddleMan.delete_worker(:worker => :example_worker, :job_key
>> => :that_long_process)
>>
>> Rai
>>
>> On Jan 3, 2008, at 8:17 PM, Zachary Powell wrote:
>>
>> > I thought I could delete a busy worker by calling
>> > MiddleMan.delete_worker, but doesn't seem to end. I believe this  
>> was
>> > possible in 0.2, is it not in 1.0?
>> >
>> > thanks,
>> > Zach
>> > _______________________________________________
>> > Backgroundrb-devel mailing list
>> > Backgroundrb-devel at rubyforge.org
>> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>
>> _______________________________________________
>> Backgroundrb-devel mailing list
>> Backgroundrb-devel at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080104/8bdd1901/attachment.html 

From gethemant at gmail.com  Thu Jan  3 16:32:49 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 4 Jan 2008 03:02:49 +0530
Subject: [Backgroundrb-devel] Memory leak and long process problem
In-Reply-To: <258cc3f50801031210r723f28denb288a41e0d6ccc03@mail.gmail.com>
References: <258cc3f50801031210r723f28denb288a41e0d6ccc03@mail.gmail.com>
Message-ID: <fb283d5e0801031332x7d911d00h45b4214f86a570c4@mail.gmail.com>

Hi Dave,

On Jan 4, 2008 1:40 AM, Dave Dupre <gobigdave at gmail.com> wrote:
> I use backgroundrb for many long tasks in my system, but I'm having issues
> with one in particular.  Two large tasks for me are importing people and
> updating companies.
>

So method below is invoked from rails using ask_work command, right?
Are you by any chance, passing uploaded  file itself to worker? If
yes, preferably don't do that. Save the file somewhere or in db and
pas the location.

And why do you need, thread_pool there? Do you want concurrent
execution of tasks, or you just want a worker queue?

>   def import_contacts(args = nil)
>     thread_pool.defer(args) do |job_id|
>       begin
>         job = ImportJob.find(job_id)
>         job.process_job
>       rescue => err
>         logger.error "MscWorker#import_contacts failed! #{err.class}:
> #{err}"
>       end
>     end
>   end
>

Similar doubts as previous worker method.

>   def update_company_from_vendor(args = nil)
>     thread_pool.defer(args) do |company_id|
>       begin
>         company = Company.find(company_id)
>         info = company.firm_info_from_vendor  # webservice call to vendor
>         if info && info.size == 1
>           company.update_from_vendor!(Company.find_firm_info_details
> _from_vendor(info[0]))  # webservice call to vendor
>         end
>       rescue => err
>         logger.error "MscWorker#update_company_from_vendor failed!
> #{err.class}: #{err}"
>       end
>     end
>   end
>
> Part of import_contacts will result in many ask_work calls to
> update_company_from_vendor while it is processing.  Importing contacts is
> heavily db dependent, but not very code intensive.  If I upload two files
> with > 1000 contacts each (two ask_work calls to import_contacts), things
> will progress along and then pause for 20-40 seconds.  There is no DB
> activity during the pause, but the backgroundrb process is using most of CPU
> (98-99%).  There are no deadlock errors when things startup again, but it
> really slows things down.  Are you using polling somewhere?
>
> Also, on my Mac, Activity Monitor is only showing 1 thread and 1.2 Gig(!!)
> of memory used.  I expected to see many threads due to my use of
> thread_pool.
>
> Since all of my processing code is in models, it is very easy to switch to
> synchronous execution.  When I execute job.process_job (see
> import_contacts), things never pause, and the ruby process never gets over
> 120Meg in size.
>
> This all leaves me with two questions:
>
> 1. Sure looks like there is a serious memory leak someplace, but I don't
> think it is my code.
>
> 2. What is the recommended method for this processing.  Currently, I have a
> single worker for my web app to call for background tasks -- each task is
> implemented as a thread pool.  I don't have much need for status since I can
> get the status from the database.  Should I change things to dynamically
> create workers?
>
> 3. I should repeat that I never saw multiple threads being created even
> though update_company_from_vendor was called 1500 times during one call to
> import_contacts.  update_company_from_vendor takes several seconds to
> execute so I know calls should have queued up.

Ruby uses green threads, so I don't think Activity Monitor will show
multiple created threads.Also, thread pool reaches its pool size
depending upon number of tasks in the queue. If queue is empty, only
one thread will be actually created initially.




-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Thu Jan  3 16:37:30 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 4 Jan 2008 03:07:30 +0530
Subject: [Backgroundrb-devel] Delete a busy worker
In-Reply-To: <F827741C-89B0-4B6F-BA59-D6138B0E7309@katipo.co.nz>
References: <ac243e420801031117t737affd7gd3cc17fc529f081c@mail.gmail.com>
	<7CC5ABD6-E14E-4C06-9A62-F87C7FAAC975@gmail.com>
	<ac243e420801031203t27ae087bq6a0335fe5a1bd456@mail.gmail.com>
	<C71AE60D-4ABB-456B-9301-59A0B3287059@gmail.com>
	<F827741C-89B0-4B6F-BA59-D6138B0E7309@katipo.co.nz>
Message-ID: <fb283d5e0801031337h5ba47237w41095920f6355872@mail.gmail.com>

On Jan 4, 2008 2:10 AM, Walter McGinnis <walter at katipo.co.nz> wrote:
> I'm experiencing this, too.  I can delete a worker after it's finished it's
> finished executing a particular ask_work with :worker_method call, but I
> can't "abort" the worker early if I want to interrupt execution of ask_work.
>
> I do call delete_worker with both the :worker and :job_key specified.
>
> So I second Zach's problem.
>

When you invoke MiddleMan.delete_worker(), you are actually calling
"exit" from within the worker. Thats the current implementation, and
by default your requests are queued and hence "exit" is not invoked
until, whatever worker is doing is finished. And hence the behavior.
But this can be changed and master can explicitly terminate a worker,
since it has process id of each worker, so will that be a more
desirable behavior?



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gobigdave at gmail.com  Thu Jan  3 16:40:55 2008
From: gobigdave at gmail.com (Dave Dupre)
Date: Thu, 3 Jan 2008 16:40:55 -0500
Subject: [Backgroundrb-devel] Thread_pool bug?
Message-ID: <258cc3f50801031340p72dfb9f5nfdce11b9493eafa0@mail.gmail.com>

I have a previous question regarding long tasks and the thread_pool (sorry
for the dup, I didn't see the first go through).  To try and track things
down, I made a change based on a suggestion found in the archives.  I moved
my import contacts worker to its own file and set the pool_size to 1.

class ImportContactsWorker < BackgrounDRb::MetaWorker
  set_worker_name :import_contacts_worker
  pool_size(1)
  def create(args = nil)
    # Restart any import jobs that didn't complete or start
    ImportJob.process_all_ready
  end
  def import_contacts(args = nil)
    thread_pool.defer(args) do |job_id|
      begin
        job = ImportJob.find(job_id)
        job.process_job
      rescue => err
        logger.error "ImportContactsWorker(#{job_id}) failed! #{err.class}:
#{err}"
      end
    end
  end
end

I started one long import job no problem. Then, I started another one, and I
expected that the thread_pool would queue it up and execute it when the
thread became available.  At least, that's what this archive says. However,
my call to ask_work blocked until the first call completed.  My ask_work
call:

MiddleMan.ask_work(:worker => :import_contacts_worker, :worker_method =>
:import_contacts, :data => job_id)

Shouldn't the thread_pool have queued to request and import_contacts
returned immediately?

These import jobs can take 10 minutes, so it's not fun for the user to hang
around and wait.

BTW, I'm still see bad memory growth with the backgroundrb tasks.  Run the
same methods outside of backgroundrb, and memory does not grow at all.

As it stands now, I need to come up with another plan or go back to the old
version of backgroundrb.

Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080103/e4880471/attachment.html 

From walter at katipo.co.nz  Thu Jan  3 16:50:16 2008
From: walter at katipo.co.nz (Walter McGinnis)
Date: Fri, 4 Jan 2008 10:50:16 +1300
Subject: [Backgroundrb-devel] Delete a busy worker
In-Reply-To: <fb283d5e0801031337h5ba47237w41095920f6355872@mail.gmail.com>
References: <ac243e420801031117t737affd7gd3cc17fc529f081c@mail.gmail.com>
	<7CC5ABD6-E14E-4C06-9A62-F87C7FAAC975@gmail.com>
	<ac243e420801031203t27ae087bq6a0335fe5a1bd456@mail.gmail.com>
	<C71AE60D-4ABB-456B-9301-59A0B3287059@gmail.com>
	<F827741C-89B0-4B6F-BA59-D6138B0E7309@katipo.co.nz>
	<fb283d5e0801031337h5ba47237w41095920f6355872@mail.gmail.com>
Message-ID: <57BCB8CD-58DB-439A-8D73-B570F5CD58A9@katipo.co.nz>


On Jan 4, 2008, at 10:37 AM, hemant wrote:

> On Jan 4, 2008 2:10 AM, Walter McGinnis <walter at katipo.co.nz> wrote:
>> I'm experiencing this, too.  I can delete a worker after it's  
>> finished it's
>> finished executing a particular ask_work with :worker_method call,  
>> but I
>> can't "abort" the worker early if I want to interrupt execution of  
>> ask_work.
>>
>> I do call delete_worker with both the :worker and :job_key specified.
>>
>> So I second Zach's problem.
>>
>
> When you invoke MiddleMan.delete_worker(), you are actually calling
> "exit" from within the worker. Thats the current implementation, and
> by default your requests are queued and hence "exit" is not invoked
> until, whatever worker is doing is finished. And hence the behavior.
> But this can be changed and master can explicitly terminate a worker,
> since it has process id of each worker, so will that be a more
> desirable behavior?
>

For me it would be, yes.  I can't speak for others.

Cheers,
Walter

From gethemant at gmail.com  Thu Jan  3 16:55:33 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 4 Jan 2008 03:25:33 +0530
Subject: [Backgroundrb-devel] Start script not working when run via Rake
In-Reply-To: <f317a9f40801030703p49091be9x91f7f5f3a1b0161e@mail.gmail.com>
References: <f317a9f40801030703p49091be9x91f7f5f3a1b0161e@mail.gmail.com>
Message-ID: <fb283d5e0801031355t7b8446cdk955afe73619b6de6@mail.gmail.com>

On Jan 3, 2008 8:33 PM, Olly Lylo <list at lylo.co.uk> wrote:
> Hi there
>
> I'm using the latest release (r300) and 'script/backgroundrb start' works
> perfectly when executed on the server, but when put inside a Rake task, it
> doesn't launch the worker processes.
>
> The Rake task is as follows:
>
> task :start_backgroundrb do
>   run "cd #{current_path} && ./script/backgroundrb start"
> end
>
> When I run this task, the following files are created in the log directory:
>
> backgroundrb.pid
> backgroundrb_debug.log
> backgroundrb_server.log
>
>
> Observations: The process ID found in backgroundrb.pid doesn't exist,
> backgroundrb_server.log is zero bytes long and backgroundrb_debug.log
> contains the following:
>
> # Logfile created on Thu Jan 03 14:43:24 +0000 2008 by /
>
>
> It looks like the process is being started by Rake, but terminates before
> the worker processes are started.   Could anyone shed any light on why this
> might be happening and how I can resolve the issue?
>
> I have also noticed that calling './script/backgroundrb stop' when
> BackgroundRB isn't running throws an exception, which causes problems when
> executing 'backgroundrb stop' in a rake task.  As a workaround I have
> modified by 'backgroundrb' script as follows:
>
> when 'stop'
>   path = "#{RAILS_HOME}/log/backgroundrb.pid"
>   pid = nil
>   begin
>     File.open(path, "r") { |pid_handle| pid =
> pid_handle.gets.strip.chomp.to_i }
>     pgid =  Process.getpgid(pid)
>     Process.kill('TERM', pid)
>     Process.kill('-TERM', pgid)
>     Process.kill('KILL', pid)
>   rescue
>     puts $!
>   ensure
>     puts "Deleting pid file"
>     File.delete(path) if File.exists?(path)
>   end
>

Okay, I have imported changes in stop part of script. Again, are you
able to start backgroundrb from command prompt? Is it that, just cap
tasks fails?

Try printing value of RAILS_HOME and see if its correct.

-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Thu Jan  3 17:00:39 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 4 Jan 2008 03:30:39 +0530
Subject: [Backgroundrb-devel] Thread_pool bug?
In-Reply-To: <258cc3f50801031340p72dfb9f5nfdce11b9493eafa0@mail.gmail.com>
References: <258cc3f50801031340p72dfb9f5nfdce11b9493eafa0@mail.gmail.com>
Message-ID: <fb283d5e0801031400u36aea1d2i921435eb0a89f965@mail.gmail.com>

On Jan 4, 2008 3:10 AM, Dave Dupre <gobigdave at gmail.com> wrote:
> I have a previous question regarding long tasks and the thread_pool (sorry
> for the dup, I didn't see the first go through).  To try and track things
> down, I made a change based on a suggestion found in the archives.  I moved
> my import contacts worker to its own file and set the pool_size to 1.
>
> class ImportContactsWorker < BackgrounDRb::MetaWorker
>   set_worker_name :import_contacts_worker
>   pool_size(1)
>   def create(args = nil)
>     # Restart any import jobs that didn't complete or start
>     ImportJob.process_all_ready
>   end
>   def import_contacts(args = nil)
>     thread_pool.defer(args) do |job_id|
>       begin
>         job = ImportJob.find(job_id)
>         job.process_job
>       rescue => err
>         logger.error "ImportContactsWorker(#{job_id}) failed! #{err.class}:
> #{err}"
>       end
>     end
>   end
>  end
>
> I started one long import job no problem. Then, I started another one, and I
> expected that the thread_pool would queue it up and execute it when the
> thread became available.  At least, that's what this archive says. However,
> my call to ask_work blocked until the first call completed.  My ask_work
> call:
>
> MiddleMan.ask_work(:worker => :import_contacts_worker, :worker_method =>
> :import_contacts, :data => job_id)
>
> Shouldn't the thread_pool have queued to request and import_contacts
> returned immediately?
>
> These import jobs can take 10 minutes, so it's not fun for the user to hang
> around and wait.
>
> BTW, I'm still see bad memory growth with the backgroundrb tasks.  Run the
> same methods outside of backgroundrb, and memory does not grow at all.
>
> As it stands now, I need to come up with another plan or go back to the old
> version of backgroundrb.

I am having a look at whole thing. Also, there is no "logger.error" so
please use "logger.info" in place. I suspect "logger.error" will throw
an exception.

Apart from above suggestion, I am having a cold hard look at thread
pool, expect it to be fixed.


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Thu Jan  3 17:01:51 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 4 Jan 2008 03:31:51 +0530
Subject: [Backgroundrb-devel] backgroundrb and rspec Stories
In-Reply-To: <29884B1F-1C5F-4257-9894-6C661672DAA0@gmail.com>
References: <29884B1F-1C5F-4257-9894-6C661672DAA0@gmail.com>
Message-ID: <fb283d5e0801031401y2e418f8n4c369f71968fbad5@mail.gmail.com>

On Jan 3, 2008 9:34 PM, Raimond Garcia <voodoorai2000 at gmail.com> wrote:
> Hi,
>
> I would like to use backgroundrb inside a story, been trying to set
> this up for a while, but I think the only solution is to use fixture
> files and unit testing.  The problem is that every time I call a new
> worker, the test database gets wiped out, and only has available the
> records in fixture files. Instead, it would be great if the test
> database maintains its state throughout the rspec Story and so can use
> and update the records of previous steps.

Or you can mock calls to database.

-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From voodoorai2000 at gmail.com  Thu Jan  3 17:38:52 2008
From: voodoorai2000 at gmail.com (Raimond Garcia)
Date: Thu, 3 Jan 2008 23:38:52 +0100
Subject: [Backgroundrb-devel] backgroundrb and rspec Stories
In-Reply-To: <fb283d5e0801031401y2e418f8n4c369f71968fbad5@mail.gmail.com>
References: <29884B1F-1C5F-4257-9894-6C661672DAA0@gmail.com>
	<fb283d5e0801031401y2e418f8n4c369f71968fbad5@mail.gmail.com>
Message-ID: <C5F0BA9B-95EF-43AC-9FC1-84A74A3372EF@gmail.com>

Yes, mocking the database calls, sounds like the best option to  
maintain the flow of the story.

I'm trying to mock it by using this:
Image.should_receive(:find).and_return(@image)
get :contoller_action_that_creates_the_worker

Now, when I'm inside the worker I use the ruby debugger to see what's  
happening, but I can still see only the fixtures, and the rspec  
mocking call seems to lose it's context and not get used.

What would you suggest to mock the database call?

Thank you!

Rai


On Jan 3, 2008, at 11:01 PM, hemant wrote:

> On Jan 3, 2008 9:34 PM, Raimond Garcia <voodoorai2000 at gmail.com>  
> wrote:
>> Hi,
>>
>> I would like to use backgroundrb inside a story, been trying to set
>> this up for a while, but I think the only solution is to use fixture
>> files and unit testing.  The problem is that every time I call a new
>> worker, the test database gets wiped out, and only has available the
>> records in fixture files. Instead, it would be great if the test
>> database maintains its state throughout the rspec Story and so can  
>> use
>> and update the records of previous steps.
>
> Or you can mock calls to database.
>
> -- 
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org


From gethemant at gmail.com  Thu Jan  3 19:22:14 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 4 Jan 2008 05:52:14 +0530
Subject: [Backgroundrb-devel] Updates and Fixes on Trunk
Message-ID: <fb283d5e0801031622g320ead66s8eec856e3e254e5c@mail.gmail.com>

Hi,

I am pushing some updates and fixes that addresses much of the issues
that we talked about since last couple of days.

1. Use  configuration option   :debug_log: false to disable
backgroundrb_debug.log.
2. As shown below, use lazy_load option to disable aggressive loading
of models ( and somewhat stupid too).
3. MiddleMan.delete_worker(:worker => :foo_worker) will now ABORT the
worker, it won't wait for tasks to finish. I am still skeptic about
the behavior,  but since that was the behavior in older version, I am
continuing it.
4. Get information about all running workers:

>> MiddleMan.all_worker_info
=> [{:worker=>:log_worker, :status=>:running, :job_key=>""},
{:worker=>:foo_worker, :status=>:running, :job_key=>""}]



---
:backgroundrb:
  :port: 11006
  :ip: localhost
  :environment: production
  :log: foreground
  :lazy_load: true
  :debug_log: false


Those who have reported any bugs in past are requested to follow up
and see if its fixed. I won't suggest to run this in production
straightaway, but do your testing first and then deploy.


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From zach at plugthegap.co.uk  Thu Jan  3 22:38:03 2008
From: zach at plugthegap.co.uk (Zachary Powell)
Date: Thu, 3 Jan 2008 22:38:03 -0500
Subject: [Backgroundrb-devel] Delete a busy worker
In-Reply-To: <57BCB8CD-58DB-439A-8D73-B570F5CD58A9@katipo.co.nz>
References: <ac243e420801031117t737affd7gd3cc17fc529f081c@mail.gmail.com>
	<7CC5ABD6-E14E-4C06-9A62-F87C7FAAC975@gmail.com>
	<ac243e420801031203t27ae087bq6a0335fe5a1bd456@mail.gmail.com>
	<C71AE60D-4ABB-456B-9301-59A0B3287059@gmail.com>
	<F827741C-89B0-4B6F-BA59-D6138B0E7309@katipo.co.nz>
	<fb283d5e0801031337h5ba47237w41095920f6355872@mail.gmail.com>
	<57BCB8CD-58DB-439A-8D73-B570F5CD58A9@katipo.co.nz>
Message-ID: <ac243e420801031938i449ce8c9lbe0779015577f9ee@mail.gmail.com>

For me it would be best (that's how the previous version worked) - though
presumably there'd be an easy way to do both. (delete_worker() or
kill_worker() etc). thanks.

On 1/3/08, Walter McGinnis <walter at katipo.co.nz> wrote:
>
>
> On Jan 4, 2008, at 10:37 AM, hemant wrote:
>
> > On Jan 4, 2008 2:10 AM, Walter McGinnis <walter at katipo.co.nz> wrote:
> >> I'm experiencing this, too.  I can delete a worker after it's
> >> finished it's
> >> finished executing a particular ask_work with :worker_method call,
> >> but I
> >> can't "abort" the worker early if I want to interrupt execution of
> >> ask_work.
> >>
> >> I do call delete_worker with both the :worker and :job_key specified.
> >>
> >> So I second Zach's problem.
> >>
> >
> > When you invoke MiddleMan.delete_worker(), you are actually calling
> > "exit" from within the worker. Thats the current implementation, and
> > by default your requests are queued and hence "exit" is not invoked
> > until, whatever worker is doing is finished. And hence the behavior.
> > But this can be changed and master can explicitly terminate a worker,
> > since it has process id of each worker, so will that be a more
> > desirable behavior?
> >
>
> For me it would be, yes.  I can't speak for others.
>
> Cheers,
> Walter
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080103/844589c5/attachment-0001.html 

From zach at plugthegap.co.uk  Thu Jan  3 22:53:29 2008
From: zach at plugthegap.co.uk (Zachary Powell)
Date: Thu, 3 Jan 2008 22:53:29 -0500
Subject: [Backgroundrb-devel] Updates and Fixes on Trunk
In-Reply-To: <fb283d5e0801031622g320ead66s8eec856e3e254e5c@mail.gmail.com>
References: <fb283d5e0801031622g320ead66s8eec856e3e254e5c@mail.gmail.com>
Message-ID: <ac243e420801031953u1ce1be07tf4aa04465ff50c4d@mail.gmail.com>

Hi Hemant,
Thanks for the quick fix. I think this works - if one wants to send a delete
that exits after a clean finish one can always do ask_work(...,
:worker_method => :exit) which would queue the exit (not sure if that would
need a defined exit method in the worker class, or if it could call the
kernel one as is, but either way it should be fine). If people want
something more explicit in the api, I recommend having two methods
(delete/kill delete/destroy etc). Thanks again.

Zach

On 1/3/08, hemant <gethemant at gmail.com> wrote:
>
> Hi,
>
> I am pushing some updates and fixes that addresses much of the issues
> that we talked about since last couple of days.
>
> 1. Use  configuration option   :debug_log: false to disable
> backgroundrb_debug.log.
> 2. As shown below, use lazy_load option to disable aggressive loading
> of models ( and somewhat stupid too).
> 3. MiddleMan.delete_worker(:worker => :foo_worker) will now ABORT the
> worker, it won't wait for tasks to finish. I am still skeptic about
> the behavior,  but since that was the behavior in older version, I am
> continuing it.
> 4. Get information about all running workers:
>
> >> MiddleMan.all_worker_info
> => [{:worker=>:log_worker, :status=>:running, :job_key=>""},
> {:worker=>:foo_worker, :status=>:running, :job_key=>""}]
>
>
>
> ---
> :backgroundrb:
>   :port: 11006
>   :ip: localhost
>   :environment: production
>   :log: foreground
>   :lazy_load: true
>   :debug_log: false
>
>
> Those who have reported any bugs in past are requested to follow up
> and see if its fixed. I won't suggest to run this in production
> straightaway, but do your testing first and then deploy.
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080103/6e9eecc7/attachment.html 

From list at lylo.co.uk  Fri Jan  4 05:07:34 2008
From: list at lylo.co.uk (Olly Lylo)
Date: Fri, 4 Jan 2008 10:07:34 +0000
Subject: [Backgroundrb-devel] Start script not working when run via Rake
In-Reply-To: <fb283d5e0801031355t7b8446cdk955afe73619b6de6@mail.gmail.com>
References: <f317a9f40801030703p49091be9x91f7f5f3a1b0161e@mail.gmail.com>
	<fb283d5e0801031355t7b8446cdk955afe73619b6de6@mail.gmail.com>
Message-ID: <f317a9f40801040207k29f57254s6d8d3d2c9b5a4ea9@mail.gmail.com>

>
> Okay, I have imported changes in stop part of script. Again, are you
> able to start backgroundrb from command prompt? Is it that, just cap
> tasks fails?



Works fine from the command prompt -- it just doesn't start when run from
the Cap task.

Try printing value of RAILS_HOME and see if its correct.
>

RAILS_HOME is correct.  I've added ':log: foreground' to my YML file and put
some logging into the backgroundrb script as detailed below, to try and
figure this out:

when 'start'
  if fork
    puts 'Exiting'
    exit
  else

    path = "#{RAILS_HOME}/log/backgroundrb.pid"
    puts "Path = #{path}"
    config_file = YAML.load(ERB.new(IO.read
("#{RAILS_HOME}/config/backgroundrb.yml")).result)
    op = File.open(path, "w")
    op.write(Process.pid().to_s)
    op.close
    if config_file[:backgroundrb][:log].nil? or
config_file[:backgroundrb][:log] != 'foreground'
      log_file = File.open(SERVER_LOGGER,"w+")
      [STDIN, STDOUT, STDERR].each {|desc| desc.reopen(log_file)}
    end

    puts "Creating proxing"
    BackgrounDRb::MasterProxy.new()
  end

puts "Done"


The output is:

Path = /var/www/apps/my_app/releases/20080103141158/log/backgroundrb.pid
Creating proxy
Exiting


This seems fine except the 'puts "Done"' statement at the end of the block
isn't being callled.   Does this give you any idea about what might be
happening?


Cheers, Olly
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080104/59bf28b2/attachment.html 

From jonathan.wallace at gmail.com  Fri Jan  4 08:15:50 2008
From: jonathan.wallace at gmail.com (Jonathan Wallace)
Date: Fri, 4 Jan 2008 08:15:50 -0500
Subject: [Backgroundrb-devel] Start script not working when run via Rake
In-Reply-To: <f317a9f40801040207k29f57254s6d8d3d2c9b5a4ea9@mail.gmail.com>
References: <f317a9f40801030703p49091be9x91f7f5f3a1b0161e@mail.gmail.com>
	<fb283d5e0801031355t7b8446cdk955afe73619b6de6@mail.gmail.com>
	<f317a9f40801040207k29f57254s6d8d3d2c9b5a4ea9@mail.gmail.com>
Message-ID: <fd0631d50801040515l7ccb1e00v9a8031bbb38281b0@mail.gmail.com>

I'd like to second this issue as well.  I haven't had time to
investigate (maybe an upgrade to the latest capistrano would fix it?)
but I have to manually stop and restart bdrb on my production server
after a deploy when we've updated the worker code.
 Jonathan

On Jan 4, 2008 5:07 AM, Olly Lylo <list at lylo.co.uk> wrote:
>
> > Okay, I have imported changes in stop part of script. Again, are you
> > able to start backgroundrb from command prompt? Is it that, just cap
> > tasks fails?
>
>
> Works fine from the command prompt -- it just doesn't start when run from
> the Cap task.
>
> > Try printing value of RAILS_HOME and see if its correct.
> >
>
> RAILS_HOME is correct.  I've added ':log: foreground' to my YML file and put
> some logging into the backgroundrb script as detailed below, to try and
> figure this out:
>
> when 'start'
>   if fork
>     puts 'Exiting'
>      exit
>   else
>     path = "#{RAILS_HOME}/log/backgroundrb.pid"
>     puts "Path = #{path}"
>     config_file =
> YAML.load(ERB.new(IO.read("#{RAILS_HOME}/config/backgroundrb.yml")).result)
>     op = File.open(path, "w")
>     op.write(Process.pid().to_s)
>     op.close
>     if config_file[:backgroundrb][:log].nil? or
> config_file[:backgroundrb][:log] != 'foreground'
>       log_file = File.open(SERVER_LOGGER,"w+")
>       [STDIN, STDOUT, STDERR].each {|desc| desc.reopen(log_file)}
>     end
>
>     puts "Creating proxing"
>     BackgrounDRb::MasterProxy.new ()
>   end
> puts "Done"
>
>
> The output is:
>
>  Path = /var/www/apps/my_app/releases/20080103141158/log/backgroundrb.pid
> Creating proxy
> Exiting
>
> This seems fine except the 'puts "Done"' statement at the end of the block
> isn't being callled.   Does this give you any idea about what might be
> happening?
>
>
> Cheers, Olly
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
Jonathan Wallace

From voodoorai2000 at gmail.com  Tue Jan  8 11:43:28 2008
From: voodoorai2000 at gmail.com (Raimond Garcia)
Date: Tue, 8 Jan 2008 17:43:28 +0100
Subject: [Backgroundrb-devel] Different environments in backgroudrb.yml
Message-ID: <892287BD-5C09-4E7D-B56C-F70F0650CDBB@gmail.com>

Hi,

Got everything setup nicely with backgroundrb for the development  
environment and running tests with rspec.  However, I have to keep  
editing the backgroundrb.yml file, to switch between development and  
testing environment.  How can I declare a development, testing and  
production environment in backgroundrb.yml?

Thanks in advance


Rai

From gethemant at gmail.com  Tue Jan  8 12:15:18 2008
From: gethemant at gmail.com (hemant kumar)
Date: Tue, 08 Jan 2008 22:45:18 +0530
Subject: [Backgroundrb-devel] Different environments in backgroudrb.yml
In-Reply-To: <892287BD-5C09-4E7D-B56C-F70F0650CDBB@gmail.com>
References: <892287BD-5C09-4E7D-B56C-F70F0650CDBB@gmail.com>
Message-ID: <1199812518.25054.9.camel@shire>

Hi
On Tue, 2008-01-08 at 17:43 +0100, Raimond Garcia wrote:
> Hi,
> 
> Got everything setup nicely with backgroundrb for the development  
> environment and running tests with rspec.  However, I have to keep  
> editing the backgroundrb.yml file, to switch between development and  
> testing environment.  How can I declare a development, testing and  
> production environment in backgroundrb.yml?
> 
> Thanks in advance

in your bdrb_test_helper.rb file put:

RAILS_ENV = 'test' at top and you are set.




From voodoorai2000 at gmail.com  Tue Jan  8 12:49:54 2008
From: voodoorai2000 at gmail.com (Raimond Garcia)
Date: Tue, 8 Jan 2008 18:49:54 +0100
Subject: [Backgroundrb-devel] Different environments in backgroudrb.yml
In-Reply-To: <1199812518.25054.9.camel@shire>
References: <892287BD-5C09-4E7D-B56C-F70F0650CDBB@gmail.com>
	<1199812518.25054.9.camel@shire>
Message-ID: <B6EE39C6-B947-4B79-8C05-AD7D0F356141@gmail.com>

Cool, what about development and production environments? should I  
just not include the environment definition in backgroundrb.yml, and  
simply start the servers using the appropriate environment each time?  
for example

RAILS_ENV=development ./script/backgroundrb start
and
RAILS_ENV=production ./script/backgroundrb start


On Jan 8, 2008, at 6:15 PM, hemant kumar wrote:

> Hi
> On Tue, 2008-01-08 at 17:43 +0100, Raimond Garcia wrote:
>> Hi,
>>
>> Got everything setup nicely with backgroundrb for the development
>> environment and running tests with rspec.  However, I have to keep
>> editing the backgroundrb.yml file, to switch between development and
>> testing environment.  How can I declare a development, testing and
>> production environment in backgroundrb.yml?
>>
>> Thanks in advance
>
> in your bdrb_test_helper.rb file put:
>
> RAILS_ENV = 'test' at top and you are set.
>
>
>


From gethemant at gmail.com  Tue Jan  8 12:57:40 2008
From: gethemant at gmail.com (hemant kumar)
Date: Tue, 08 Jan 2008 23:27:40 +0530
Subject: [Backgroundrb-devel] Different environments in backgroudrb.yml
In-Reply-To: <B6EE39C6-B947-4B79-8C05-AD7D0F356141@gmail.com>
References: <892287BD-5C09-4E7D-B56C-F70F0650CDBB@gmail.com>
	<1199812518.25054.9.camel@shire>
	<B6EE39C6-B947-4B79-8C05-AD7D0F356141@gmail.com>
Message-ID: <1199815060.25054.12.camel@shire>

Oh No, that won't work out, for running BackgrounDRb itself, you must
have proper entry in backgroundrb.yml file.

So, I will suggest you to have different files for development and
production environment and keep backgroundrb.yml file out of SVN. A cap
task can copy backgroundrb.yml.production to backgroundrb.yml when
deploying on machines.



On Tue, 2008-01-08 at 18:49 +0100, Raimond Garcia wrote:
> Cool, what about development and production environments? should I  
> just not include the environment definition in backgroundrb.yml, and  
> simply start the servers using the appropriate environment each time?  
> for example
> 
> RAILS_ENV=development ./script/backgroundrb start
> and
> RAILS_ENV=production ./script/backgroundrb start
> 
> 
> On Jan 8, 2008, at 6:15 PM, hemant kumar wrote:
> 
> > Hi
> > On Tue, 2008-01-08 at 17:43 +0100, Raimond Garcia wrote:
> >> Hi,
> >>
> >> Got everything setup nicely with backgroundrb for the development
> >> environment and running tests with rspec.  However, I have to keep
> >> editing the backgroundrb.yml file, to switch between development and
> >> testing environment.  How can I declare a development, testing and
> >> production environment in backgroundrb.yml?
> >>
> >> Thanks in advance
> >
> > in your bdrb_test_helper.rb file put:
> >
> > RAILS_ENV = 'test' at top and you are set.
> >
> >
> >
> 


From gobigdave at gmail.com  Tue Jan  8 13:22:29 2008
From: gobigdave at gmail.com (Dave Dupre)
Date: Tue, 8 Jan 2008 13:22:29 -0500
Subject: [Backgroundrb-devel] Different environments in backgroudrb.yml
In-Reply-To: <1199815060.25054.12.camel@shire>
References: <892287BD-5C09-4E7D-B56C-F70F0650CDBB@gmail.com>
	<1199812518.25054.9.camel@shire>
	<B6EE39C6-B947-4B79-8C05-AD7D0F356141@gmail.com>
	<1199815060.25054.12.camel@shire>
Message-ID: <258cc3f50801081022j2b83175di5498b55e056022e6@mail.gmail.com>

I use a cap task to generate the file rather than copy it.

In cap 2.0:

after "deploy:update_code", "backgroundrb:write_config"

namespace :backgroundrb do
  desc "Write backgroundrb config file"
  task :write_config, :roles => :app do
    rails_env = fetch(:rails_env, 'production')
    script = <<-SH
:backgroundrb:
  :ip: localhost
  :port: 11006
  :environment: #{rails_env}
SH
    put script, "#{release_path}/config/backgroundrb.yml", :mode => 0644
  end
end
end

I use the same technique for my spin script and just about any file that
needs slightly different contents depending on the environment.  You can
even use ERB to generate your apache scripts.  I find it much easier than
maintaining several versions of the same file.

Dave

On Jan 8, 2008 12:57 PM, hemant kumar <gethemant at gmail.com> wrote:

> Oh No, that won't work out, for running BackgrounDRb itself, you must
> have proper entry in backgroundrb.yml file.
>
> So, I will suggest you to have different files for development and
> production environment and keep backgroundrb.yml file out of SVN. A cap
> task can copy backgroundrb.yml.production to backgroundrb.yml when
> deploying on machines.
>
>
>
> On Tue, 2008-01-08 at 18:49 +0100, Raimond Garcia wrote:
> > Cool, what about development and production environments? should I
> > just not include the environment definition in backgroundrb.yml, and
> > simply start the servers using the appropriate environment each time?
> > for example
> >
> > RAILS_ENV=development ./script/backgroundrb start
> > and
> > RAILS_ENV=production ./script/backgroundrb start
> >
> >
> > On Jan 8, 2008, at 6:15 PM, hemant kumar wrote:
> >
> > > Hi
> > > On Tue, 2008-01-08 at 17:43 +0100, Raimond Garcia wrote:
> > >> Hi,
> > >>
> > >> Got everything setup nicely with backgroundrb for the development
> > >> environment and running tests with rspec.  However, I have to keep
> > >> editing the backgroundrb.yml file, to switch between development and
> > >> testing environment.  How can I declare a development, testing and
> > >> production environment in backgroundrb.yml?
> > >>
> > >> Thanks in advance
> > >
> > > in your bdrb_test_helper.rb file put:
> > >
> > > RAILS_ENV = 'test' at top and you are set.
> > >
> > >
> > >
> >
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080108/603235e0/attachment-0001.html 

From voodoorai2000 at gmail.com  Tue Jan  8 13:48:48 2008
From: voodoorai2000 at gmail.com (Raimond Garcia)
Date: Tue, 8 Jan 2008 19:48:48 +0100
Subject: [Backgroundrb-devel] Different environments in backgroudrb.yml
In-Reply-To: <258cc3f50801081022j2b83175di5498b55e056022e6@mail.gmail.com>
References: <892287BD-5C09-4E7D-B56C-F70F0650CDBB@gmail.com>
	<1199812518.25054.9.camel@shire>
	<B6EE39C6-B947-4B79-8C05-AD7D0F356141@gmail.com>
	<1199815060.25054.12.camel@shire>
	<258cc3f50801081022j2b83175di5498b55e056022e6@mail.gmail.com>
Message-ID: <2968516C-0959-4294-AFBF-696BB3C3A150@gmail.com>

Great!

Let me give you a little more context of my situation.

We are trying to setup cruisecontrol, and so, I guess we should start  
backgroundrb in the test environment, run all the tests and then  
restart backgroundrb in the production environment so that it runs  
smoothly in staging and production.

I guess both the solution to have different backgroundrb.yml files and  
running the appropriate capistrano tasks in the correct order would  
all work.

I'll talk to our lead developer in charge of cruisecontrol, and figure  
out how he is using capistrano to deal with testing environments and  
then apply your solutions to our specific case.

Thanks guys!


Rai


On Jan 8, 2008, at 7:22 PM, Dave Dupre wrote:

> I use a cap task to generate the file rather than copy it.
>
> In cap 2.0:
>
> after "deploy:update_code", "backgroundrb:write_config"
>
> namespace :backgroundrb do
>   desc "Write backgroundrb config file"
>   task :write_config, :roles => :app do
>     rails_env = fetch(:rails_env, 'production')
>     script = <<-SH
> :backgroundrb:
>   :ip: localhost
>   :port: 11006
>   :environment: #{rails_env}
> SH
>     put script, "#{release_path}/config/backgroundrb.yml", :mode =>  
> 0644
>   end
> end
> end
>
> I use the same technique for my spin script and just about any file  
> that needs slightly different contents depending on the  
> environment.  You can even use ERB to generate your apache scripts.   
> I find it much easier than maintaining several versions of the same  
> file.
>
> Dave
>
> On Jan 8, 2008 12:57 PM, hemant kumar <gethemant at gmail.com> wrote:
> Oh No, that won't work out, for running BackgrounDRb itself, you must
> have proper entry in backgroundrb.yml file.
>
> So, I will suggest you to have different files for development and
> production environment and keep backgroundrb.yml file out of SVN. A  
> cap
> task can copy backgroundrb.yml.production to backgroundrb.yml when
> deploying on machines.
>
>
>
> On Tue, 2008-01-08 at 18:49 +0100, Raimond Garcia wrote:
> > Cool, what about development and production environments? should I
> > just not include the environment definition in backgroundrb.yml, and
> > simply start the servers using the appropriate environment each  
> time?
> > for example
> >
> > RAILS_ENV=development ./script/backgroundrb start
> > and
> > RAILS_ENV=production ./script/backgroundrb start
> >
> >
> > On Jan 8, 2008, at 6:15 PM, hemant kumar wrote:
> >
> > > Hi
> > > On Tue, 2008-01-08 at 17:43 +0100, Raimond Garcia wrote:
> > >> Hi,
> > >>
> > >> Got everything setup nicely with backgroundrb for the development
> > >> environment and running tests with rspec.  However, I have to  
> keep
> > >> editing the backgroundrb.yml file, to switch between  
> development and
> > >> testing environment.  How can I declare a development, testing  
> and
> > >> production environment in backgroundrb.yml ?
> > >>
> > >> Thanks in advance
> > >
> > > in your bdrb_test_helper.rb file put:
> > >
> > > RAILS_ENV = 'test' at top and you are set.
> > >
> > >
> > >
> >
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080108/2bac2f00/attachment.html 

From zach at plugthegap.co.uk  Tue Jan  8 20:33:14 2008
From: zach at plugthegap.co.uk (Zachary Powell)
Date: Tue, 8 Jan 2008 20:33:14 -0500
Subject: [Backgroundrb-devel] Saving server
Message-ID: <ac243e420801081733v377b6f4bg97bca908e53c9769@mail.gmail.com>

Hi all,

If I modified the server script with
i=0
begin
    BackgrounDRb::MasterProxy.new()
rescue => e
    #some code to email notice of error
    i +=1
    retry if i < 5
end

would that work / be wise? (vs checking up on the process externally via
cron, and restarting it if it crashes)

thanks,

zacha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080108/8d4e62d2/attachment.html 

From gethemant at gmail.com  Tue Jan  8 20:33:16 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 9 Jan 2008 07:03:16 +0530
Subject: [Backgroundrb-devel] [ANN] BackgrounDRb 1.0.1 release available now
Message-ID: <fb283d5e0801081733l2945d90emdd20d8b0e6871cca@mail.gmail.com>

Hi Folks,

I am glad to announce a 1.0.1 release of BackgrounDRb library. It has
numerous fixes and some nice feature updates, so please upgrade.

Code:
  http://svn.devjavu.com/backgroundrb/tags/release-1.0.1/

Updates and Fixes:
 *  Fixed issue with Mysql Lost connections
 *  Message debugging can be disabled now with 'debug_log' option in
configuration file( :debug_log: false)
 *  Use lazy load (:lazy_load: true) of models in master
 * MiddleMan.delete_worker aborts a worker immediately now.
 * Get information about all running workers. ( MiddleMan.all_worker_info)
 * Enable setting of thread pool size. Tasks can be queued easily now.
 * Introspect a running worker ( MiddleMan.worker_info, note that this
is different that MiddleMan.ask_status, since later will return status
objects set by you in worker code)
 * Removed messing around with Core classes. BackgrounDRb doesn't
change behavior of any core classes now.
 * Worker status objects stay even after a worker dies.
 * BackgrounDRb can now store worker results/status objects in Memcache now.
   #backgroundbr.yml
      :result_storage:
         :memcache: "10.10.10.2.11211,10.10.10.6:11211"
 * Fixed issues with thread pool.
 * Committed Patches by Jason for test helper.
 * Committed patch for honoring wday option by Adam.

News:
 * Advanced Rails Recipes book contains a full recipe on using
BackgrounDRb now, hurray.

Upcoming:
 * Distributed workers and persistent worker queues.

Credits:
 * Ezra and Skaar for taking BackgrounDRb so far.
 * Francis for EventMachine and inspiration.
 * Jason for helping out with thread and mysql lost connection issues.
 * Adam for wday patch.
 * All the early adopters.

Usage and Installation Instructions:
 * http://backgroundrb.rubyforge.org


Last but not the least, if you are using BackgrounDRb plugin and wan't
to help, please send your patches either to track or to the list. I
would like to point that, there are NO outstanding bugs
thats unfixed with this version of BackgrounDRb. So give it a spin and
drop in a mail to the list, if you have any problems.

Also, if you are still using older version of BackgrounDRb, I will
suggest you to upgrade. It shouldn't be painful. Older version of
BackgrounDRb has few unresolved bugs because of dependency on Slave
library ( Yeah, I do not understand that library), so please upgrade.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Tue Jan  8 20:37:41 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 9 Jan 2008 07:07:41 +0530
Subject: [Backgroundrb-devel] Saving server
In-Reply-To: <ac243e420801081733v377b6f4bg97bca908e53c9769@mail.gmail.com>
References: <ac243e420801081733v377b6f4bg97bca908e53c9769@mail.gmail.com>
Message-ID: <fb283d5e0801081737r6b3ac1e6r73048ba7c16e7286@mail.gmail.com>

On Jan 9, 2008 7:03 AM, Zachary Powell <zach at plugthegap.co.uk> wrote:
> Hi all,
>
> If I modified the server script with
>
> i=0
> begin
>     BackgrounDRb::MasterProxy.new()
> rescue => e
>     #some code to email notice of error
>     i +=1
>     retry if i < 5
> end
>
> would that work / be wise? (vs checking up on the process externally via
> cron, and restarting it if it crashes)
>

I think that will work. But you should have a monit(not cron) script
anyways for checking up status of Master Process.

-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From chuyeow at gmail.com  Wed Jan  9 01:29:37 2008
From: chuyeow at gmail.com (Cheah Chu Yeow)
Date: Wed, 9 Jan 2008 14:29:37 +0800
Subject: [Backgroundrb-devel] [PATCH] x 2 - Fix for "null pointer exception"
	when ask_status is called before worker is run, and docfixes
Message-ID: <74ce3950801082229k788fd560i77184877fb1c326d@mail.gmail.com>

Sorry for attaching patches in emails to the mailing list, but I'm still
unable to login to Devjavu for Backgroundrb (I'm able to login just fine for
Merb's Devjavu and submitted a ticket successfully).
Anyway, I encountered a bug when trying to #ask_status of a worker before
it's run (or rather, before its result_hash has been set to something). Not
sure why it's no longer being initialized to an empty hash (since
http://backgroundrb.devjavu.com/changeset/279). I've attached a patch,
defaults_for_packet_master.diff for this.

update_readme.diff attempts to format the README so that it displays better
in HTML and also removes mention of worker tests being generated from the
worker_generator.

Let me know if there's a better way for me to submit patches!

Cheers,

Chu Yeow
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080109/11cb0823/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: update_readme.diff
Type: application/octet-stream
Size: 10804 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080109/11cb0823/attachment-0002.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: defaults_for_packet_master.diff
Type: application/octet-stream
Size: 1250 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080109/11cb0823/attachment-0003.obj 

From gethemant at gmail.com  Wed Jan  9 02:10:59 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 9 Jan 2008 12:40:59 +0530
Subject: [Backgroundrb-devel] [PATCH] x 2 - Fix for "null pointer
	exception" when ask_status is called before worker is run,
	and docfixes
In-Reply-To: <74ce3950801082229k788fd560i77184877fb1c326d@mail.gmail.com>
References: <74ce3950801082229k788fd560i77184877fb1c326d@mail.gmail.com>
Message-ID: <fb283d5e0801082310m26936092m75dad87af25c18ab@mail.gmail.com>

Hey,

On Jan 9, 2008 11:59 AM, Cheah Chu Yeow <chuyeow at gmail.com> wrote:
> Sorry for attaching patches in emails to the mailing list, but I'm still
> unable to login to Devjavu for Backgroundrb (I'm able to login just fine for
> Merb's Devjavu and submitted a ticket successfully).
>
>
> Anyway, I encountered a bug when trying to #ask_status of a worker before
> it's run (or rather, before its result_hash has been set to something). Not
> sure why it's no longer being initialized to an empty hash (since
> http://backgroundrb.devjavu.com/changeset/279). I've attached a patch,
> defaults_for_packet_master.diff for this.
>
> update_readme.diff attempts to format the README so that it displays better
> in HTML and also removes mention of worker tests being generated from the
> worker_generator.
>
> Let me know if there's a better way for me to submit patches!

Its weird that Devjavu is not letting you submit patches. I will try
to follow up on this.

Thanks for the patches and I will have them in trunk asap.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From chuyeow at gmail.com  Wed Jan  9 02:16:16 2008
From: chuyeow at gmail.com (Cheah Chu Yeow)
Date: Wed, 9 Jan 2008 15:16:16 +0800
Subject: [Backgroundrb-devel] [PATCH] x 2 - Fix for "null pointer
	exception" when ask_status is called before worker is run,
	and docfixes
In-Reply-To: <fb283d5e0801082310m26936092m75dad87af25c18ab@mail.gmail.com>
References: <74ce3950801082229k788fd560i77184877fb1c326d@mail.gmail.com>
	<fb283d5e0801082310m26936092m75dad87af25c18ab@mail.gmail.com>
Message-ID: <74ce3950801082316i1fccbe31w165c345c4b16503a@mail.gmail.com>

Thanks!
Hope this will help debug what's going on with Devjavu: when I login (it's
the right password, since if I try an incorrect password it spews an error
message), it redirects back to wherever I came from, but the 'Login |
Register' links are still showing and I can't create a new ticket.

Cheers,

Chu Yeow


On Jan 9, 2008 3:10 PM, hemant <gethemant at gmail.com> wrote:

> Hey,
>
> On Jan 9, 2008 11:59 AM, Cheah Chu Yeow <chuyeow at gmail.com> wrote:
> > Sorry for attaching patches in emails to the mailing list, but I'm still
> > unable to login to Devjavu for Backgroundrb (I'm able to login just fine
> for
> > Merb's Devjavu and submitted a ticket successfully).
> >
> >
> > Anyway, I encountered a bug when trying to #ask_status of a worker
> before
> > it's run (or rather, before its result_hash has been set to something).
> Not
> > sure why it's no longer being initialized to an empty hash (since
> > http://backgroundrb.devjavu.com/changeset/279). I've attached a patch,
> > defaults_for_packet_master.diff for this.
> >
> > update_readme.diff attempts to format the README so that it displays
> better
> > in HTML and also removes mention of worker tests being generated from
> the
> > worker_generator.
> >
> > Let me know if there's a better way for me to submit patches!
>
> Its weird that Devjavu is not letting you submit patches. I will try
> to follow up on this.
>
> Thanks for the patches and I will have them in trunk asap.
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>



-- 
Chu Yeow
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080109/26278a03/attachment.html 

From cschlaefcke at wms-network.de  Wed Jan  9 11:00:21 2008
From: cschlaefcke at wms-network.de (Christian Schlaefcke)
Date: Wed, 9 Jan 2008 17:00:21 +0100 (CET)
Subject: [Backgroundrb-devel] Windows Support?
Message-ID: <24046.159.103.208.26.1199894421.squirrel@www.wms-network.de>

Hi Folks,

it?s quite a while that I used BackgrounDRb and I was hoping that I get
the chance on working with it again, as I read so many promising things
here in the list!

When I last worked with it (Rel. 0.21) there was no way of running it
under Windows. Is this restriction still valid? I hope not, as I
desperately need to get it working with InstantRails in a Windows
environment for some new requirements I need to implement.

Thanks & Regards,

Christian



From gethemant at gmail.com  Wed Jan  9 14:12:57 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 10 Jan 2008 00:42:57 +0530
Subject: [Backgroundrb-devel] [PATCH] x 2 - Fix for "null pointer
	exception" when ask_status is called before worker is run,
	and docfixes
In-Reply-To: <74ce3950801082316i1fccbe31w165c345c4b16503a@mail.gmail.com>
References: <74ce3950801082229k788fd560i77184877fb1c326d@mail.gmail.com>
	<fb283d5e0801082310m26936092m75dad87af25c18ab@mail.gmail.com>
	<74ce3950801082316i1fccbe31w165c345c4b16503a@mail.gmail.com>
Message-ID: <fb283d5e0801091112l3c8de81dp71ee4326db66c9e6@mail.gmail.com>

Hi,

On Jan 9, 2008 12:46 PM, Cheah Chu Yeow <chuyeow at gmail.com> wrote:
> Thanks!
>
> Hope this will help debug what's going on with Devjavu: when I login (it's
> the right password, since if I try an incorrect password it spews an error
> message), it redirects back to wherever I came from, but the 'Login |
> Register' links are still showing and I can't create a new ticket.

Well the issue with result hash is fixed and I have updated 1.0.1
release tag as well as trunk.

I am starting work on Test Cases now. I saw your comment about lack of
test cases and I am working to amend the situation.
Between my office work, other projects, often things go wary. I have
created the infrastructure for testing the plugin and stuff, I will be
glad if you or anyone is willing to help.

I have been a user of BackgrounDRb since Ezra released first version
and followed the development closely. 0.2.1 release although a major
step forward had nasty unresolved bugs because of DRb and Slave
library. You can browse mailing list archives of ruby-talk and
backgroundrb and find that Slave is a glorious hack, but a hack
nonetheless ( Ara himself admits code is quite difficult to follow ).

So Ezra got busy and moved to better and bigger things, he asked if
anyone is willing to work on it and I volunteered. I think, Ezra
handed over the project not because he was overly impressed with my
contribution, but because NO ONE else stepped forward.

After taking up project, I was getting lots of bug reports ( again
mailing list archives is a proof). Now sad thing is, I was not able to
fix them. Why? Because fork(), threads and DRb was just a disaster. I
tried fixing existing stuff, but honestly it was a mess. I had lots of
workers running under heavy load and they would disappear. Whats more,
changes to slave library screwed up the plugin big time.


So, I did what I could do, I wrote an entire network programming
framework, removed threads, removed slave and drb and made a new
release. I am sure, its not been a smooth ride since then and I am
thankful to early adopters. I also perhaps hurried towards 1.0
release, I should kept versioning incremental. But anyways, we had a
programming model, which was totally transparent and simple ( for me
anyways ). I believe simple stuff should be kept simple ( again thanks
to Zed for the tip), so if you or anyone has time to look into
internals of BackgrounDRb, you will find it pretty easy to understand.
On the other hand,

Abstracting network IO in automated testing is hard and that explains
lack of test cases. But I am not defending it, I have been guilty of
it. I can only amend the situation I hope.

From gethemant at gmail.com  Wed Jan  9 14:15:50 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 10 Jan 2008 00:45:50 +0530
Subject: [Backgroundrb-devel] [PATCH] x 2 - Fix for "null pointer
	exception" when ask_status is called before worker is run,
	and docfixes
In-Reply-To: <fb283d5e0801091112l3c8de81dp71ee4326db66c9e6@mail.gmail.com>
References: <74ce3950801082229k788fd560i77184877fb1c326d@mail.gmail.com>
	<fb283d5e0801082310m26936092m75dad87af25c18ab@mail.gmail.com>
	<74ce3950801082316i1fccbe31w165c345c4b16503a@mail.gmail.com>
	<fb283d5e0801091112l3c8de81dp71ee4326db66c9e6@mail.gmail.com>
Message-ID: <fb283d5e0801091115n3600341cpb6b46eef3ae4d7c1@mail.gmail.com>

On Jan 10, 2008 12:42 AM, hemant <gethemant at gmail.com> wrote:
> Hi,
>
> On Jan 9, 2008 12:46 PM, Cheah Chu Yeow <chuyeow at gmail.com> wrote:
> > Thanks!
> >
> > Hope this will help debug what's going on with Devjavu: when I login (it's
> > the right password, since if I try an incorrect password it spews an error
> > message), it redirects back to wherever I came from, but the 'Login |
> > Register' links are still showing and I can't create a new ticket.
>
> Well the issue with result hash is fixed and I have updated 1.0.1
> release tag as well as trunk.
>
> I am starting work on Test Cases now. I saw your comment about lack of
> test cases and I am working to amend the situation.
> Between my office work, other projects, often things go wary. I have
> created the infrastructure for testing the plugin and stuff, I will be
> glad if you or anyone is willing to help.
>
> I have been a user of BackgrounDRb since Ezra released first version
> and followed the development closely. 0.2.1 release although a major
> step forward had nasty unresolved bugs because of DRb and Slave
> library. You can browse mailing list archives of ruby-talk and
> backgroundrb and find that Slave is a glorious hack, but a hack
> nonetheless ( Ara himself admits code is quite difficult to follow ).
>
> So Ezra got busy and moved to better and bigger things, he asked if
> anyone is willing to work on it and I volunteered. I think, Ezra
> handed over the project not because he was overly impressed with my
> contribution, but because NO ONE else stepped forward.
>
> After taking up project, I was getting lots of bug reports ( again
> mailing list archives is a proof). Now sad thing is, I was not able to
> fix them. Why? Because fork(), threads and DRb was just a disaster. I
> tried fixing existing stuff, but honestly it was a mess. I had lots of
> workers running under heavy load and they would disappear. Whats more,
> changes to slave library screwed up the plugin big time.
>
>
> So, I did what I could do, I wrote an entire network programming
> framework, removed threads, removed slave and drb and made a new
> release. I am sure, its not been a smooth ride since then and I am
> thankful to early adopters. I also perhaps hurried towards 1.0
> release, I should kept versioning incremental. But anyways, we had a
> programming model, which was totally transparent and simple ( for me
> anyways ). I believe simple stuff should be kept simple ( again thanks
> to Zed for the tip), so if you or anyone has time to look into
> internals of BackgrounDRb, you will find it pretty easy to understand.
> On the other hand,

Try reading code of drb of slave. I am sure they offer lot more
features, but its turtles all the way down.

>
> Abstracting network IO in automated testing is hard and that explains
> lack of test cases. But I am not defending it, I have been guilty of
> it. I can only amend the situation I hope.
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From stevie at slowbicycle.com  Wed Jan  9 14:17:32 2008
From: stevie at slowbicycle.com (Stevie Clifton)
Date: Wed, 9 Jan 2008 11:17:32 -0800
Subject: [Backgroundrb-devel] [ANN] BackgrounDRb 1.0 pre-release
	available now
In-Reply-To: <1197370248.9002.13.camel@shire>
References: <fb283d5e0712101547m43a4febaqbf3d855545adb34e@mail.gmail.com>
	<74b000990712110153h7f6ef991p8034fb4d1ae1a281@mail.gmail.com>
	<1197370248.9002.13.camel@shire>
Message-ID: <a426f29f0801091117w6e60ec2cpf1ef891a91bc9af4@mail.gmail.com>

Hi Hemant,

Sorry for referring to an old thread, but I'm confused about appropriate
times to use thread_pool vs putting sockets in the select loop (or if that
has been deprecated).

In your thread_pool example, you have this:

def fetch_url(url)
  puts "fetching url #{url}"
  thread_pool.defer(url) do |url|
    begin
      data = Net::HTTP.get(url,'/')
      File.open("#{RAILS_ROOT}/log/pages.txt" ,"w") do |fl|
        fl.puts(data)
      end
   rescue
     logger.info "Error downloading page"
   end
 end
end

But in this thread (from Dec 11) you mentioned that it is much better to put
connections in the select loop via Backgroundrb::HTTP.open to stay with the
evented paradigm instead of directly calling Net::HTTP.  In my case, I want
to open up a bunch of lightweight connections to various servers, but I'd
like to have at most 20 open at a time.  In this case, would it make sense
to use a thread_pool of size 20 to manage the size of the connection pool,
but also use Backgroundrb:: HTTP.open for better socket concurrency?  Just
trying to get this all straight.

Thanks!

stevie


On 12/11/07, hemant kumar <gethemant at gmail.com> wrote:
>
> Hi
>
> On Tue, 2007-12-11 at 10:53 +0100, Emil Marceta wrote:
>
> > Does this means that slave/daemons are not the dependency anymore?
>
> Yes, its gone. bdrb no longer depends on slave and daemons.
>
> > By 'not encouraged' do you mean that 1.0 is not supporting multiple
> > threads in the worker or just as a general guidance?
> >
> > Could you please comment, how would you approach the following
> > scenario with 1.0. Currently, we have a worker that creates threads
> > that process financial payment transactions. An http request sends
> > several 10s or 100s payment transaction records. They are handled by
> > the single worker instance. Within the worker there is a pool of
> > threads created that is calculated based on the number of
> > transactions. For example for 200 transactions there will be 20
> > threads where each thread handles 10 requests in a squence. Each
> > transaction takes about 3-5 seconds, so our throughput is
> > significantly improved by internal worker parallelization with a
> > thread pool. The worker periodically updates custom backgroundjob
> > databse record, so that following ajax request from the client can
> > read the status of the worker process. The job is identified with the
> > worker key that is stored in the session.
>
> Its not encouraged, thats all. You can still have threads in your
> workers. However, I am planning to add thread pool feature in bdrb
> itself, that should simplify things a bit.
>
> Also ideally, when using EventDriven network programming, you want all
> your sockets within select loop for efficiency. So, you wouldn't need
> any damn threads, if you can use a HTTP handler that works in Evented
> manner. What i mean to say is, you don't do this:
>
> a = Net::HTTP.get("http://www.google.com")
>
> but you do,
>
> Backgroundrb::HTTP.open("http://www.google.com") do |data|
> process_data(data)
> end
>
> What I am trying to illustrate is, when you ask to open, google.com
> page, evented model allows you to attach callback ( the block in this
> case ), which will be called when data arrives from google.com, rather
> than waiting for it in a thread. So, BackgrounDRb::HTTP.open() returns
> immediately. And you are concurrent as hell.
>
> But this is not possible, because if you are charging cards, then you
> are probably using ActiveMerchant which is using Net::HTTP and which
> blocks when you make request. But trust me, writing a simple http client
> is not that difficult, there is already connect() available in all
> workers.
>
>
> > How this works with fastcgi or multiple mongrel based engines where it
> > is not guaranteed to hit the same process with the next request? We
> > are using custom database tables and code for sharing the status
> > information now but I was wandering whether the plumbing includes
> > something to address this.
>
> Thats no problem at all, BackgrounDRb is a TCP server, so if you have
> followed the README file, no matter from which machine, you are making
> the request if you are specifying worker X, then its guaranteed to hit
> the same worker(with optional job_key if you are starting your worker
> dynamically)
>
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080109/cc7a4947/attachment-0001.html 

From gethemant at gmail.com  Wed Jan  9 14:25:13 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 10 Jan 2008 00:55:13 +0530
Subject: [Backgroundrb-devel] [ANN] BackgrounDRb 1.0 pre-release
	available now
In-Reply-To: <a426f29f0801091117w6e60ec2cpf1ef891a91bc9af4@mail.gmail.com>
References: <fb283d5e0712101547m43a4febaqbf3d855545adb34e@mail.gmail.com>
	<74b000990712110153h7f6ef991p8034fb4d1ae1a281@mail.gmail.com>
	<1197370248.9002.13.camel@shire>
	<a426f29f0801091117w6e60ec2cpf1ef891a91bc9af4@mail.gmail.com>
Message-ID: <fb283d5e0801091125q6ce795d6sf71e210df5a51a49@mail.gmail.com>

Hi Stevie,

The problem is BackgrounDRb::HTTP doesn't exist. I just tried to
explain that, if you are really concerned about performance you can
implement HTTP client in a non blocking manner and you won't need
threads. For example, EventMachine guys have already implemented HTTP
client over evented sockets. You can use their code, I guess.

But its not easy task, unless BackgrounDRb itself provides this, so
for the time being go with threads.




On Jan 10, 2008 12:47 AM, Stevie Clifton <stevie at slowbicycle.com> wrote:
>
> Hi Hemant,
>
> Sorry for referring to an old thread, but I'm confused about appropriate
> times to use thread_pool vs putting sockets in the select loop (or if that
> has been deprecated).
>
> In your thread_pool example, you have this:
>
> def fetch_url(url)
>   puts "fetching url #{url}"
>   thread_pool.defer(url) do |url|
>     begin
>       data = Net::HTTP.get(url,'/')
>       File.open("#{RAILS_ROOT}/log/pages.txt" ,"w") do |fl|
>         fl.puts(data)
>       end
>    rescue
>      logger.info "Error downloading page"
>    end
>  end
> end
>
> But in this thread (from Dec 11) you mentioned that it is much better to put
> connections in the select loop via Backgroundrb::HTTP.open to stay with the
> evented paradigm instead of directly calling Net::HTTP.  In my case, I want
> to open up a bunch of lightweight connections to various servers, but I'd
> like to have at most 20 open at a time.  In this case, would it make sense
> to use a thread_pool of size 20 to manage the size of the connection pool,
> but also use Backgroundrb:: HTTP.open for better socket concurrency?  Just
> trying to get this all straight.
>
> Thanks!
>
> stevie
>
> On 12/11/07, hemant kumar <gethemant at gmail.com> wrote:
> > Hi
> >
> > On Tue, 2007-12-11 at 10:53 +0100, Emil Marceta wrote:
> >
> > > Does this means that slave/daemons are not the dependency anymore?
> >
> > Yes, its gone. bdrb no longer depends on slave and daemons.
> >
> > > By 'not encouraged' do you mean that 1.0 is not supporting multiple
> > > threads in the worker or just as a general guidance?
> > >
> > > Could you please comment, how would you approach the following
> > > scenario with 1.0. Currently, we have a worker that creates threads
> > > that process financial payment transactions. An http request sends
> > > several 10s or 100s payment transaction records. They are handled by
> > > the single worker instance. Within the worker there is a pool of
> > > threads created that is calculated based on the number of
> > > transactions. For example for 200 transactions there will be 20
> > > threads where each thread handles 10 requests in a squence. Each
> > > transaction takes about 3-5 seconds, so our throughput is
> > > significantly improved by internal worker parallelization with a
> > > thread pool. The worker periodically updates custom backgroundjob
> > > databse record, so that following ajax request from the client can
> > > read the status of the worker process. The job is identified with the
> > > worker key that is stored in the session.
> >
> > Its not encouraged, thats all. You can still have threads in your
> > workers. However, I am planning to add thread pool feature in bdrb
> > itself, that should simplify things a bit.
> >
> > Also ideally, when using EventDriven network programming, you want all
> > your sockets within select loop for efficiency. So, you wouldn't need
> > any damn threads, if you can use a HTTP handler that works in Evented
> > manner. What i mean to say is, you don't do this:
> >
> > a = Net::HTTP.get("http://www.google.com")
> >
> > but you do,
> >
> > Backgroundrb::HTTP.open("http://www.google.com") do |data|
> > process_data(data)
> > end
> >
> > What I am trying to illustrate is, when you ask to open, google.com
> > page, evented model allows you to attach callback ( the block in this
> > case ), which will be called when data arrives from google.com, rather
> > than waiting for it in a thread. So, BackgrounDRb::HTTP.open() returns
> > immediately. And you are concurrent as hell.
> >
> > But this is not possible, because if you are charging cards, then you
> > are probably using ActiveMerchant which is using Net::HTTP and which
> > blocks when you make request. But trust me, writing a simple http client
> > is not that difficult, there is already connect() available in all
> > workers.
> >
> >
> > > How this works with fastcgi or multiple mongrel based engines where it
> > > is not guaranteed to hit the same process with the next request? We
> > > are using custom database tables and code for sharing the status
> > > information now but I was wandering whether the plumbing includes
> > > something to address this.
> >
> > Thats no problem at all, BackgrounDRb is a TCP server, so if you have
> > followed the README file, no matter from which machine, you are making
> > the request if you are specifying worker X, then its guaranteed to hit
> > the same worker(with optional job_key if you are starting your worker
> > dynamically)
> >
> >
> >
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >
>
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From chuyeow at gmail.com  Wed Jan  9 21:29:35 2008
From: chuyeow at gmail.com (Cheah Chu Yeow)
Date: Thu, 10 Jan 2008 10:29:35 +0800
Subject: [Backgroundrb-devel] [PATCH] x 2 - Fix for "null pointer
	exception" when ask_status is called before worker is run,
	and docfixes
In-Reply-To: <fb283d5e0801091115n3600341cpb6b46eef3ae4d7c1@mail.gmail.com>
References: <74ce3950801082229k788fd560i77184877fb1c326d@mail.gmail.com>
	<fb283d5e0801082310m26936092m75dad87af25c18ab@mail.gmail.com>
	<74ce3950801082316i1fccbe31w165c345c4b16503a@mail.gmail.com>
	<fb283d5e0801091112l3c8de81dp71ee4326db66c9e6@mail.gmail.com>
	<fb283d5e0801091115n3600341cpb6b46eef3ae4d7c1@mail.gmail.com>
Message-ID: <74ce3950801091829x2bbe2105k21d53cc0282c713f@mail.gmail.com>

Hey Hemant, thanks for the detailed explanation. First off, let me just say
that I really appreciate you stepping up to take over - I knew how it
happened (Ezra not having time and such). And I think you are referring to
my comment in http://dominiek.com/articles/2008/1/9/wigitize-com-post-digg.
Dominiek (that blogger) also calls you a prick in an earlier blog post which
of course is terribly uncalled for.
I'd like to help out with add testcases (or specs actually, which I prefer
more in this case since RSpec allows you to do mocking really easily) so
when I do get some time (I spend most of my "Open Source time" on Rails)
I'll be sure to chime in with some patches.

Anyway, don't let it get to you, you're doing a good job but I can see that
there's probably not enough hands helping by contributing back to
BackgrounDRb - a one-man show is not easy!

Cheers,

Chu Yeow



On Jan 10, 2008 3:15 AM, hemant <gethemant at gmail.com> wrote:

> On Jan 10, 2008 12:42 AM, hemant <gethemant at gmail.com> wrote:
> > Hi,
> >
> > On Jan 9, 2008 12:46 PM, Cheah Chu Yeow <chuyeow at gmail.com> wrote:
> > > Thanks!
> > >
> > > Hope this will help debug what's going on with Devjavu: when I login
> (it's
> > > the right password, since if I try an incorrect password it spews an
> error
> > > message), it redirects back to wherever I came from, but the 'Login |
> > > Register' links are still showing and I can't create a new ticket.
> >
> > Well the issue with result hash is fixed and I have updated 1.0.1
> > release tag as well as trunk.
> >
> > I am starting work on Test Cases now. I saw your comment about lack of
> > test cases and I am working to amend the situation.
> > Between my office work, other projects, often things go wary. I have
> > created the infrastructure for testing the plugin and stuff, I will be
> > glad if you or anyone is willing to help.
> >
> > I have been a user of BackgrounDRb since Ezra released first version
> > and followed the development closely. 0.2.1 release although a major
> > step forward had nasty unresolved bugs because of DRb and Slave
> > library. You can browse mailing list archives of ruby-talk and
> > backgroundrb and find that Slave is a glorious hack, but a hack
> > nonetheless ( Ara himself admits code is quite difficult to follow ).
> >
> > So Ezra got busy and moved to better and bigger things, he asked if
> > anyone is willing to work on it and I volunteered. I think, Ezra
> > handed over the project not because he was overly impressed with my
> > contribution, but because NO ONE else stepped forward.
> >
> > After taking up project, I was getting lots of bug reports ( again
> > mailing list archives is a proof). Now sad thing is, I was not able to
> > fix them. Why? Because fork(), threads and DRb was just a disaster. I
> > tried fixing existing stuff, but honestly it was a mess. I had lots of
> > workers running under heavy load and they would disappear. Whats more,
> > changes to slave library screwed up the plugin big time.
> >
> >
> > So, I did what I could do, I wrote an entire network programming
> > framework, removed threads, removed slave and drb and made a new
> > release. I am sure, its not been a smooth ride since then and I am
> > thankful to early adopters. I also perhaps hurried towards 1.0
> > release, I should kept versioning incremental. But anyways, we had a
> > programming model, which was totally transparent and simple ( for me
> > anyways ). I believe simple stuff should be kept simple ( again thanks
> > to Zed for the tip), so if you or anyone has time to look into
> > internals of BackgrounDRb, you will find it pretty easy to understand.
> > On the other hand,
>
> Try reading code of drb of slave. I am sure they offer lot more
> features, but its turtles all the way down.
>
> >
> > Abstracting network IO in automated testing is hard and that explains
> > lack of test cases. But I am not defending it, I have been guilty of
> > it. I can only amend the situation I hope.
> >
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>



-- 
Chu Yeow
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080110/7d9a2250/attachment.html 

From dburkes at infoteria.com  Thu Jan 10 11:38:10 2008
From: dburkes at infoteria.com (Danny Burkes)
Date: Thu, 10 Jan 2008 08:38:10 -0800
Subject: [Backgroundrb-devel] Worker suicide
Message-ID: <3CF8D7A7-31D8-4DF4-8315-E4191314C5AA@infoteria.com>

I have a worker which wants to delete itself after it finishes its  
work- how can it do that?

Best Regards,

Danny Burkes

From zach at plugthegap.co.uk  Thu Jan 10 11:58:57 2008
From: zach at plugthegap.co.uk (Zachary Powell)
Date: Thu, 10 Jan 2008 11:58:57 -0500
Subject: [Backgroundrb-devel] Worker suicide
In-Reply-To: <3CF8D7A7-31D8-4DF4-8315-E4191314C5AA@infoteria.com>
References: <3CF8D7A7-31D8-4DF4-8315-E4191314C5AA@infoteria.com>
Message-ID: <ac243e420801100858i38d590dbma8fb2526619d888f@mail.gmail.com>

Danny,
put "exit" on the last line

def create
  #... do stuff
  exit
end

(can be called anywhere in the worker, in other methods etc).


On 1/10/08, Danny Burkes <dburkes at infoteria.com> wrote:
>
> I have a worker which wants to delete itself after it finishes its
> work- how can it do that?
>
> Best Regards,
>
> Danny Burkes
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080110/808d0d3a/attachment.html 

From dburkes at infoteria.com  Thu Jan 10 12:13:14 2008
From: dburkes at infoteria.com (Danny Burkes)
Date: Thu, 10 Jan 2008 09:13:14 -0800
Subject: [Backgroundrb-devel] Worker suicide
In-Reply-To: <ac243e420801100858i38d590dbma8fb2526619d888f@mail.gmail.com>
References: <3CF8D7A7-31D8-4DF4-8315-E4191314C5AA@infoteria.com>
	<ac243e420801100858i38d590dbma8fb2526619d888f@mail.gmail.com>
Message-ID: <9C2FF5C8-FC09-4A7E-A66C-E9726D919EFD@infoteria.com>

> put "exit" on the last line
>
> def create
>   #... do stuff
>   exit
> end
>
> (can be called anywhere in the worker, in other methods etc).
>

That doesn't seem to work- I still see the ruby process in ps.  This  
is a dynamically-created worker, and I want to exit the entire process  
once it is finished.  Any other ideas?

Thanks,

Danny

From zach at plugthegap.co.uk  Thu Jan 10 12:17:57 2008
From: zach at plugthegap.co.uk (Zachary Powell)
Date: Thu, 10 Jan 2008 12:17:57 -0500
Subject: [Backgroundrb-devel] Worker suicide
In-Reply-To: <9C2FF5C8-FC09-4A7E-A66C-E9726D919EFD@infoteria.com>
References: <3CF8D7A7-31D8-4DF4-8315-E4191314C5AA@infoteria.com>
	<ac243e420801100858i38d590dbma8fb2526619d888f@mail.gmail.com>
	<9C2FF5C8-FC09-4A7E-A66C-E9726D919EFD@infoteria.com>
Message-ID: <ac243e420801100917j101d418hae63f67792520532@mail.gmail.com>

exit is definitely the command (its the ruby kernel method for ending the
process). though i've never running into it not working. are you sure its
getting as far as the exit command (not hanging up on a loop etc)? try
logger.info('before exit') and 'after exit' to see if that shows up in the
backgroundrb.log.
i'm assuming you've read the docs and understand the "set_no_auto_load
true") method etc, so ruling out the possibility of it just been a different
copy of the worker..
On 1/10/08, Danny Burkes <dburkes at infoteria.com> wrote:
>
> > put "exit" on the last line
> >
> > def create
> >   #... do stuff
> >   exit
> > end
> >
> > (can be called anywhere in the worker, in other methods etc).
> >
>
> That doesn't seem to work- I still see the ruby process in ps.  This
> is a dynamically-created worker, and I want to exit the entire process
> once it is finished.  Any other ideas?
>
> Thanks,
>
> Danny
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080110/127e1264/attachment-0001.html 

From dburkes at infoteria.com  Thu Jan 10 12:29:31 2008
From: dburkes at infoteria.com (Danny Burkes)
Date: Thu, 10 Jan 2008 09:29:31 -0800
Subject: [Backgroundrb-devel] Worker suicide
In-Reply-To: <ac243e420801100917j101d418hae63f67792520532@mail.gmail.com>
References: <3CF8D7A7-31D8-4DF4-8315-E4191314C5AA@infoteria.com>
	<ac243e420801100858i38d590dbma8fb2526619d888f@mail.gmail.com>
	<9C2FF5C8-FC09-4A7E-A66C-E9726D919EFD@infoteria.com>
	<ac243e420801100917j101d418hae63f67792520532@mail.gmail.com>
Message-ID: <CE18C481-A03A-4940-B242-72C564E5CB10@infoteria.com>

> exit is definitely the command (its the ruby kernel method for  
> ending the process). though i've never running into it not working.  
> are you sure its getting as far as the exit command (not hanging up  
> on a loop etc)? try logger.info('before exit') and 'after exit' to  
> see if that shows up in the backgroundrb.log.
>

Ah, thanks- a closer examination of the logfile revealed that you are  
right.

Thanks for your help-

D

From joshea at nooked.com  Thu Jan 10 14:42:00 2008
From: joshea at nooked.com (John O'Shea)
Date: Thu, 10 Jan 2008 19:42:00 +0000
Subject: [Backgroundrb-devel] Worker suicide
In-Reply-To: <ac243e420801100917j101d418hae63f67792520532@mail.gmail.com>
References: <3CF8D7A7-31D8-4DF4-8315-E4191314C5AA@infoteria.com>
	<ac243e420801100858i38d590dbma8fb2526619d888f@mail.gmail.com>
	<9C2FF5C8-FC09-4A7E-A66C-E9726D919EFD@infoteria.com>
	<ac243e420801100917j101d418hae63f67792520532@mail.gmail.com>
Message-ID: <906500BB-9040-4A26-BC97-342B1B2F494B@nooked.com>

By co-incidence I have the same use case - the (dynamically created)  
worker process is exiting just fine when it hits the exit at the end  
of the create method.  However, subsequent calls to  
MiddleMan.query_all_workers still lists the worker and its last known  
status.  I have a web UI for monitoring worker activity so this is a  
little misleading for our admins.

Is there any way to get the master to realize the worker is gone?  I  
am also seeing "Some read error" messages in the backroundrb console  
and judging from a related discussion at http://rubyforge.org/pipermail/backgroundrb-devel/2007-December/001146.html 
  the master isn't cleaning up the socket connection to the worker  
(and removing the worker from its internal lists).

(I'm running from the 1.0 release tag btw).

John.

On 10 Jan 2008, at 17:17, Zachary Powell wrote:

> exit is definitely the command (its the ruby kernel method for  
> ending the process). though i've never running into it not working.  
> are you sure its getting as far as the exit command (not hanging up  
> on a loop etc)? try logger.info('before exit') and 'after exit' to  
> see if that shows up in the backgroundrb.log.
>
> i'm assuming you've read the docs and understand the  
> "set_no_auto_load true") method etc, so ruling out the possibility  
> of it just been a different copy of the worker..
>
> On 1/10/08, Danny Burkes <dburkes at infoteria.com> wrote:
> > put "exit" on the last line
> >
> > def create
> >   #... do stuff
> >   exit
> > end
> >
> > (can be called anywhere in the worker, in other methods etc).
> >
>
> That doesn't seem to work- I still see the ruby process in ps.  This
> is a dynamically-created worker, and I want to exit the entire process
> once it is finished.  Any other ideas?
>
> Thanks,
>
> Danny
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080110/784595ea/attachment.html 

From gethemant at gmail.com  Thu Jan 10 15:16:37 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 11 Jan 2008 01:46:37 +0530
Subject: [Backgroundrb-devel] Worker suicide
In-Reply-To: <906500BB-9040-4A26-BC97-342B1B2F494B@nooked.com>
References: <3CF8D7A7-31D8-4DF4-8315-E4191314C5AA@infoteria.com>
	<ac243e420801100858i38d590dbma8fb2526619d888f@mail.gmail.com>
	<9C2FF5C8-FC09-4A7E-A66C-E9726D919EFD@infoteria.com>
	<ac243e420801100917j101d418hae63f67792520532@mail.gmail.com>
	<906500BB-9040-4A26-BC97-342B1B2F494B@nooked.com>
Message-ID: <fb283d5e0801101216m70662238gb4f9e06f914791f4@mail.gmail.com>

On Jan 11, 2008 1:12 AM, John O'Shea <joshea at nooked.com> wrote:
> By co-incidence I have the same use case - the (dynamically created) worker
> process is exiting just fine when it hits the exit at the end of the create
> method.  However, subsequent calls to MiddleMan.query_all_workers still
> lists the worker and its last known status.  I have a web UI for monitoring
> worker activity so this is a little misleading for our admins.

Thats intentional, although name is a misnomer I admit. But if you
want to find the workers which are currently running,use
MiddleMan.all_worker_info

>
> Is there any way to get the master to realize the worker is gone?  I am also
> seeing "Some read error" messages in the backroundrb console and judging
> from a related discussion at
> http://rubyforge.org/pipermail/backgroundrb-devel/2007-December/001146.html
> the master isn't cleaning up the socket connection to the worker (and
> removing the worker from its internal lists).
>
> (I'm running from the 1.0 release tag btw).

Master DOES realize when a worker is gone and please upgrade to trunk
or 1.0.1 release. Its rather important and above
method(all_worker_info) has been added in 1.0.1 only.

-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Thu Jan 10 15:31:34 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 11 Jan 2008 02:01:34 +0530
Subject: [Backgroundrb-devel] Windows Support?
In-Reply-To: <24046.159.103.208.26.1199894421.squirrel@www.wms-network.de>
References: <24046.159.103.208.26.1199894421.squirrel@www.wms-network.de>
Message-ID: <fb283d5e0801101231pc558fet7e6d2fda6e0d70f2@mail.gmail.com>

Hi Christian,

Terribly sorry for this late reply. But BackgrounDRb can't work on
Windows still.
If you or someone is willing to port UNIX Socket mechanism with named
pipes and stuff, I am all for helping that guy out.


On Jan 9, 2008 9:30 PM, Christian Schlaefcke <cschlaefcke at wms-network.de> wrote:
> Hi Folks,
>
> it?s quite a while that I used BackgrounDRb and I was hoping that I get
> the chance on working with it again, as I read so many promising things
> here in the list!
>
> When I last worked with it (Rel. 0.21) there was no way of running it
> under Windows. Is this restriction still valid? I hope not, as I
> desperately need to get it working with InstantRails in a Windows
> environment for some new requirements I need to implement.
>
> Thanks & Regards,
>
> Christian
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Thu Jan 10 15:34:45 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 11 Jan 2008 02:04:45 +0530
Subject: [Backgroundrb-devel] [PATCH] x 2 - Fix for "null pointer
	exception" when ask_status is called before worker is run,
	and docfixes
In-Reply-To: <74ce3950801091829x2bbe2105k21d53cc0282c713f@mail.gmail.com>
References: <74ce3950801082229k788fd560i77184877fb1c326d@mail.gmail.com>
	<fb283d5e0801082310m26936092m75dad87af25c18ab@mail.gmail.com>
	<74ce3950801082316i1fccbe31w165c345c4b16503a@mail.gmail.com>
	<fb283d5e0801091112l3c8de81dp71ee4326db66c9e6@mail.gmail.com>
	<fb283d5e0801091115n3600341cpb6b46eef3ae4d7c1@mail.gmail.com>
	<74ce3950801091829x2bbe2105k21d53cc0282c713f@mail.gmail.com>
Message-ID: <fb283d5e0801101234ibda18a6r6e614f0e50adc05b@mail.gmail.com>

Hi Cheah,

On Jan 10, 2008 7:59 AM, Cheah Chu Yeow <chuyeow at gmail.com> wrote:
> Hey Hemant, thanks for the detailed explanation. First off, let me just say
> that I really appreciate you stepping up to take over - I knew how it
> happened (Ezra not having time and such). And I think you are referring to
> my comment in  http://dominiek.com/articles/2008/1/9/wigitize-com-post-digg.
> Dominiek (that blogger) also calls you a prick in an earlier blog post which
> of course is terribly uncalled for.
>
>  I'd like to help out with add testcases (or specs actually, which I prefer
> more in this case since RSpec allows you to do mocking really easily) so
> when I do get some time (I spend most of my "Open Source time" on Rails)
> I'll be sure to chime in with some patches.
>
>  Anyway, don't let it get to you, you're doing a good job but I can see that
> there's probably not enough hands helping by contributing back to
> BackgrounDRb - a one-man show is not easy!


Thanks man. I prefer Test/Spec because it has worked better than RSpec
for me in past. Test Helpers are written with Test/Spec in mind.
I will be working on this, coming weekend. If you want to help, I will
be available on #backgroundrb on freenode.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From joshea at nooked.com  Thu Jan 10 18:22:44 2008
From: joshea at nooked.com (John O'Shea)
Date: Thu, 10 Jan 2008 23:22:44 +0000
Subject: [Backgroundrb-devel] Worker suicide
In-Reply-To: <fb283d5e0801101216m70662238gb4f9e06f914791f4@mail.gmail.com>
References: <3CF8D7A7-31D8-4DF4-8315-E4191314C5AA@infoteria.com>
	<ac243e420801100858i38d590dbma8fb2526619d888f@mail.gmail.com>
	<9C2FF5C8-FC09-4A7E-A66C-E9726D919EFD@infoteria.com>
	<ac243e420801100917j101d418hae63f67792520532@mail.gmail.com>
	<906500BB-9040-4A26-BC97-342B1B2F494B@nooked.com>
	<fb283d5e0801101216m70662238gb4f9e06f914791f4@mail.gmail.com>
Message-ID: <75F1DAF2-450D-49A1-8F98-2B4D0ACE7090@nooked.com>

hemant,
	Thanks for the pointer, works well on HEAD.  For the benefit of  
others, the all_worker_info method returns an array of hashes (one for  
each worker) containing the following keys
:worker (value is name)
:job_key (value is an empty string if no job key assigned)
:status (value seems to be hardcoded to 'running')

I'm not sure if the hardcoded status is intentional but I'm working  
around with something like

@workers = MiddleMan.all_worker_info
@workers.each { |worker|
   worker[:status]= worker[:job_key].empty? ?
     MiddleMan.ask_status(:worker => worker[:worker]) :
     MiddleMan.ask_status(:worker => worker[:worker], :job_key =>  
worker[:job_key])
}

John.

On 10 Jan 2008, at 20:16, hemant wrote:

> On Jan 11, 2008 1:12 AM, John O'Shea <joshea at nooked.com> wrote:
>> By co-incidence I have the same use case - the (dynamically  
>> created) worker
>> process is exiting just fine when it hits the exit at the end of  
>> the create
>> method.  However, subsequent calls to MiddleMan.query_all_workers  
>> still
>> lists the worker and its last known status.  I have a web UI for  
>> monitoring
>> worker activity so this is a little misleading for our admins.
>
> Thats intentional, although name is a misnomer I admit. But if you
> want to find the workers which are currently running,use
> MiddleMan.all_worker_info
>
>>
>> Is there any way to get the master to realize the worker is gone?   
>> I am also
>> seeing "Some read error" messages in the backroundrb console and  
>> judging
>> from a related discussion at
>> http://rubyforge.org/pipermail/backgroundrb-devel/2007-December/001146.html
>> the master isn't cleaning up the socket connection to the worker (and
>> removing the worker from its internal lists).
>>
>> (I'm running from the 1.0 release tag btw).
>
> Master DOES realize when a worker is gone and please upgrade to trunk
> or 1.0.1 release. Its rather important and above
> method(all_worker_info) has been added in 1.0.1 only.
>
> -- 
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org


From gethemant at gmail.com  Thu Jan 10 18:40:03 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 11 Jan 2008 05:10:03 +0530
Subject: [Backgroundrb-devel] Worker suicide
In-Reply-To: <75F1DAF2-450D-49A1-8F98-2B4D0ACE7090@nooked.com>
References: <3CF8D7A7-31D8-4DF4-8315-E4191314C5AA@infoteria.com>
	<ac243e420801100858i38d590dbma8fb2526619d888f@mail.gmail.com>
	<9C2FF5C8-FC09-4A7E-A66C-E9726D919EFD@infoteria.com>
	<ac243e420801100917j101d418hae63f67792520532@mail.gmail.com>
	<906500BB-9040-4A26-BC97-342B1B2F494B@nooked.com>
	<fb283d5e0801101216m70662238gb4f9e06f914791f4@mail.gmail.com>
	<75F1DAF2-450D-49A1-8F98-2B4D0ACE7090@nooked.com>
Message-ID: <fb283d5e0801101540r3401c88as8cea190bbf8094b6@mail.gmail.com>

Hi John,

On Jan 11, 2008 4:52 AM, John O'Shea <joshea at nooked.com> wrote:
> hemant,
>         Thanks for the pointer, works well on HEAD.  For the benefit of
> others, the all_worker_info method returns an array of hashes (one for
> each worker) containing the following keys
> :worker (value is name)
> :job_key (value is an empty string if no job key assigned)
> :status (value seems to be hardcoded to 'running')

Hmm, the API has become a bit confusing. But I will try to clear it up
and I will be putting this up in READE too.

As you probably know, BackgrounDRb offers ability to store result
hashes from workers, so as even when a worker is died and no longer
running, you can query the results computed by it. This is immensely
useful.

MiddleMan.ask_status() method returns the result set for this
particular worker. Result set will be available even after a worker is
no longer running. Similarly, MiddleMan.query_all_workers is for
getting all available result sets in one go.

BackgrounDRb 1.0.1 offers ability to detect running workers and stuff
and worker_info methods are solely for that purpose.
For example:

MiddleMan.worker_info(:worker => :foo_worker)

Will return :status = :running if 'foo_worker' is running and will
return :status = :stopped if not running. I do not think apart from
these two values any other state can be possible for a worker and
hence its hardcoded.
Similarly MiddleMan.all_worker_info returns information about all
currently running workers.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From list at lylo.co.uk  Fri Jan 11 08:13:00 2008
From: list at lylo.co.uk (Olly Lylo)
Date: Fri, 11 Jan 2008 13:13:00 +0000
Subject: [Backgroundrb-devel] BackgrounDRb::PacketLogger issue
Message-ID: <f317a9f40801110513k659ba4e0y386478c998540cde@mail.gmail.com>

Hi there
I have recently upgraded to v1.0.1 and I've noticed an issue which I don't
believe was present in the 0.2 release.   In my worker class I am logging
errors as follows:

    logger.error "the error message"

However, this now causes an exception to be thrown in the new version:

  undefined method `error' for #<BackgrounDRb::PacketLogger:0xb6f60ed0>

Looking at the docs for PacketLogger (
http://backgroundrb.rubyforge.org/classes/BackgrounDRb/PacketLogger.html) it
seems that only debug and info have been defined.  Should I submit a patch
for adding 'warn', 'error' and 'fatal' so that this interface matches that
of the Ruby logger class?

Many thanks,
Olly
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080111/e0aab5ff/attachment-0001.html 

From gethemant at gmail.com  Fri Jan 11 09:34:22 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 11 Jan 2008 20:04:22 +0530
Subject: [Backgroundrb-devel] BackgrounDRb::PacketLogger issue
In-Reply-To: <f317a9f40801110513k659ba4e0y386478c998540cde@mail.gmail.com>
References: <f317a9f40801110513k659ba4e0y386478c998540cde@mail.gmail.com>
Message-ID: <fb283d5e0801110634j1bbb5fbavc4f016df44279ca9@mail.gmail.com>

Hi

On Jan 11, 2008 6:43 PM, Olly Lylo <list at lylo.co.uk> wrote:
> Hi there
>
> I have recently upgraded to v1.0.1 and I've noticed an issue which I don't
> believe was present in the 0.2 release.   In my worker class I am logging
> errors as follows:
>
>     logger.error "the error message"
>
> However, this now causes an exception to be thrown in the new version:
>
>   undefined method `error' for #<BackgrounDRb::PacketLogger:0xb6f60ed0>
>
> Looking at the docs for PacketLogger (
> http://backgroundrb.rubyforge.org/classes/BackgrounDRb/PacketLogger.html) it
> seems that only debug and info have been defined.  Should I submit a patch
> for adding 'warn', 'error' and 'fatal' so that this interface matches that
> of the Ruby logger class?
>

This is the current code:

http://svn.devjavu.com/backgroundrb/tags/release-1.0.1/server/meta_worker.rb

That RDoc is a bit old. Also, I have never really got around to
implement full logger functionality. If you can implement that and
submit the patch with proper test cases, it would be great.


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From tim at pivotib.com  Fri Jan 11 16:19:46 2008
From: tim at pivotib.com (Tim Glen)
Date: Fri, 11 Jan 2008 16:19:46 -0500
Subject: [Backgroundrb-devel] getting started
Message-ID: <3C0E2D79-1483-423D-B33A-949A923DA858@pivotib.com>

Hey all,

Been reading through all the old posts (and the docs) to familiarize  
myself but I still feel a bit lost.

I'm on trunk and am having a bit of a hard time knowing where to start  
with the particular I'm trying to solve.

So far, I've downloaded backgroundrb, done the basic rails setup,  
created my worker and started it using `script/backgroundrb start`.  
Here's the behaviour I'm looking to create:
1. When a file is uploaded, the worker needs to be triggered to start  
processing the file (this is a 20-60 minute job, depending on the file)
2. The user should be redirected to a page where they can see a status  
of the processing job (the ideal would be a percentage or status bar  
but I realize that's an implementation concern). most importantly,  
they need to know when it's been completed.

Here are my questions - since this only needs to take place every  
month or so, I don't really feel like I need a long-running worker  
necessarily. From the docs, it seemed like I could call:
MiddleMan.ask_work(:worker => :sales_processor, :worker_method  
=> :do_work, :data => @upload.id)
In the :do_work method, I'm calling register_status but am receiving  
nil when I call ask_status

I guess I just feel like I'm shooting in the dark here - not sure if  
i'm going in the right direction at all. Should I be using ask_work  
with register_status? Perhaps send_request (though I don't want the  
http request to wait for the worker to be completed)? Does the worker  
need to be running on a schedule before I can use either of these?

I would appreciate any direction.

thanks,
tim

From gethemant at gmail.com  Fri Jan 11 17:06:08 2008
From: gethemant at gmail.com (hemant)
Date: Sat, 12 Jan 2008 03:36:08 +0530
Subject: [Backgroundrb-devel] getting started
In-Reply-To: <3C0E2D79-1483-423D-B33A-949A923DA858@pivotib.com>
References: <3C0E2D79-1483-423D-B33A-949A923DA858@pivotib.com>
Message-ID: <fb283d5e0801111406r3d3aebc6o1f4366dda7efc45d@mail.gmail.com>

Hi Tim,

On Jan 12, 2008 2:49 AM, Tim Glen <tim at pivotib.com> wrote:
> Hey all,
>
> Been reading through all the old posts (and the docs) to familiarize
> myself but I still feel a bit lost.
>
> I'm on trunk and am having a bit of a hard time knowing where to start
> with the particular I'm trying to solve.
>
> So far, I've downloaded backgroundrb, done the basic rails setup,
> created my worker and started it using `script/backgroundrb start`.
> Here's the behaviour I'm looking to create:
> 1. When a file is uploaded, the worker needs to be triggered to start
> processing the file (this is a 20-60 minute job, depending on the file)
> 2. The user should be redirected to a page where they can see a status
> of the processing job (the ideal would be a percentage or status bar
> but I realize that's an implementation concern). most importantly,
> they need to know when it's been completed.
>
> Here are my questions - since this only needs to take place every
> month or so, I don't really feel like I need a long-running worker
> necessarily. From the docs, it seemed like I could call:
> MiddleMan.ask_work(:worker => :sales_processor, :worker_method
> => :do_work, :data => @upload.id)
> In the :do_work method, I'm calling register_status but am receiving
> nil when I call ask_status
>
> I guess I just feel like I'm shooting in the dark here - not sure if
> i'm going in the right direction at all. Should I be using ask_work
> with register_status? Perhaps send_request (though I don't want the
> http request to wait for the worker to be completed)? Does the worker
> need to be running on a schedule before I can use either of these?
>
> I would appreciate any direction.

Start your long running task like this:


MiddleMan.new_worker(:worker => :sales_processor_worker,:data => @upload.id)
# I think, you need _worker there, just check the name thats there in
worker class. Whatever name is used
# there, you must use it here too.

Now in sales_processor_worker

class SalesProcessorWorker < BackgrunDRb::MetaWorker
  set_worker_name :sales_processor_worker
  set_no_auto_load true
  # whatever you passed as an :data is available here as an argument
  def create(args = nil)
    register_status("Processing started")
    # do some more processing here
  end
end

>From controller get status like this:

MiddleMan.ask_status(:worker => :sales_processor_worker)

Thats about it. The key is, if you don't need a perpetually running
worker around, disable auto loading of worker and start your worker
using MiddleMan.new_worker and then query status. Hope this helps.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From m.heigl at gmx.net  Sun Jan 13 13:21:05 2008
From: m.heigl at gmx.net (Matthias Heigl)
Date: Sun, 13 Jan 2008 19:21:05 +0100
Subject: [Backgroundrb-devel] {worker} after :end-time worker freaks out
Message-ID: <478A5691.4070704@gmx.net>

Hi all,

i was using bdrb a while ago and it wasn't stable enough in version 
0.2.1 for doing the job - it loss jobs or did execute only the first one 
invoked and so on.

Now i gave the new version a try and i found a bug at playing around.

When i start a worker_method over the normal Unix scheduler bdrb will 
start as much as possible the worker_method after the :end-time is reached.

I verifyed the behavior on two different boxes:

1. Mac OS X 10.5.1

2. Debian Lenny

Both using Rub 1.8.6 an backgroundrb 1.0.

My Test-Files

backgroundrb.yml
- - - - - -
:backgroundrb:
   :port: 11006
   :ip: localhost
   :environment: development
:schedules:
   :mytest_worker:
     :doit:
       :trigger_args:
         :start: <%= Time.now + 1.seconds %>
         :end: <%= Time.now + 1.minute %>
         :repeat_interval: <%= 10.seconds %>
- - - - - -

mytest_worker.rb
- - - - - -
class MytestWorker < BackgrounDRb::MetaWorker
   set_worker_name :mytest_worker
   def create(args = nil)
     # this method is called, when worker is loaded for the first time
     @counter=0
   end
   def doit
     @counter += 1
     logger.debug("mytest: doit #{@counter} - #{Time.now}")
   end
end
- - - - - -

After i start bdrb with ./script/backgroundrb start my backgroundrb.log 
looks like this:
- - - - - -
mytest_worker started
Schedules for worker loaded
mytest: doit 1 - Sun Jan 13 19:07:19 +0100 2008
mytest: doit 2 - Sun Jan 13 19:07:29 +0100 2008
mytest: doit 3 - Sun Jan 13 19:07:39 +0100 2008
mytest: doit 4 - Sun Jan 13 19:07:49 +0100 2008
mytest: doit 5 - Sun Jan 13 19:07:59 +0100 2008
mytest: doit 6 - Sun Jan 13 19:08:09 +0100 2008
mytest: doit 7 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 8 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 9 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 10 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 11 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 12 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 13 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 14 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 15 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 16 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 17 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 18 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 19 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 20 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 21 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 22 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 23 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 24 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 25 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 26 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 27 - Sun Jan 13 19:08:19 +0100 2008
mytest: doit 28 - Sun Jan 13 19:08:19 +0100 2008
... (about 180 per second)
- - - - - -

For my projekt i didn't need the Unix scheduler style, but while testing 
if the new version will meet the requirements i noticed this.

Maybe i did something wrong, but in the version 0.2.1 the :end-time 
works - i think.

My Question about best practise will be part of my next mail to keep 
apart the issues.

Cheers,

Matze

From gethemant at gmail.com  Sun Jan 13 15:03:24 2008
From: gethemant at gmail.com (hemant kumar)
Date: Mon, 14 Jan 2008 01:33:24 +0530
Subject: [Backgroundrb-devel] {worker} after :end-time worker freaks out
In-Reply-To: <478A5691.4070704@gmx.net>
References: <478A5691.4070704@gmx.net>
Message-ID: <1200254604.13854.1.camel@shire>

Hi Matthias,

On Sun, 2008-01-13 at 19:21 +0100, Matthias Heigl wrote:
> Hi all,
> 
> i was using bdrb a while ago and it wasn't stable enough in version 
> 0.2.1 for doing the job - it loss jobs or did execute only the first one 
> invoked and so on.
> 
> Now i gave the new version a try and i found a bug at playing around.
> 
> When i start a worker_method over the normal Unix scheduler bdrb will 
> start as much as possible the worker_method after the :end-time is reached.
> 
> I verifyed the behavior on two different boxes:
> 
> 1. Mac OS X 10.5.1
> 
> 2. Debian Lenny
> 
> Both using Rub 1.8.6 an backgroundrb 1.0.
> 
> My Test-Files
> 
> backgroundrb.yml
> - - - - - -
> :backgroundrb:
>    :port: 11006
>    :ip: localhost
>    :environment: development
> :schedules:
>    :mytest_worker:
>      :doit:
>        :trigger_args:
>          :start: <%= Time.now + 1.seconds %>
>          :end: <%= Time.now + 1.minute %>
>          :repeat_interval: <%= 10.seconds %>
> - - - - - -
> 
> mytest_worker.rb
> - - - - - -
> class MytestWorker < BackgrounDRb::MetaWorker
>    set_worker_name :mytest_worker
>    def create(args = nil)
>      # this method is called, when worker is loaded for the first time
>      @counter=0
>    end
>    def doit
>      @counter += 1
>      logger.debug("mytest: doit #{@counter} - #{Time.now}")
>    end
> end
> - - - - - -
> 
> After i start bdrb with ./script/backgroundrb start my backgroundrb.log 
> looks like this:
> - - - - - -
> mytest_worker started
> Schedules for worker loaded
> mytest: doit 1 - Sun Jan 13 19:07:19 +0100 2008
> mytest: doit 2 - Sun Jan 13 19:07:29 +0100 2008
> mytest: doit 3 - Sun Jan 13 19:07:39 +0100 2008
> mytest: doit 4 - Sun Jan 13 19:07:49 +0100 2008
> mytest: doit 5 - Sun Jan 13 19:07:59 +0100 2008
> mytest: doit 6 - Sun Jan 13 19:08:09 +0100 2008
> mytest: doit 7 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 8 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 9 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 10 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 11 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 12 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 13 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 14 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 15 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 16 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 17 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 18 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 19 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 20 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 21 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 22 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 23 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 24 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 25 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 26 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 27 - Sun Jan 13 19:08:19 +0100 2008
> mytest: doit 28 - Sun Jan 13 19:08:19 +0100 2008
> ... (about 180 per second)
> - - - - - -
> 
> For my projekt i didn't need the Unix scheduler style, but while testing 
> if the new version will meet the requirements i noticed this.
> 
> Maybe i did something wrong, but in the version 0.2.1 the :end-time 
> works - i think.
> 
> My Question about best practise will be part of my next mail to keep 
> apart the issues.
> 
> Cheers,

Yes, I am able to reproduce the error and hence I have fixed it in trunk as well as 1.0.1 release.

I will suggest you to use 1.0.1 release since it contains many fixes over 1.0 release:

http://svn.devjavu.com/backgroundrb/tags/release-1.0.1/




From m.heigl at gmx.net  Sun Jan 13 18:19:43 2008
From: m.heigl at gmx.net (Matthias Heigl)
Date: Mon, 14 Jan 2008 00:19:43 +0100
Subject: [Backgroundrb-devel] right usage of bdrb
Message-ID: <478A9C8F.1070907@gmx.net>

Hi,

i'm going to implement a syndication-service, which will get lists in 
xml with some meta-data an enclosed video files, which will get encoded 
   at the end. The syndication run will be startet every five minutes of 
a full hour.

So i thought to build 4 Worker. One for checking which feeds to 
syndicate (syndication_worker) at a specific time, one for processing 
the list (import_worker), one for fetching the clips (download_worker) 
and one for encoding (encoding) worker.

In my tests all went fine, all my jobs are invoked properly.

Q1)
Is this procedure right in the intention of backgroundrb?
I meen, there will be at worst up to about hundret of download_worker 
started in one syndication run. Is bdrb able to handle such a queue, 
without to lose one job?

Q2)
Some downloads are really slow, so i would rather start about 5 
downloads in parallel and queue the encoding in one queue again.

                    syndication
                         |
                       import
        /        /       |        \        \
    download download download download download
        \        \       |        /        /
                      encoder

Is this possible?

I know i can give another job_key, maybe the name of the content-source, 
but this can go up to 20 or more download-queues in one syndication run 
and this is too much.

Thanks in advance!

Cheers,

Matze


------------------------------------------------

backgroundrb.yml
- - - - - - -
:backgroundrb:
   :ip: 0.0.0.0
   :port: 11006
   :environment: production
:schedules:
   :syndication_worker:
     :checksyndicate:
       :trigger_args: 0 5 * * * * *
- - - - - - -

syndication_worker.rb
- - - - - - -
class SyndicationWorker < BackgrounDRb::MetaWorker
   set_worker_name :syndication_worker
   def create(args = nil)
     # this method is called, when worker is loaded for the first time
   end
   def checksyndicate
     syndications = # all Syndication in this hour
     syndications.each do |syndication|
       MiddleMan.ask_work(:worker => import_worker, :worker_method => 
import, :data => syndication.id)
     end
   end
- - - - - - -

import_worker.rb
- - - - - - -
class ImportWorker < BackgrounDRb::MetaWorker
   set_worker_name :import_worker
   def create(args = nil)
     # this method is called, when worker is loaded for the first time
   end
   def import(feedid)
     feed = Feed.find_by_id(feedid)
     # fetch feed
     feed.items.each do |item|
       # create item
       MiddleMan.ask_work(:worker => download_worker, :worker_method => 
download, :data => item.id)
     end
   end
- - - - - - -

download_worker.rb
- - - - - - -
class DownloadWorker < BackgrounDRb::MetaWorker
   set_worker_name :download_worker
   def create(args = nil)
     # this method is called, when worker is loaded for the first time
   end
   def download(itemid)
     item = Item.find_by_id(:itemid)
     # fetch enclosure
     MiddleMan.ask_work(:worker => encoder_worker, :worker_method => 
encode, :data => item.id)
   end
- - - - - - -

encoder_worker.rb
- - - - - - -
class EncoderWorker < BackgrounDRb::MetaWorker
   set_worker_name :encoder_worker
   def create(args = nil)
     # this method is called, when worker is loaded for the first time
   end
   def download(itemid)
     item = Item.find_by_id(:itemid)
     # encode item
   end
   - - - - - -

From gethemant at gmail.com  Sun Jan 13 22:22:12 2008
From: gethemant at gmail.com (hemant kumar)
Date: Mon, 14 Jan 2008 08:52:12 +0530
Subject: [Backgroundrb-devel] right usage of bdrb
In-Reply-To: <478A9C8F.1070907@gmx.net>
References: <478A9C8F.1070907@gmx.net>
Message-ID: <1200280932.13854.35.camel@shire>

Hi Matthias,

On Mon, 2008-01-14 at 00:19 +0100, Matthias Heigl wrote:
> Hi,
> 
> i'm going to implement a syndication-service, which will get lists in 
> xml with some meta-data an enclosed video files, which will get encoded 
>    at the end. The syndication run will be startet every five minutes of 
> a full hour.
> 
> So i thought to build 4 Worker. One for checking which feeds to 
> syndicate (syndication_worker) at a specific time, one for processing 
> the list (import_worker), one for fetching the clips (download_worker) 
> and one for encoding (encoding) worker.
> 
> In my tests all went fine, all my jobs are invoked properly.
> 
> Q1)
> Is this procedure right in the intention of backgroundrb?
> I meen, there will be at worst up to about hundret of download_worker 
> started in one syndication run. Is bdrb able to handle such a queue, 
> without to lose one job?
> 
> Q2)
> Some downloads are really slow, so i would rather start about 5 
> downloads in parallel and queue the encoding in one queue again.
> 
>                     syndication
>                          |
>                        import
>         /        /       |        \        \
>     download download download download download
>         \        \       |        /        /
>                       encoder
> 
> Is this possible?

Yup, but let me suggest an alternative architecture and see for yourself
if this will work better or not. Here is the deal:

class SyndicationWorker < BackgrounDRb::MetaWorker
  def create(args = nil)
    add_periodic_timer(300) { search_for_syndication }
  end
  def search_for_syndication
    # when you are saving syndications which are to be imported 
    # downloaded and encoded. Rather than calling another worker
    # from here, you should have a flag for each Feed, and each
    # item indicating that this feed needs to be imported, 
    # downloaded and encoded
    syndications
  end
end

class ImportWorker < BackgrounDRb::MetaWorker
  def create(args = nil)
    add_periodic_timer(300) { import_syndication }
  end
  def import_syndication
     new_feeds = Feed.find(:all,:conditions => {:to_be_imported => \ 
            true })
     new_feeds.each do |feed|
       # fetch the feed and have two flags in associated item
       # indicating item is to be downloaded and encoded
       fetch_feed(feed)
       # update the 'to_be_imported' flag for given feed
       # generally, we should prefer all this logic to go into
       # model itself ( remember thick model, thin controllers)
       # but I am writing here for demonstration
       feed.to_be_imported = false
       feed.save
     end
  end
end

class DownloadWorker < BackgrounDRb::MetaWorker
  # assuming you want 5 downloads to go concurrently
  # I am setting thread pool size of 5
  # default pool size is of 20 threads
  pool_size 5
  def create(args = nil)
    add_periodic_timer(300) { download_syndication }
  end
  def download_syndication
    new_items = Item.find(:conditions => {:to_be_downloaded => \
       true})
    new_items.each do |item_id|
      thread_pool.defer(item_id) do |item_id|
        # using item_id, rather than item itself for
        # thread local objects, with underlying db 
        # connection of their own.
        item = Item.find_by_id(item_id)
        download_video(item)
        # update the to_be_downloaded flag for item.
      end
    end
  end
end

class EncodeWorker < BackgrounDRb::MetaWorker
  def create(args = nil)
    add_periodic_timer(300) { encode_video }
  end
  def encode_video
    new_encodes = Item.find(:conditions => {:to_be_encoded => \
      true })
    new_encodes.each do |item|
      encode_item(item)
      # update the to_be_encoded flag and save the model
    end
  end
end


I guess, above architecture will work quite well, and I am avoiding
inter dependence in workers and rather using table itself as a queue. It
has multiple advantages that way. Tables are persistent, so you will
always know, which feeds are to be fetched, downloaded and encoded and
which are already done. 

> 
> I know i can give another job_key, maybe the name of the content-source, 
> but this can go up to 20 or more download-queues in one syndication run 
> and this is too much.
> 
> Thanks in advance!
> 
> Cheers,
> 
> Matze
> 
> 
> ------------------------------------------------
> 
> backgroundrb.yml
> - - - - - - -
> :backgroundrb:
>    :ip: 0.0.0.0
>    :port: 11006
>    :environment: production
> :schedules:
>    :syndication_worker:
>      :checksyndicate:
>        :trigger_args: 0 5 * * * * *
> - - - - - - -
> 
> syndication_worker.rb
> - - - - - - -
> class SyndicationWorker < BackgrounDRb::MetaWorker
>    set_worker_name :syndication_worker
>    def create(args = nil)
>      # this method is called, when worker is loaded for the first time
>    end
>    def checksyndicate
>      syndications = # all Syndication in this hour
>      syndications.each do |syndication|
>        MiddleMan.ask_work(:worker => import_worker, :worker_method => 
> import, :data => syndication.id)
>      end
>    end
> - - - - - - -
> 
> import_worker.rb
> - - - - - - -
> class ImportWorker < BackgrounDRb::MetaWorker
>    set_worker_name :import_worker
>    def create(args = nil)
>      # this method is called, when worker is loaded for the first time
>    end
>    def import(feedid)
>      feed = Feed.find_by_id(feedid)
>      # fetch feed
>      feed.items.each do |item|
>        # create item
>        MiddleMan.ask_work(:worker => download_worker, :worker_method => 
> download, :data => item.id)
>      end
>    end
> - - - - - - -
> 
> download_worker.rb
> - - - - - - -
> class DownloadWorker < BackgrounDRb::MetaWorker
>    set_worker_name :download_worker
>    def create(args = nil)
>      # this method is called, when worker is loaded for the first time
>    end
>    def download(itemid)
>      item = Item.find_by_id(:itemid)
>      # fetch enclosure
>      MiddleMan.ask_work(:worker => encoder_worker, :worker_method => 
> encode, :data => item.id)
>    end
> - - - - - - -
> 
> encoder_worker.rb
> - - - - - - -
> class EncoderWorker < BackgrounDRb::MetaWorker
>    set_worker_name :encoder_worker
>    def create(args = nil)
>      # this method is called, when worker is loaded for the first time
>    end
>    def download(itemid)
>      item = Item.find_by_id(:itemid)
>      # encode item
>    end
>    - - - - - -
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel


From gethemant at gmail.com  Sun Jan 13 22:56:46 2008
From: gethemant at gmail.com (hemant kumar)
Date: Mon, 14 Jan 2008 09:26:46 +0530
Subject: [Backgroundrb-devel] right usage of bdrb
In-Reply-To: <1200280932.13854.35.camel@shire>
References: <478A9C8F.1070907@gmx.net>  <1200280932.13854.35.camel@shire>
Message-ID: <1200283006.13854.46.camel@shire>

Hi Matthias,

On Mon, 2008-01-14 at 08:52 +0530, hemant kumar wrote:
> Hi Matthias,
> Yup, but let me suggest an alternative architecture and see for yourself
> if this will work better or not. Here is the deal:
> 
> class SyndicationWorker < BackgrounDRb::MetaWorker
>   def create(args = nil)
>     add_periodic_timer(300) { search_for_syndication }
>   end
>   def search_for_syndication
>     # when you are saving syndications which are to be imported 
>     # downloaded and encoded. Rather than calling another worker
>     # from here, you should have a flag for each Feed, and each
>     # item indicating that this feed needs to be imported, 
>     # downloaded and encoded
>     syndications
>   end
> end
> 
> class ImportWorker < BackgrounDRb::MetaWorker
>   def create(args = nil)
>     add_periodic_timer(300) { import_syndication }
>   end
>   def import_syndication
>      new_feeds = Feed.find(:all,:conditions => {:to_be_imported => \ 
>             true })
>      new_feeds.each do |feed|
>        # fetch the feed and have two flags in associated item
>        # indicating item is to be downloaded and encoded
>        fetch_feed(feed)
>        # update the 'to_be_imported' flag for given feed
>        # generally, we should prefer all this logic to go into
>        # model itself ( remember thick model, thin controllers)
>        # but I am writing here for demonstration
>        feed.to_be_imported = false
>        feed.save
>      end
>   end
> end
> 
> class DownloadWorker < BackgrounDRb::MetaWorker
>   # assuming you want 5 downloads to go concurrently
>   # I am setting thread pool size of 5
>   # default pool size is of 20 threads
>   pool_size 5
>   def create(args = nil)
>     add_periodic_timer(300) { download_syndication }
>   end
>   def download_syndication
>     new_items = Item.find(:conditions => {:to_be_downloaded => \
>        true})
>     new_items.each do |item_id|
>       thread_pool.defer(item_id) do |item_id|
>         # using item_id, rather than item itself for
>         # thread local objects, with underlying db 
>         # connection of their own.
>         item = Item.find_by_id(item_id)
>         download_video(item)
>         # update the to_be_downloaded flag for item.
>       end
>     end
>   end
> end
> 
> class EncodeWorker < BackgrounDRb::MetaWorker
>   def create(args = nil)
>     add_periodic_timer(300) { encode_video }
>   end
>   def encode_video
>     new_encodes = Item.find(:conditions => {:to_be_encoded => \
>       true })
>     new_encodes.each do |item|
>       encode_item(item)
>       # update the to_be_encoded flag and save the model
>     end
>   end
> end
> 
> 
> I guess, above architecture will work quite well, and I am avoiding
> inter dependence in workers and rather using table itself as a queue. It
> has multiple advantages that way. Tables are persistent, so you will
> always know, which feeds are to be fetched, downloaded and encoded and
> which are already done. 
> 

Okay, I digress. Above architecture will not work well, when you will
have multiple copies of same workers running parallely.


So, you can use your existing code, with thing I suggested for download
worker. Also, if you want to queue jobs invoked with ask_work, I will
suggest you to use defer. What I am saying is:

class ImportWorker < BackgrounDRb::MetaWorker
  pool_size 1
  def import(feed_id)
    thread_pool.defer { |b_feed_id| start_import(t_feed_it) }
  end
end

And invoke above job as usual:

MiddleMan.ask_work(:worker => :import_worker, :worker_method
=> :import, :data => feed_id)

#defer will make sure that, your task is queued and having thread pool
size of 1, makes sure that only one job is running concurrently.





From m.heigl at gmx.net  Mon Jan 14 18:02:52 2008
From: m.heigl at gmx.net (Matthias Heigl)
Date: Tue, 15 Jan 2008 00:02:52 +0100
Subject: [Backgroundrb-devel] {worker} after :end-time worker freaks out
In-Reply-To: <1200254604.13854.1.camel@shire>
References: <478A5691.4070704@gmx.net> <1200254604.13854.1.camel@shire>
Message-ID: <478BEA1C.4010200@gmx.net>

hemant kumar schrieb:
> Hi Matthias,
> 
> On Sun, 2008-01-13 at 19:21 +0100, Matthias Heigl wrote:
>> Hi all,
>>
>> i was using bdrb a while ago and it wasn't stable enough in version 
>> 0.2.1 for doing the job - it loss jobs or did execute only the first one 
>> invoked and so on.
>>
>> Now i gave the new version a try and i found a bug at playing around.
>>
>> When i start a worker_method over the normal Unix scheduler bdrb will 
>> start as much as possible the worker_method after the :end-time is reached.
>>
>> I verifyed the behavior on two different boxes:
>>
>> 1. Mac OS X 10.5.1
>>
>> 2. Debian Lenny
>>
>> Both using Rub 1.8.6 an backgroundrb 1.0.
>>
>> My Test-Files
>>
>> backgroundrb.yml
>> - - - - - -
>> :backgroundrb:
>>    :port: 11006
>>    :ip: localhost
>>    :environment: development
>> :schedules:
>>    :mytest_worker:
>>      :doit:
>>        :trigger_args:
>>          :start: <%= Time.now + 1.seconds %>
>>          :end: <%= Time.now + 1.minute %>
>>          :repeat_interval: <%= 10.seconds %>
>> - - - - - -
>>
>> mytest_worker.rb
>> - - - - - -
>> class MytestWorker < BackgrounDRb::MetaWorker
>>    set_worker_name :mytest_worker
>>    def create(args = nil)
>>      # this method is called, when worker is loaded for the first time
>>      @counter=0
>>    end
>>    def doit
>>      @counter += 1
>>      logger.debug("mytest: doit #{@counter} - #{Time.now}")
>>    end
>> end
>> - - - - - -
>>
>> After i start bdrb with ./script/backgroundrb start my backgroundrb.log 
>> looks like this:
>> - - - - - -
>> mytest_worker started
>> Schedules for worker loaded
>> mytest: doit 1 - Sun Jan 13 19:07:19 +0100 2008
>> mytest: doit 2 - Sun Jan 13 19:07:29 +0100 2008
>> mytest: doit 3 - Sun Jan 13 19:07:39 +0100 2008
>> mytest: doit 4 - Sun Jan 13 19:07:49 +0100 2008
>> mytest: doit 5 - Sun Jan 13 19:07:59 +0100 2008
>> mytest: doit 6 - Sun Jan 13 19:08:09 +0100 2008
>> mytest: doit 7 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 8 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 9 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 10 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 11 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 12 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 13 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 14 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 15 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 16 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 17 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 18 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 19 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 20 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 21 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 22 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 23 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 24 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 25 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 26 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 27 - Sun Jan 13 19:08:19 +0100 2008
>> mytest: doit 28 - Sun Jan 13 19:08:19 +0100 2008
>> ... (about 180 per second)
>> - - - - - -
>>
>> For my projekt i didn't need the Unix scheduler style, but while testing 
>> if the new version will meet the requirements i noticed this.
>>
>> Maybe i did something wrong, but in the version 0.2.1 the :end-time 
>> works - i think.
>>
>> My Question about best practise will be part of my next mail to keep 
>> apart the issues.
>>
>> Cheers,
> 
> Yes, I am able to reproduce the error and hence I have fixed it in trunk as well as 1.0.1 release.
> 
> I will suggest you to use 1.0.1 release since it contains many fixes over 1.0 release:
> 
> http://svn.devjavu.com/backgroundrb/tags/release-1.0.1/
> 

I installed the 1.0.1 release and everything looks fine now.

This fix was really quick!

Cheers,

Matze


From m.heigl at gmx.net  Mon Jan 14 18:12:32 2008
From: m.heigl at gmx.net (Matthias Heigl)
Date: Tue, 15 Jan 2008 00:12:32 +0100
Subject: [Backgroundrb-devel] right usage of bdrb
In-Reply-To: <1200283006.13854.46.camel@shire>
References: <478A9C8F.1070907@gmx.net> <1200280932.13854.35.camel@shire>
	<1200283006.13854.46.camel@shire>
Message-ID: <478BEC60.3000805@gmx.net>

hemant kumar schrieb:
> Hi Matthias,
> 
> On Mon, 2008-01-14 at 08:52 +0530, hemant kumar wrote:
>> Hi Matthias,
>> Yup, but let me suggest an alternative architecture and see for yourself
>> if this will work better or not. Here is the deal:
>>
>> class SyndicationWorker < BackgrounDRb::MetaWorker
>>   def create(args = nil)
>>     add_periodic_timer(300) { search_for_syndication }
>>   end
>>   def search_for_syndication
>>     # when you are saving syndications which are to be imported 
>>     # downloaded and encoded. Rather than calling another worker
>>     # from here, you should have a flag for each Feed, and each
>>     # item indicating that this feed needs to be imported, 
>>     # downloaded and encoded
>>     syndications
>>   end
>> end
>>
>> class ImportWorker < BackgrounDRb::MetaWorker
>>   def create(args = nil)
>>     add_periodic_timer(300) { import_syndication }
>>   end
>>   def import_syndication
>>      new_feeds = Feed.find(:all,:conditions => {:to_be_imported => \ 
>>             true })
>>      new_feeds.each do |feed|
>>        # fetch the feed and have two flags in associated item
>>        # indicating item is to be downloaded and encoded
>>        fetch_feed(feed)
>>        # update the 'to_be_imported' flag for given feed
>>        # generally, we should prefer all this logic to go into
>>        # model itself ( remember thick model, thin controllers)
>>        # but I am writing here for demonstration
>>        feed.to_be_imported = false
>>        feed.save
>>      end
>>   end
>> end
>>
>> class DownloadWorker < BackgrounDRb::MetaWorker
>>   # assuming you want 5 downloads to go concurrently
>>   # I am setting thread pool size of 5
>>   # default pool size is of 20 threads
>>   pool_size 5
>>   def create(args = nil)
>>     add_periodic_timer(300) { download_syndication }
>>   end
>>   def download_syndication
>>     new_items = Item.find(:conditions => {:to_be_downloaded => \
>>        true})
>>     new_items.each do |item_id|
>>       thread_pool.defer(item_id) do |item_id|
>>         # using item_id, rather than item itself for
>>         # thread local objects, with underlying db 
>>         # connection of their own.
>>         item = Item.find_by_id(item_id)
>>         download_video(item)
>>         # update the to_be_downloaded flag for item.
>>       end
>>     end
>>   end
>> end
>>
>> class EncodeWorker < BackgrounDRb::MetaWorker
>>   def create(args = nil)
>>     add_periodic_timer(300) { encode_video }
>>   end
>>   def encode_video
>>     new_encodes = Item.find(:conditions => {:to_be_encoded => \
>>       true })
>>     new_encodes.each do |item|
>>       encode_item(item)
>>       # update the to_be_encoded flag and save the model
>>     end
>>   end
>> end
>>
>>
>> I guess, above architecture will work quite well, and I am avoiding
>> inter dependence in workers and rather using table itself as a queue. It
>> has multiple advantages that way. Tables are persistent, so you will
>> always know, which feeds are to be fetched, downloaded and encoded and
>> which are already done. 
>>
> 
> Okay, I digress. Above architecture will not work well, when you will
> have multiple copies of same workers running parallely.
> 
> 
> So, you can use your existing code, with thing I suggested for download
> worker. Also, if you want to queue jobs invoked with ask_work, I will
> suggest you to use defer. What I am saying is:
> 
> class ImportWorker < BackgrounDRb::MetaWorker
>   pool_size 1
>   def import(feed_id)
>     thread_pool.defer { |b_feed_id| start_import(t_feed_it) }
>   end
> end
> 
> And invoke above job as usual:
> 
> MiddleMan.ask_work(:worker => :import_worker, :worker_method
> => :import, :data => feed_id)
> 
> #defer will make sure that, your task is queued and having thread pool
> size of 1, makes sure that only one job is running concurrently.
> 

Thanks for the tipp, i will give thread_pool a try.

The only problem using the periodic_timer is that it's possibly taken 20 
minutes the first encoding starts under some circumstances, but it 
really will be reliable even if the process crashes or something else 
went wrong.

If it takes too long if a new content source is added by a content 
manager, so my telephone will ring "is the import still running?" :)

Thanks for your quick reply!

Matze

From clint at ctro.net  Tue Jan 15 18:31:34 2008
From: clint at ctro.net (Clint Troxel)
Date: Tue, 15 Jan 2008 16:31:34 -0700
Subject: [Backgroundrb-devel] actionmailer the background
Message-ID: <81CE4428-DA3E-4FD6-B496-88F7AE8F8ED0@ctro.net>

I have bdrb successfully set up and running in a local project.  It's  
doing two things:

1) processing large csv files and using ActiveRecord to insert
2) sending large batches of emails using ActionMailer

In development both work well.  I see my ActionMailer emails in the  
log (though it's the development.log -- odd?)

On the production server csv processing works well, but sending emails  
throws errors in the background.

Note:  I can successfully upload a csv, then fail trying to send  
emails, then successfully upload another csv file.  Also, sending  
emails works in production when used outside of backgroundrb.

backgroundrb_debug.log reports this

PGError: no connection to the server
: SELECT a.attname, format_type(a.atttypid, a.atttypmod), d.adsrc,  
a.attnotnull
FROM pg_attribute a LEFT JOIN pg_attrdef d
ON a.attrelid = d.adrelid AND a.attnum = d.adnum
WHERE a.attrelid = 'sandboxed_libraries'::regclass
AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum

/usr/local/projects/ecera/vendor/rails/activerecord/lib/active_record/ 
connection_adapters/abstract_adapter.rb:128:in `log'
/usr/local/projects/ecera/vendor/rails/activerecord/lib/active_record/ 
connection_adapters/postgresql_adapter.rb:142:in `query'
/usr/local/projects/ecera/vendor/rails/activerecord/lib/active_record/ 
connection_adapters/postgresql_adapter.rb:528:in `column_definitions'
/usr/local/projects/ecera/vendor/rails/activerecord/lib/active_record/ 
connection_adapters/postgresql_adapter.rb:223:in `columns'

Can't seem to turn anything up after a day of googling.
Does this ring any bells?

Thanks,
Clint

From gethemant at gmail.com  Tue Jan 15 19:22:38 2008
From: gethemant at gmail.com (hemant kumar)
Date: Wed, 16 Jan 2008 05:52:38 +0530
Subject: [Backgroundrb-devel] actionmailer the background
In-Reply-To: <81CE4428-DA3E-4FD6-B496-88F7AE8F8ED0@ctro.net>
References: <81CE4428-DA3E-4FD6-B496-88F7AE8F8ED0@ctro.net>
Message-ID: <1200442958.6652.10.camel@shire>

Hi Clint,

Which version of BackgrounDRb plugin you are using? Please use trunk or
1.0.1 release tag.

Looks like your database is dropping connection. Also, you need to set,
production environment through backgroundrb.yml file, so as production
database configuration is loaded in production servers.

Something like this:

#backgroundrb.yml
:backgroundrb:
  :port: 11006
  :ip: 0.0.0.0
  :environment: production


Hope this helps.


On Tue, 2008-01-15 at 16:31 -0700, Clint Troxel wrote:
> I have bdrb successfully set up and running in a local project.  It's  
> doing two things:
> 
> 1) processing large csv files and using ActiveRecord to insert
> 2) sending large batches of emails using ActionMailer
> 
> In development both work well.  I see my ActionMailer emails in the  
> log (though it's the development.log -- odd?)
> 
> On the production server csv processing works well, but sending emails  
> throws errors in the background.
> 
> Note:  I can successfully upload a csv, then fail trying to send  
> emails, then successfully upload another csv file.  Also, sending  
> emails works in production when used outside of backgroundrb.
> 
> backgroundrb_debug.log reports this
> 
> PGError: no connection to the server
> : SELECT a.attname, format_type(a.atttypid, a.atttypmod), d.adsrc,  
> a.attnotnull
> FROM pg_attribute a LEFT JOIN pg_attrdef d
> ON a.attrelid = d.adrelid AND a.attnum = d.adnum
> WHERE a.attrelid = 'sandboxed_libraries'::regclass
> AND a.attnum > 0 AND NOT a.attisdropped
> ORDER BY a.attnum
> 
> /usr/local/projects/ecera/vendor/rails/activerecord/lib/active_record/ 
> connection_adapters/abstract_adapter.rb:128:in `log'
> /usr/local/projects/ecera/vendor/rails/activerecord/lib/active_record/ 
> connection_adapters/postgresql_adapter.rb:142:in `query'
> /usr/local/projects/ecera/vendor/rails/activerecord/lib/active_record/ 
> connection_adapters/postgresql_adapter.rb:528:in `column_definitions'
> /usr/local/projects/ecera/vendor/rails/activerecord/lib/active_record/ 
> connection_adapters/postgresql_adapter.rb:223:in `columns'
> 
> Can't seem to turn anything up after a day of googling.
> Does this ring any bells?
> 
> Thanks,
> Clint
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel


From mengkuan at gmail.com  Wed Jan 16 03:27:55 2008
From: mengkuan at gmail.com (Meng Kuan)
Date: Wed, 16 Jan 2008 16:27:55 +0800
Subject: [Backgroundrb-devel] mysterious crash of a particular worker
Message-ID: <4345930A-FC93-4638-B2A4-ACFF2B35761E@gmail.com>

Hi,

I am using the latest checkout from backgroundrb release 1.0.1. I  
have a worker called status_checker that periodically (every minute)  
checks the status of certain hosts over the network. It works fine at  
the beginning but after a while the worker will mysteriously  
disappear and stop working. I have other workers running but they do  
not disappear like this worker does.

These are the logs generated from logger.info calls for this  
particular worker.

2008-01-16 15:55 StatusChecker: check method started
2008-01-16 15:55 StatusChecker: processing host head0
2008-01-16 15:55 StatusChecker: processing domain webconsole
2008-01-16 15:55 StatusChecker: processing host host1
2008-01-16 15:55 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 15:55 StatusChecker: check method completed
2008-01-16 15:56 StatusChecker: check method started
2008-01-16 15:56 StatusChecker: processing host head0
2008-01-16 15:56 StatusChecker: processing domain webconsole
2008-01-16 15:56 StatusChecker: processing host host1
2008-01-16 15:56 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 15:56 StatusChecker: check method completed
2008-01-16 15:57 StatusChecker: check method started
2008-01-16 15:57 StatusChecker: processing host head0
2008-01-16 15:57 StatusChecker: processing domain webconsole
2008-01-16 15:57 StatusChecker: processing host host1
2008-01-16 15:57 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 15:57 StatusChecker: check method completed
2008-01-16 15:58 StatusChecker: check method started
2008-01-16 15:58 StatusChecker: processing host head0
2008-01-16 15:58 StatusChecker: processing domain webconsole
2008-01-16 15:58 StatusChecker: processing host host1
2008-01-16 15:58 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 15:58 StatusChecker: check method completed
2008-01-16 15:59 StatusChecker: check method started
2008-01-16 15:59 StatusChecker: processing host head0
2008-01-16 15:59 StatusChecker: processing domain webconsole
2008-01-16 15:59 StatusChecker: processing host host1
2008-01-16 15:59 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 15:59 StatusChecker: check method completed
2008-01-16 16:00 StatusChecker: check method started
2008-01-16 16:00 StatusChecker: processing host head0
2008-01-16 16:00 StatusChecker: processing domain webconsole
2008-01-16 16:00 StatusChecker: processing host host1
2008-01-16 16:00 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 16:00 StatusChecker: check method completed
2008-01-16 16:01 StatusChecker: check method started
2008-01-16 16:01 StatusChecker: processing host head0
2008-01-16 16:01 StatusChecker: processing domain webconsole
2008-01-16 16:01 StatusChecker: processing host host1
2008-01-16 16:01 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 16:01 StatusChecker: check method completed
2008-01-16 16:02 StatusChecker: check method started
2008-01-16 16:02 StatusChecker: processing host head0
2008-01-16 16:02 StatusChecker: processing domain webconsole
2008-01-16 16:02 StatusChecker: processing host host1
2008-01-16 16:02 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 16:02 StatusChecker: check method completed
2008-01-16 16:03 StatusChecker: check method started
2008-01-16 16:03 StatusChecker: processing host head0
2008-01-16 16:03 StatusChecker: processing domain webconsole
2008-01-16 16:03 StatusChecker: processing host host1
2008-01-16 16:03 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 16:03 StatusChecker: check method completed
2008-01-16 16:04 StatusChecker: check method started
2008-01-16 16:04 StatusChecker: processing host head0
2008-01-16 16:04 StatusChecker: processing domain webconsole
2008-01-16 16:04 StatusChecker: processing host host1
2008-01-16 16:04 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 16:04 StatusChecker: check method completed
2008-01-16 16:05 StatusChecker: check method started
2008-01-16 16:05 StatusChecker: processing host head0
2008-01-16 16:05 StatusChecker: processing domain webconsole
2008-01-16 16:05 StatusChecker: processing host host1
2008-01-16 16:05 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 16:05 StatusChecker: check method completed
2008-01-16 16:06 StatusChecker: check method started
2008-01-16 16:06 StatusChecker: processing host head0
2008-01-16 16:06 StatusChecker: processing domain webconsole
2008-01-16 16:06 StatusChecker: processing host host1
2008-01-16 16:06 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 16:06 StatusChecker: check method completed
2008-01-16 16:07 StatusChecker: check method started
2008-01-16 16:07 StatusChecker: processing host head0
2008-01-16 16:07 StatusChecker: processing domain webconsole
2008-01-16 16:07 StatusChecker: processing host host1
2008-01-16 16:07 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 16:07 StatusChecker: check method completed
2008-01-16 16:08 StatusChecker: check method started
2008-01-16 16:08 StatusChecker: processing host head0
2008-01-16 16:08 StatusChecker: processing domain webconsole
2008-01-16 16:08 StatusChecker: processing host host1
2008-01-16 16:08 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 16:08 StatusChecker: check method completed
2008-01-16 16:09 StatusChecker: check method started
2008-01-16 16:09 StatusChecker: processing host head0
2008-01-16 16:09 StatusChecker: processing domain webconsole
2008-01-16 16:09 StatusChecker: processing host host1
2008-01-16 16:09 StatusChecker: error encountered for host host1:  
RuntimeError: Unable to ssh to 192.168.1.2 as root
2008-01-16 16:09 StatusChecker: check method completed
2008-01-16 16:10 StatusChecker: check method started
2008-01-16 16:10 StatusChecker: processing host head0
2008-01-16 16:10 StatusChecker: processing domain webconsole
2008-01-16 16:10 StatusChecker: processing host host1

After 16:10 the worker seems to have disappeared. I found the  
following message in backgroundrb_debug.log:

/home/webcon/apps/webconsole/releases/20080115103838/vendor/plugins/ 
backgroundrb/server/meta_worker.rb:329:in `check_for
_timer_events': undefined method `info' for nil:NilClass (NoMethodError)
         from /home/webcon/apps/webconsole/releases/20080115103838/ 
vendor/plugins/backgroundrb/framework/core.rb:148:in `
start_reactor'
         from /home/webcon/apps/webconsole/releases/20080115103838/ 
vendor/plugins/backgroundrb/framework/core.rb:147:in `
loop'
         from /home/webcon/apps/webconsole/releases/20080115103838/ 
vendor/plugins/backgroundrb/framework/core.rb:147:in `
start_reactor'
         from /home/webcon/apps/webconsole/releases/20080115103838/ 
vendor/plugins/backgroundrb/framework/worker.rb:21:in
`start_worker'
         from /home/webcon/apps/webconsole/releases/20080115103838/ 
vendor/plugins/backgroundrb/framework/packet_master.rb
:134:in `fork_and_load'
         from /home/webcon/apps/webconsole/releases/20080115103838/ 
vendor/plugins/backgroundrb/framework/packet_master.rb
:98:in `load_workers'
         from /home/webcon/apps/webconsole/releases/20080115103838/ 
vendor/plugins/backgroundrb/framework/packet_master.rb
:93:in `each'
         from /home/webcon/apps/webconsole/releases/20080115103838/ 
vendor/plugins/backgroundrb/framework/packet_master.rb
:93:in `load_workers'
         from /home/webcon/apps/webconsole/releases/20080115103838/ 
vendor/plugins/backgroundrb/framework/packet_master.rb
:19:in `run'
         from /home/webcon/apps/webconsole/releases/20080115103838/ 
vendor/plugins/backgroundrb/server/master_worker.rb:16
3:in `initialize'
         from script/backgroundrb:41:in `new'
         from script/backgroundrb:41


The code for this worker can be found here:

   https://svn.sxven.com/webconsole/trunk/lib/workers/status_checker.rb


Any pointers on how to debug this is much appreciated.

cheers,
mengkuan




From gethemant at gmail.com  Wed Jan 16 10:16:10 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 16 Jan 2008 20:46:10 +0530
Subject: [Backgroundrb-devel] mysterious crash of a particular worker
In-Reply-To: <4345930A-FC93-4638-B2A4-ACFF2B35761E@gmail.com>
References: <4345930A-FC93-4638-B2A4-ACFF2B35761E@gmail.com>
Message-ID: <fb283d5e0801160716y35c3e127n22e0f69836d1397d@mail.gmail.com>

Hi Meng,

On Jan 16, 2008 1:57 PM, Meng Kuan <mengkuan at gmail.com> wrote:
> Hi,
>
> I am using the latest checkout from backgroundrb release 1.0.1. I
> have a worker called status_checker that periodically (every minute)
> checks the status of certain hosts over the network. It works fine at
> the beginning but after a while the worker will mysteriously
> disappear and stop working. I have other workers running but they do
> not disappear like this worker does.
>
> These are the logs generated from logger.info calls for this
> particular worker.
>
> 2008-01-16 15:55 StatusChecker: check method started
> 2008-01-16 15:55 StatusChecker: processing host head0
> 2008-01-16 15:55 StatusChecker: processing domain webconsole
> 2008-01-16 15:55 StatusChecker: processing host host1
> 2008-01-16 15:55 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 15:55 StatusChecker: check method completed
> 2008-01-16 15:56 StatusChecker: check method started
> 2008-01-16 15:56 StatusChecker: processing host head0
> 2008-01-16 15:56 StatusChecker: processing domain webconsole
> 2008-01-16 15:56 StatusChecker: processing host host1
> 2008-01-16 15:56 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 15:56 StatusChecker: check method completed
> 2008-01-16 15:57 StatusChecker: check method started
> 2008-01-16 15:57 StatusChecker: processing host head0
> 2008-01-16 15:57 StatusChecker: processing domain webconsole
> 2008-01-16 15:57 StatusChecker: processing host host1
> 2008-01-16 15:57 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 15:57 StatusChecker: check method completed
> 2008-01-16 15:58 StatusChecker: check method started
> 2008-01-16 15:58 StatusChecker: processing host head0
> 2008-01-16 15:58 StatusChecker: processing domain webconsole
> 2008-01-16 15:58 StatusChecker: processing host host1
> 2008-01-16 15:58 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 15:58 StatusChecker: check method completed
> 2008-01-16 15:59 StatusChecker: check method started
> 2008-01-16 15:59 StatusChecker: processing host head0
> 2008-01-16 15:59 StatusChecker: processing domain webconsole
> 2008-01-16 15:59 StatusChecker: processing host host1
> 2008-01-16 15:59 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 15:59 StatusChecker: check method completed
> 2008-01-16 16:00 StatusChecker: check method started
> 2008-01-16 16:00 StatusChecker: processing host head0
> 2008-01-16 16:00 StatusChecker: processing domain webconsole
> 2008-01-16 16:00 StatusChecker: processing host host1
> 2008-01-16 16:00 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:00 StatusChecker: check method completed
> 2008-01-16 16:01 StatusChecker: check method started
> 2008-01-16 16:01 StatusChecker: processing host head0
> 2008-01-16 16:01 StatusChecker: processing domain webconsole
> 2008-01-16 16:01 StatusChecker: processing host host1
> 2008-01-16 16:01 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:01 StatusChecker: check method completed
> 2008-01-16 16:02 StatusChecker: check method started
> 2008-01-16 16:02 StatusChecker: processing host head0
> 2008-01-16 16:02 StatusChecker: processing domain webconsole
> 2008-01-16 16:02 StatusChecker: processing host host1
> 2008-01-16 16:02 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:02 StatusChecker: check method completed
> 2008-01-16 16:03 StatusChecker: check method started
> 2008-01-16 16:03 StatusChecker: processing host head0
> 2008-01-16 16:03 StatusChecker: processing domain webconsole
> 2008-01-16 16:03 StatusChecker: processing host host1
> 2008-01-16 16:03 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:03 StatusChecker: check method completed
> 2008-01-16 16:04 StatusChecker: check method started
> 2008-01-16 16:04 StatusChecker: processing host head0
> 2008-01-16 16:04 StatusChecker: processing domain webconsole
> 2008-01-16 16:04 StatusChecker: processing host host1
> 2008-01-16 16:04 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:04 StatusChecker: check method completed
> 2008-01-16 16:05 StatusChecker: check method started
> 2008-01-16 16:05 StatusChecker: processing host head0
> 2008-01-16 16:05 StatusChecker: processing domain webconsole
> 2008-01-16 16:05 StatusChecker: processing host host1
> 2008-01-16 16:05 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:05 StatusChecker: check method completed
> 2008-01-16 16:06 StatusChecker: check method started
> 2008-01-16 16:06 StatusChecker: processing host head0
> 2008-01-16 16:06 StatusChecker: processing domain webconsole
> 2008-01-16 16:06 StatusChecker: processing host host1
> 2008-01-16 16:06 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:06 StatusChecker: check method completed
> 2008-01-16 16:07 StatusChecker: check method started
> 2008-01-16 16:07 StatusChecker: processing host head0
> 2008-01-16 16:07 StatusChecker: processing domain webconsole
> 2008-01-16 16:07 StatusChecker: processing host host1
> 2008-01-16 16:07 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:07 StatusChecker: check method completed
> 2008-01-16 16:08 StatusChecker: check method started
> 2008-01-16 16:08 StatusChecker: processing host head0
> 2008-01-16 16:08 StatusChecker: processing domain webconsole
> 2008-01-16 16:08 StatusChecker: processing host host1
> 2008-01-16 16:08 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:08 StatusChecker: check method completed
> 2008-01-16 16:09 StatusChecker: check method started
> 2008-01-16 16:09 StatusChecker: processing host head0
> 2008-01-16 16:09 StatusChecker: processing domain webconsole
> 2008-01-16 16:09 StatusChecker: processing host host1
> 2008-01-16 16:09 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:09 StatusChecker: check method completed
> 2008-01-16 16:10 StatusChecker: check method started
> 2008-01-16 16:10 StatusChecker: processing host head0
> 2008-01-16 16:10 StatusChecker: processing domain webconsole
> 2008-01-16 16:10 StatusChecker: processing host host1
>
> After 16:10 the worker seems to have disappeared. I found the
> following message in backgroundrb_debug.log:
>
> /home/webcon/apps/webconsole/releases/20080115103838/vendor/plugins/
> backgroundrb/server/meta_worker.rb:329:in `check_for
> _timer_events': undefined method `info' for nil:NilClass (NoMethodError)
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/core.rb:148:in `
> start_reactor'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/core.rb:147:in `
> loop'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/core.rb:147:in `
> start_reactor'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/worker.rb:21:in
> `start_worker'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/packet_master.rb
> :134:in `fork_and_load'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/packet_master.rb
> :98:in `load_workers'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/packet_master.rb
> :93:in `each'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/packet_master.rb
> :93:in `load_workers'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/packet_master.rb
> :19:in `run'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/server/master_worker.rb:16
> 3:in `initialize'
>          from script/backgroundrb:41:in `new'
>          from script/backgroundrb:41
>
>
> The code for this worker can be found here:
>
>    https://svn.sxven.com/webconsole/trunk/lib/workers/status_checker.rb
>
>
> Any pointers on how to debug this is much appreciated.
>

Can you sync your plugin with trunk and see if its fixed? Looks like
we had a bug in logger functionality.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From tim at pivotib.com  Wed Jan 16 12:48:08 2008
From: tim at pivotib.com (Tim Glen)
Date: Wed, 16 Jan 2008 12:48:08 -0500
Subject: [Backgroundrb-devel] getting started
In-Reply-To: <fb283d5e0801111406r3d3aebc6o1f4366dda7efc45d@mail.gmail.com>
References: <3C0E2D79-1483-423D-B33A-949A923DA858@pivotib.com>
	<fb283d5e0801111406r3d3aebc6o1f4366dda7efc45d@mail.gmail.com>
Message-ID: <9110369F-688A-4BBB-80A7-25612191B56A@pivotib.com>

Hey there,

On 11-Jan-08, at 5:06 PM, hemant wrote:

> Hi Tim,
>
> On Jan 12, 2008 2:49 AM, Tim Glen <tim at pivotib.com> wrote:
>> <removed snippet>
>>
>> So far, I've downloaded backgroundrb, done the basic rails setup,
>> created my worker and started it using `script/backgroundrb start`.
>> Here's the behaviour I'm looking to create:
>> 1. When a file is uploaded, the worker needs to be triggered to start
>> processing the file (this is a 20-60 minute job, depending on the  
>> file)
>> 2. The user should be redirected to a page where they can see a  
>> status
>> of the processing job (the ideal would be a percentage or status bar
>> but I realize that's an implementation concern). most importantly,
>> they need to know when it's been completed.
>>
>> <removed snippet>
>
> Start your long running task like this:
>
>
> MiddleMan.new_worker(:worker => :sales_processor_worker,:data =>  
> @upload.id)
> # I think, you need _worker there, just check the name thats there in
> worker class. Whatever name is used
> # there, you must use it here too.
>
> Now in sales_processor_worker
>
> class SalesProcessorWorker < BackgrunDRb::MetaWorker
>  set_worker_name :sales_processor_worker
>  set_no_auto_load true
>  # whatever you passed as an :data is available here as an argument
>  def create(args = nil)
>    register_status("Processing started")
>    # do some more processing here
>  end
> end
>
> From controller get status like this:
>
> MiddleMan.ask_status(:worker => :sales_processor_worker)
>
> Thats about it. The key is, if you don't need a perpetually running
> worker around, disable auto loading of worker and start your worker
> using MiddleMan.new_worker and then query status. Hope this helps.

okay, cool.

I've settled in the 1.0.1 release and am using that now rather than  
trunk.

A few followup questions:
1. should my new_worker call include a :job_key? There does exist the  
possibility for more than one worker to be created at a time. If so,  
does the ask_status differentiate between the workers?
2. What's the best way to "exit" from a worker - can I just call exit  
at the end or should I call delete_worker when it's complete. It  
_seems_ like if I just exit, the 11006 port stays open after each  
worker is completed and no longer running (according to `ps aux`),  
which doesn't seem right to me. These ports stay open even after I  
`script/backgroundrb stop`. Since there's no process attached to them,  
I'm unsure how to close the ports manually:
   timg:marks tim$ netstat -at | grep 11006
   tcp6       0      0  localhost.51482        localhost.11006         
TIME_WAIT
   tcp6       0      0  localhost.51483        localhost.11006         
TIME_WAIT
   tcp6       0      0  localhost.51484        localhost.11006         
TIME_WAIT
   tcp6       0      0  localhost.51485        localhost.11006         
TIME_WAIT
   tcp6       0      0  localhost.51486        localhost.11006         
TIME_WAIT
   tcp6       0      0  localhost.51487        localhost.11006         
TIME_WAIT
3. I'm using rspec as a specing framework. From a relatively recent  
thread (see here: http://rubyforge.org/pipermail/backgroundrb-devel/2008-January/001280.html) 
, my understanding was to leave the ":environment: development" key in  
my backgroundrb.yml file and just put RAILS_ENV="test" in my  
bdrb_spec_helper.rb file (virtually identical to the bdrb_test_helper  
file). I've done this, but when I run `script/backgroundrb start`, it  
still inherits the development environment and therefore can't do any  
db interaction. I've been reduced to changing my config file whenever  
I want to change from development to test or vice versa. Am I meant to  
start up backgroundrb from within the test environment somehow?

Hopefully this all makes sense.

take care,
tim

From gethemant at gmail.com  Wed Jan 16 13:15:30 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 16 Jan 2008 23:45:30 +0530
Subject: [Backgroundrb-devel] getting started
In-Reply-To: <9110369F-688A-4BBB-80A7-25612191B56A@pivotib.com>
References: <3C0E2D79-1483-423D-B33A-949A923DA858@pivotib.com>
	<fb283d5e0801111406r3d3aebc6o1f4366dda7efc45d@mail.gmail.com>
	<9110369F-688A-4BBB-80A7-25612191B56A@pivotib.com>
Message-ID: <fb283d5e0801161015o5fa1f7bax4032e7d813634173@mail.gmail.com>

Hi Tim

On Jan 16, 2008 11:18 PM, Tim Glen <tim at pivotib.com> wrote:
<snip>
> A few followup questions:
> 1. should my new_worker call include a :job_key? There does exist the
> possibility for more than one worker to be created at a time. If so,
> does the ask_status differentiate between the workers?

It sure differentiate between workers.You can include job_key if you
want same worker to be running in two different processes.


> 2. What's the best way to "exit" from a worker - can I just call exit
> at the end or should I call delete_worker when it's complete. It
> _seems_ like if I just exit, the 11006 port stays open after each
> worker is completed and no longer running (according to `ps aux`),
> which doesn't seem right to me. These ports stay open even after I
> `script/backgroundrb stop`. Since there's no process attached to them,
> I'm unsure how to close the ports manually:
>    timg:marks tim$ netstat -at | grep 11006
>    tcp6       0      0  localhost.51482        localhost.11006
> TIME_WAIT
>    tcp6       0      0  localhost.51483        localhost.11006
> TIME_WAIT
>    tcp6       0      0  localhost.51484        localhost.11006
> TIME_WAIT
>    tcp6       0      0  localhost.51485        localhost.11006
> TIME_WAIT
>    tcp6       0      0  localhost.51486        localhost.11006
> TIME_WAIT
>    tcp6       0      0  localhost.51487        localhost.11006
> TIME_WAIT

You can call "exit" from a worker. However, when you call exit on a
worker, only that worker dies, master process stays, for handling more
requests.
However, your last bit of information about port still being used even
after bdrb is completely stopped is weird and could be OS specific. I
will investigate further.

> 3. I'm using rspec as a specing framework. From a relatively recent
> thread (see here: http://rubyforge.org/pipermail/backgroundrb-devel/2008-January/001280.html)
> , my understanding was to leave the ":environment: development" key in
> my backgroundrb.yml file and just put RAILS_ENV="test" in my
> bdrb_spec_helper.rb file (virtually identical to the bdrb_test_helper
> file). I've done this, but when I run `script/backgroundrb start`, it
> still inherits the development environment and therefore can't do any
> db interaction. I've been reduced to changing my config file whenever
> I want to change from development to test or vice versa. Am I meant to
> start up backgroundrb from within the test environment somehow?

You mean to say that, even in test mode, development environment is loaded?
There is something weird here, that helper shouldn't load any of
backgroundrb files and is just for enabling testing of workers.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From tim at pivotib.com  Wed Jan 16 16:33:54 2008
From: tim at pivotib.com (Tim Glen)
Date: Wed, 16 Jan 2008 16:33:54 -0500
Subject: [Backgroundrb-devel] getting started
In-Reply-To: <fb283d5e0801161015o5fa1f7bax4032e7d813634173@mail.gmail.com>
References: <3C0E2D79-1483-423D-B33A-949A923DA858@pivotib.com>
	<fb283d5e0801111406r3d3aebc6o1f4366dda7efc45d@mail.gmail.com>
	<9110369F-688A-4BBB-80A7-25612191B56A@pivotib.com>
	<fb283d5e0801161015o5fa1f7bax4032e7d813634173@mail.gmail.com>
Message-ID: <560CE86A-34E9-4A56-BCEA-C5D7EDF1462B@pivotib.com>

Hi Hemant,

On 16-Jan-08, at 1:15 PM, hemant wrote:

> Hi Tim
>
> On Jan 16, 2008 11:18 PM, Tim Glen <tim at pivotib.com> wrote:
> <snip>
>> A few followup questions:
>> 1. should my new_worker call include a :job_key? There does exist the
>> possibility for more than one worker to be created at a time. If so,
>> does the ask_status differentiate between the workers?
>
> It sure differentiate between workers.You can include job_key if you
> want same worker to be running in two different processes.

k cool. I had assumed so, but wanted to make sure.

>> 2. What's the best way to "exit" from a worker - can I just call exit
>> at the end or should I call delete_worker when it's complete. It
>> _seems_ like if I just exit, the 11006 port stays open after each
>> worker is completed and no longer running (according to `ps aux`),
>> which doesn't seem right to me. These ports stay open even after I
>> `script/backgroundrb stop`. Since there's no process attached to  
>> them,
>> I'm unsure how to close the ports manually:
>>   timg:marks tim$ netstat -at | grep 11006
>>   tcp6       0      0  localhost.51482        localhost.11006
>> TIME_WAIT
>>   tcp6       0      0  localhost.51483        localhost.11006
>> TIME_WAIT
>>   tcp6       0      0  localhost.51484        localhost.11006
>> TIME_WAIT
>>   tcp6       0      0  localhost.51485        localhost.11006
>> TIME_WAIT
>>   tcp6       0      0  localhost.51486        localhost.11006
>> TIME_WAIT
>>   tcp6       0      0  localhost.51487        localhost.11006
>> TIME_WAIT
>
> You can call "exit" from a worker. However, when you call exit on a
> worker, only that worker dies, master process stays, for handling more
> requests.

This I expected, yes. I just looked more at this - these ports stay  
open as long as the master_worker is still running. Those ports  
actually remain open _after_ I have stopped the master worker (using  
either `script/backgroundrb stop` or `kill`, but seem to close  
themselves after a few minutes (I haven't been able to get more  
accurate than that). If the worker-specific ports are _meant_ to hang  
around while the master_worker is running, that's all good. Also, if  
they just take a few minutes to get "reaped" after the process is  
completed, that's okay too.

> However, your last bit of information about port still being used even
> after bdrb is completely stopped is weird and could be OS specific. I
> will investigate further.

Specifically, i'm developing on osx. The production machine is also os  
x or I would be less concerned about it. Again - this could be  
expected behaviour, but it seemed odd to me

>> 3. I'm using rspec as a specing framework. From a relatively recent
>> thread (see here: http://rubyforge.org/pipermail/backgroundrb-devel/2008-January/001280.html)
>> , my understanding was to leave the ":environment: development" key  
>> in
>> my backgroundrb.yml file and just put RAILS_ENV="test" in my
>> bdrb_spec_helper.rb file (virtually identical to the bdrb_test_helper
>> file). I've done this, but when I run `script/backgroundrb start`, it
>> still inherits the development environment and therefore can't do any
>> db interaction. I've been reduced to changing my config file whenever
>> I want to change from development to test or vice versa. Am I meant  
>> to
>> start up backgroundrb from within the test environment somehow?
>
> You mean to say that, even in test mode, development environment is  
> loaded?
> There is something weird here, that helper shouldn't load any of
> backgroundrb files and is just for enabling testing of workers.

yes, I'm saying that having RAILS_ENV = 'test' at the top of my test  
helper file seems to make no difference to the environment. Let me  
clarify what I'm doing (and my assumptions) a little further. First,  
my assumption is that I need to have the master_worker running  
(through `script/backgroundrb start`) in order to run my tests. When  
it's not, I get a backgroundrb connection error so that seems valid.  
Also, I am testing using the MiddleMan object like so:
MiddleMan.new_worker(:worker => :sales_processor_worker, :job_key => "sales_processor_#{@upload.id 
}", :data => @upload.id)
  MiddleMan.ask_status(:worker => :sales_processor_worker, :job_key =>  
"sales_processor_#{@upload.id}").should_not be_nil

there are slightly more complex things that I'm doing in the testing,  
but that's the gist of it. Is there a better way? I can't run the  
Worker directly from within the test since the Worker contains an  
`exit` call. If you want to see actual spec, code, I can send it. I  
_could_ stub out the MiddleMan functionality but wanted to see actual  
functionality in my specs since it was also part of the discovery  
process for me.

take care,
tim


From felixclack at gmail.com  Wed Jan 16 18:33:30 2008
From: felixclack at gmail.com (felix clack)
Date: Thu, 17 Jan 2008 10:33:30 +1100
Subject: [Backgroundrb-devel] Backgroundrb 1.0.1: uninitialized constant
	Packet::ClassHelpers (NameError)
Message-ID: <85dc64750801161533q7f38fa45x4518ce4f214bef92@mail.gmail.com>

Hi,

just updated backgroundrb to release 1.0.1 and am getting this error when
trying to start backgroundrb.
vendor/plugins/backgroundrb/framework/core.rb:18: uninitialized constant
Packet::ClassHelpers (NameError)

Just updated from r298 where everything was working fine.

If I comment out the offending line then backgroundrb starts but I can't
connect to it from my app.

Anyone have any ideas as to what may be the problem?

-- 
thanks,
Felix
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080117/5bc8e51c/attachment-0001.html 

From kenglish at gmail.com  Wed Jan 16 19:21:14 2008
From: kenglish at gmail.com (Kenglish)
Date: Wed, 16 Jan 2008 14:21:14 -1000
Subject: [Backgroundrb-devel] Windows
Message-ID: <478E9F7A.7020307@gmail.com>

Any plans to get version 1.0 working on Windows? I am getting :


script/backgroundrb:24:in `fork': fork() function is unimplemented on 
this machine (NotImplementedError)    from script/backgroundrb:24

-- 
Aloha,

Kevin English
Honolulu, HI
808.271.5125
kenglish at gmail.com
www.kenglish77.com


From gethemant at gmail.com  Wed Jan 16 21:26:12 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 17 Jan 2008 07:56:12 +0530
Subject: [Backgroundrb-devel] Backgroundrb 1.0.1: uninitialized constant
	Packet::ClassHelpers (NameError)
In-Reply-To: <85dc64750801161533q7f38fa45x4518ce4f214bef92@mail.gmail.com>
References: <85dc64750801161533q7f38fa45x4518ce4f214bef92@mail.gmail.com>
Message-ID: <fb283d5e0801161826t37039738v75cc8ac696cfa1f1@mail.gmail.com>

Hi,

On Jan 17, 2008 5:03 AM, felix clack <felixclack at gmail.com> wrote:
> Hi,
>
> just updated backgroundrb to release 1.0.1 and am getting this error when
> trying to start backgroundrb.
> vendor/plugins/backgroundrb/framework/core.rb:18: uninitialized constant
> Packet::ClassHelpers (NameError)
>
> Just updated from r298 where everything was working fine.
>
> If I comment out the offending line then backgroundrb starts but I can't
> connect to it from my app.
>
> Anyone have any ideas as to what may be the problem?

Did you remove old backgroundrb script from RAILS_ROOT/script/
directory? If not remove that file and run:

rake backgroundrb:setup



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Wed Jan 16 21:27:47 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 17 Jan 2008 07:57:47 +0530
Subject: [Backgroundrb-devel] Windows
In-Reply-To: <478E9F7A.7020307@gmail.com>
References: <478E9F7A.7020307@gmail.com>
Message-ID: <fb283d5e0801161827q4c6b8539rff7a5cce209a8be6@mail.gmail.com>

Hi

On Jan 17, 2008 5:51 AM, Kenglish <kenglish at gmail.com> wrote:
> Any plans to get version 1.0 working on Windows? I am getting :
>
>
> script/backgroundrb:24:in `fork': fork() function is unimplemented on
> this machine (NotImplementedError)    from script/backgroundrb:24

I have my hands full, if anyone is willing to help in getting Windows
support up, then we can work together, otherwise no.


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Wed Jan 16 21:34:55 2008
From: gethemant at gmail.com (hemant kumar)
Date: Thu, 17 Jan 2008 08:04:55 +0530
Subject: [Backgroundrb-devel] getting started
In-Reply-To: <560CE86A-34E9-4A56-BCEA-C5D7EDF1462B@pivotib.com>
References: <3C0E2D79-1483-423D-B33A-949A923DA858@pivotib.com>
	<fb283d5e0801111406r3d3aebc6o1f4366dda7efc45d@mail.gmail.com>
	<9110369F-688A-4BBB-80A7-25612191B56A@pivotib.com>
	<fb283d5e0801161015o5fa1f7bax4032e7d813634173@mail.gmail.com>
	<560CE86A-34E9-4A56-BCEA-C5D7EDF1462B@pivotib.com>
Message-ID: <1200537295.6593.3.camel@shire>

Hi Tim,

On Wed, 2008-01-16 at 16:33 -0500, Tim Glen wrote:
<snip>

> 
> yes, I'm saying that having RAILS_ENV = 'test' at the top of my test  
> helper file seems to make no difference to the environment. Let me  
> clarify what I'm doing (and my assumptions) a little further. First,  
> my assumption is that I need to have the master_worker running  
> (through `script/backgroundrb start`) in order to run my tests. When  
> it's not, I get a backgroundrb connection error so that seems valid.  
> Also, I am testing using the MiddleMan object like so:
> MiddleMan.new_worker(:worker => :sales_processor_worker, :job_key => "sales_processor_#{@upload.id 
> }", :data => @upload.id)
>   MiddleMan.ask_status(:worker => :sales_processor_worker, :job_key =>  
> "sales_processor_#{@upload.id}").should_not be_nil
> 
> there are slightly more complex things that I'm doing in the testing,  
> but that's the gist of it. Is there a better way? I can't run the  
> Worker directly from within the test since the Worker contains an  
> `exit` call. If you want to see actual spec, code, I can send it. I  
> _could_ stub out the MiddleMan functionality but wanted to see actual  
> functionality in my specs since it was also part of the discovery  
> process for me.

You don't need to have the master process running, when you are doing
testing. The idea is that helper should abstract away all the network IO
stuff. Can I see your spec helper code?




From felixclack at gmail.com  Wed Jan 16 21:47:20 2008
From: felixclack at gmail.com (felix clack)
Date: Thu, 17 Jan 2008 13:47:20 +1100
Subject: [Backgroundrb-devel] Backgroundrb 1.0.1: uninitialized constant
	Packet::ClassHelpers (NameError)
In-Reply-To: <fb283d5e0801161826t37039738v75cc8ac696cfa1f1@mail.gmail.com>
References: <85dc64750801161533q7f38fa45x4518ce4f214bef92@mail.gmail.com>
	<fb283d5e0801161826t37039738v75cc8ac696cfa1f1@mail.gmail.com>
Message-ID: <85dc64750801161847o787b255epb4e456b1025add30@mail.gmail.com>

Thanks.

I tried that, however I still get the same error. I'm able to get things
working if I comment out the line
include Packet::ClassHelper.

The problem I experience now is I'm getting Packet::Disconnect errors during
my worker scraping other websites. I'm guessing this problem is not a bug in
backgroundrb but probably with the sites I'm connecting to.

On 17/01/2008, hemant <gethemant at gmail.com> wrote:
>
> Hi,
>
> On Jan 17, 2008 5:03 AM, felix clack <felixclack at gmail.com> wrote:
> > Hi,
> >
> > just updated backgroundrb to release 1.0.1 and am getting this error
> when
> > trying to start backgroundrb.
> > vendor/plugins/backgroundrb/framework/core.rb:18: uninitialized constant
> > Packet::ClassHelpers (NameError)
> >
> > Just updated from r298 where everything was working fine.
> >
> > If I comment out the offending line then backgroundrb starts but I can't
> > connect to it from my app.
> >
> > Anyone have any ideas as to what may be the problem?
>
> Did you remove old backgroundrb script from RAILS_ROOT/script/
> directory? If not remove that file and run:
>
> rake backgroundrb:setup
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>



-- 
thanks,
Felix
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080117/79fe6188/attachment.html 

From gethemant at gmail.com  Wed Jan 16 22:08:46 2008
From: gethemant at gmail.com (hemant kumar)
Date: Thu, 17 Jan 2008 08:38:46 +0530
Subject: [Backgroundrb-devel] Backgroundrb 1.0.1: uninitialized	constant
	Packet::ClassHelpers (NameError)
In-Reply-To: <85dc64750801161847o787b255epb4e456b1025add30@mail.gmail.com>
References: <85dc64750801161533q7f38fa45x4518ce4f214bef92@mail.gmail.com>
	<fb283d5e0801161826t37039738v75cc8ac696cfa1f1@mail.gmail.com>
	<85dc64750801161847o787b255epb4e456b1025add30@mail.gmail.com>
Message-ID: <1200539326.6593.7.camel@shire>

Hi

On Thu, 2008-01-17 at 13:47 +1100, felix clack wrote:
> Thanks.
> 
> I tried that, however I still get the same error. I'm able to get
> things working if I comment out the line 
> include Packet::ClassHelper.
> 
> The problem I experience now is I'm getting Packet::Disconnect errors
> during my worker scraping other websites. I'm guessing this problem is
> not a bug in backgroundrb but probably with the sites I'm connecting
> to. 

I doubt that backgroundrb will work properly if you comment that line.
check if framework/packet.rb has require 'class_helpers' in it.

> 
> On 17/01/2008, hemant <gethemant at gmail.com> wrote:
>         Hi,
>         
>         On Jan 17, 2008 5:03 AM, felix clack <felixclack at gmail.com> wrote:
>         > Hi,
>         >
>         > just updated backgroundrb to release 1.0.1 and am getting this error when 
>         > trying to start backgroundrb.
>         > vendor/plugins/backgroundrb/framework/core.rb:18: uninitialized constant
>         > Packet::ClassHelpers (NameError)
>         >
>         > Just updated from r298 where everything was working fine. 
>         >
>         > If I comment out the offending line then backgroundrb starts but I can't
>         > connect to it from my app.
>         >
>         > Anyone have any ideas as to what may be the problem?
>         
>         Did you remove old backgroundrb script from RAILS_ROOT/script/ 
>         directory? If not remove that file and run:
>         
>         rake backgroundrb:setup
>         
>         
>         
>         --
>         Let them talk of their oriental summer climes of everlasting
>         conservatories; give me the privilege of making my own summer with my 
>         own coals.
>         
>         http://gnufied.org
> 
> 
> 
> -- 
> thanks,
> Felix 
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel


From felixclack at gmail.com  Wed Jan 16 22:31:19 2008
From: felixclack at gmail.com (felix clack)
Date: Thu, 17 Jan 2008 14:31:19 +1100
Subject: [Backgroundrb-devel] Backgroundrb 1.0.1: uninitialized constant
	Packet::ClassHelpers (NameError)
In-Reply-To: <1200539326.6593.7.camel@shire>
References: <85dc64750801161533q7f38fa45x4518ce4f214bef92@mail.gmail.com>
	<fb283d5e0801161826t37039738v75cc8ac696cfa1f1@mail.gmail.com>
	<85dc64750801161847o787b255epb4e456b1025add30@mail.gmail.com>
	<1200539326.6593.7.camel@shire>
Message-ID: <85dc64750801161931y46a2f22bic9136d3031412d8a@mail.gmail.com>

Hi hemant,

thanks for the help but unfortunately that hasn't worked either. Still the
same error message when attempting to start backgroundrb.

The require line you suggested wasn't present in packet.rb so I added it,
however the same error message still appeared.

I've also checked out a fresh copy of backgroundrb to ensure I haven't
introduced any errors in the code. Doesn't seem to have made any difference
to the error.

On 17/01/2008, hemant kumar <gethemant at gmail.com> wrote:
>
> Hi
>
> On Thu, 2008-01-17 at 13:47 +1100, felix clack wrote:
> > Thanks.
> >
> > I tried that, however I still get the same error. I'm able to get
> > things working if I comment out the line
> > include Packet::ClassHelper.
> >
> > The problem I experience now is I'm getting Packet::Disconnect errors
> > during my worker scraping other websites. I'm guessing this problem is
> > not a bug in backgroundrb but probably with the sites I'm connecting
> > to.
>
> I doubt that backgroundrb will work properly if you comment that line.
> check if framework/packet.rb has require 'class_helpers' in it.
>
> >
> > On 17/01/2008, hemant <gethemant at gmail.com> wrote:
> >         Hi,
> >
> >         On Jan 17, 2008 5:03 AM, felix clack <felixclack at gmail.com>
> wrote:
> >         > Hi,
> >         >
> >         > just updated backgroundrb to release 1.0.1 and am getting this
> error when
> >         > trying to start backgroundrb.
> >         > vendor/plugins/backgroundrb/framework/core.rb:18:
> uninitialized constant
> >         > Packet::ClassHelpers (NameError)
> >         >
> >         > Just updated from r298 where everything was working fine.
> >         >
> >         > If I comment out the offending line then backgroundrb starts
> but I can't
> >         > connect to it from my app.
> >         >
> >         > Anyone have any ideas as to what may be the problem?
> >
> >         Did you remove old backgroundrb script from RAILS_ROOT/script/
> >         directory? If not remove that file and run:
> >
> >         rake backgroundrb:setup
> >
> >
> >
> >         --
> >         Let them talk of their oriental summer climes of everlasting
> >         conservatories; give me the privilege of making my own summer
> with my
> >         own coals.
> >
> >         http://gnufied.org
> >
> >
> >
> > --
> > thanks,
> > Felix
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
>


-- 
thanks,
Felix
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080117/011da427/attachment.html 

From iambenbryant at gmail.com  Thu Jan 17 03:46:44 2008
From: iambenbryant at gmail.com (Benjamin H. Bryant)
Date: Thu, 17 Jan 2008 03:46:44 -0500
Subject: [Backgroundrb-devel] Inconsistent Model creation behavior
Message-ID: <9E9080C2-55D2-4AB4-AAE8-09E63078F8E6@gmail.com>

Hi,

I have a relatively simple Worker: (I added the line numbers to the  
email, they are not in the worker)

#####################################################################################

class ImagetestWorker < BackgrounDRb::MetaWorker

   set_worker_name :imagetest_worker
   def create(args = nil)

   end

   def save_image(args)

     @newbook = Book.new
     @newbook.comment = "hello"
     @newbook.save

  14   @newerimage = Image.new
  15   @newerimage.user_id = args[:user_id]
  16   @newerimage.uploaded_data = args[:temppath]
         @newerimage.save

   end

end

#####################################################################################

It will create and save the Book object without problems.  However, it  
will not create the Image object and instead I receive this error:

#####################################################################################

workers/imagetest_worker.rb:15:in `save_image': You have a nil object  
when you didn't expect it! (NoMethodError)
You might have expected an instance of ActiveRecord::Base.
The error occurred while evaluating nil.[]	from /Users/BBryant/Sites/ 
date_site/trunk/vendor/plugins/backgroundrb/server/meta_worker.rb: 
223:in `send'
	from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/ 
server/meta_worker.rb:223:in `process_request'
	from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/ 
server/meta_worker.rb:207:in `receive_data'
	from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/ 
framework/worker.rb:54:in `receive_internal_data'
	from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/BackgrounDRB/ 
lib/../framework/bin_parser.rb:29:in `call'
	from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/BackgrounDRB/ 
lib/../framework/bin_parser.rb:29:in `extract'
	from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/ 
framework/worker.rb:52:in `receive_internal_data'
	from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/ 
framework/worker.rb:48:in `handle_internal_messages'
	 ... 10 levels...
	from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/ 
framework/packet_master.rb:19:in `run'
	from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/ 
server/master_worker.rb:163:in `initialize'
	from script/backgroundrb:41:in `new'
	from script/backgroundrb:41

####################################################################################

I can create and save an instance of the Image model from a controller  
without problems.
This is driving me crazy.

Thanks in advance.

Ben


From gethemant at gmail.com  Thu Jan 17 04:28:45 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 17 Jan 2008 14:58:45 +0530
Subject: [Backgroundrb-devel] Backgroundrb 1.0.1: uninitialized constant
	Packet::ClassHelpers (NameError)
In-Reply-To: <85dc64750801161931y46a2f22bic9136d3031412d8a@mail.gmail.com>
References: <85dc64750801161533q7f38fa45x4518ce4f214bef92@mail.gmail.com>
	<fb283d5e0801161826t37039738v75cc8ac696cfa1f1@mail.gmail.com>
	<85dc64750801161847o787b255epb4e456b1025add30@mail.gmail.com>
	<1200539326.6593.7.camel@shire>
	<85dc64750801161931y46a2f22bic9136d3031412d8a@mail.gmail.com>
Message-ID: <fb283d5e0801170128s5e57685m3b67a2a115f7cc55@mail.gmail.com>

Hi Felix,

On Jan 17, 2008 9:01 AM, felix clack <felixclack at gmail.com> wrote:
> Hi hemant,
>
> thanks for the help but unfortunately that hasn't worked either. Still the
> same error message when attempting to start backgroundrb.
>
> The require line you suggested wasn't present in packet.rb so I added it,
> however the same error message still appeared.
>
> I've also checked out a fresh copy of backgroundrb to ensure I haven't
> introduced any errors in the code. Doesn't seem to have made any difference
> to the error.

There is something fishy going in your setup? Checkout this file:

http://svn.devjavu.com/backgroundrb/tags/release-1.0.1/framework/packet.rb

It has the line "require 'class_helpers'", so I am surprised to see
that despite using 1.0.1 tag, you didn't had that line.
Also, there could be a possibility that you have a file with same name
elsewhere in your load_path. Can you check and tell me, if the
class_helpers file I mentioned is actually being loaded or not?



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Thu Jan 17 04:30:17 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 17 Jan 2008 15:00:17 +0530
Subject: [Backgroundrb-devel] Inconsistent Model creation behavior
In-Reply-To: <9E9080C2-55D2-4AB4-AAE8-09E63078F8E6@gmail.com>
References: <9E9080C2-55D2-4AB4-AAE8-09E63078F8E6@gmail.com>
Message-ID: <fb283d5e0801170130y60254847paf45e7834b20d903@mail.gmail.com>

Hi

On Jan 17, 2008 2:16 PM, Benjamin H. Bryant <iambenbryant at gmail.com> wrote:
> Hi,
>
> I have a relatively simple Worker: (I added the line numbers to the
> email, they are not in the worker)
>
> #####################################################################################
>
> class ImagetestWorker < BackgrounDRb::MetaWorker
>
>    set_worker_name :imagetest_worker
>    def create(args = nil)
>
>    end
>
>    def save_image(args)
>
>      @newbook = Book.new
>      @newbook.comment = "hello"
>      @newbook.save
>
>   14   @newerimage = Image.new
>   15   @newerimage.user_id = args[:user_id]
>   16   @newerimage.uploaded_data = args[:temppath]
>          @newerimage.save
>
>    end
>
> end
>
> #####################################################################################
>
> It will create and save the Book object without problems.  However, it
> will not create the Image object and instead I receive this error:
>
> #####################################################################################
>
> workers/imagetest_worker.rb:15:in `save_image': You have a nil object
> when you didn't expect it! (NoMethodError)
> You might have expected an instance of ActiveRecord::Base.
> The error occurred while evaluating nil.[]      from /Users/BBryant/Sites/
> date_site/trunk/vendor/plugins/backgroundrb/server/meta_worker.rb:
> 223:in `send'
>         from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/
> server/meta_worker.rb:223:in `process_request'
>         from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/
> server/meta_worker.rb:207:in `receive_data'
>         from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/
> framework/worker.rb:54:in `receive_internal_data'
>         from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/BackgrounDRB/
> lib/../framework/bin_parser.rb:29:in `call'
>         from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/BackgrounDRB/
> lib/../framework/bin_parser.rb:29:in `extract'
>         from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/
> framework/worker.rb:52:in `receive_internal_data'
>         from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/
> framework/worker.rb:48:in `handle_internal_messages'
>          ... 10 levels...
>         from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/
> framework/packet_master.rb:19:in `run'
>         from /Users/BBryant/Sites/date_site/trunk/vendor/plugins/backgroundrb/
> server/master_worker.rb:163:in `initialize'
>         from script/backgroundrb:41:in `new'
>         from script/backgroundrb:41
>
> ####################################################################################
>

Ok so you are invoking this task from rails right? Can you show us the
code thats being used to invoke above mentioned task?

-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Thu Jan 17 06:30:15 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 17 Jan 2008 17:00:15 +0530
Subject: [Backgroundrb-devel] Inconsistent Model creation behavior
In-Reply-To: <5A2A67A2-B75D-4238-A029-4FDB9FC75709@gmail.com>
References: <9E9080C2-55D2-4AB4-AAE8-09E63078F8E6@gmail.com>
	<fb283d5e0801170130y60254847paf45e7834b20d903@mail.gmail.com>
	<5A2A67A2-B75D-4238-A029-4FDB9FC75709@gmail.com>
Message-ID: <fb283d5e0801170330j6dc49978w141fee14a7af5970@mail.gmail.com>

On Jan 17, 2008 3:21 PM, Benjamin H. Bryant <iambenbryant at gmail.com> wrote:
> Sure, the code in the controller is:
>
>   MiddleMan.ask_work(:worker => :imagetest_worker, :worker_method
> => :save_image,  :user_id => session[:user_id], :temppath =>
> params[:newimage][:uploaded_data].path)
>

It should be:

 MiddleMan.ask_work(:worker => :imagetest_worker, :worker_method
=> :save_image, :data => { :user_id => session[:user_id], :temppath =>
params[:newimage][:uploaded_data].path} )

that :data thing is a must, your arguments aren't getting passed properly.

-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From jnutting at gmail.com  Thu Jan 17 12:00:05 2008
From: jnutting at gmail.com (Jack Nutting)
Date: Thu, 17 Jan 2008 18:00:05 +0100
Subject: [Backgroundrb-devel] periodic scheduling
Message-ID: <f84841020801170900v1dafbd99n2c8275aec3c195f@mail.gmail.com>

I've been using backgroundrb since back in March 2007 or so.  It's
been working mostly OK for me, but since the old version doesn't seem
to work out of the box with rails 2.0 for me, I decided to test out
the latest version.

So, I'm looking at the scheduling, and trying to figure out the best
option for what I want to do.  What I have is a set of workers that
each poll the database occasionally.  I'd like to set it up so that
each of them can poll for work, do the work (however long that may
take), then pause for a "sleep time" and start over.  With the old
version of backgroundrb, I was never able to get any of the scheduling
options to quite do what I wanted to do, so I just ran without
scheduling. I manually started all my workers by calling
MiddleMan.new_worker in a script, passing in the "sleep time", and
then implemented each worker something like this:

  def do_work(args)
    @args = args
    @sleep_time = @args[:sleep_time]
    logger.info "CampaignStarter #{jobkey} started"
    while(true)
      # do the actual work in another method
      main_work
      sleep @sleep_time
    end
  end

That seemed to work pretty well for the most part.

So now, looking at the current version.  I'm guessing that it's
probably not OK for me to use create() the way I was using do_work()
in the past;  Presumably create() is supposed to return at some point.
 Besides which, one of my workers needs to have multiple instances, so
I have to pass in a job_key somewhere.  I'm guessing that I should
call MiddleMan.new_worker(), telling it to run my main_work() method
with some scheduling option, but I'm not sure which scheduling option
to use.  I don't want to schedule it to run every 10 seconds, since
each call to main_work could take anything up to several minutes, and
I don't want the invocations piling up. The docs also seem to suggest
that I could do things my old way by calling MiddleMan.ask_work() and
letting it run forever, but it's not clear whether I can pass in a
job_key there.  I'd love any suggestions here...

Next, is there any way from inside a worker to see what its job_key
is?  With the old version I've been using the jobkey as a way to split
up some of the work requests, so each instance of my worker class
needs to know its own key in order to find what work is assigned to
it.

Finally, it's not at all clear to me how the current version handles
multiple instances and threads.  For example, if I call
MiddleMan.new_worker(), does it actually create a new instance of my
worker class (as the old version did), and if so, is that instance in
a new thread of its own?  How about MiddleMan.ask_work() and
MiddleMan.send_request()?

-- 
// jack
// http://www.nuthole.com

From gethemant at gmail.com  Thu Jan 17 12:41:37 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 17 Jan 2008 23:11:37 +0530
Subject: [Backgroundrb-devel] periodic scheduling
In-Reply-To: <f84841020801170900v1dafbd99n2c8275aec3c195f@mail.gmail.com>
References: <f84841020801170900v1dafbd99n2c8275aec3c195f@mail.gmail.com>
Message-ID: <fb283d5e0801170941l761b1855pc15bb208a524bfc9@mail.gmail.com>

Hi Jack,

On Jan 17, 2008 10:30 PM, Jack Nutting <jnutting at gmail.com> wrote:
> I've been using backgroundrb since back in March 2007 or so.  It's
> been working mostly OK for me, but since the old version doesn't seem
> to work out of the box with rails 2.0 for me, I decided to test out
> the latest version.
>
> So, I'm looking at the scheduling, and trying to figure out the best
> option for what I want to do.  What I have is a set of workers that
> each poll the database occasionally.  I'd like to set it up so that
> each of them can poll for work, do the work (however long that may
> take), then pause for a "sleep time" and start over.  With the old
> version of backgroundrb, I was never able to get any of the scheduling
> options to quite do what I wanted to do, so I just ran without
> scheduling. I manually started all my workers by calling
> MiddleMan.new_worker in a script, passing in the "sleep time", and
> then implemented each worker something like this:
>
>   def do_work(args)
>     @args = args
>     @sleep_time = @args[:sleep_time]
>     logger.info "CampaignStarter #{jobkey} started"
>     while(true)
>       # do the actual work in another method
>       main_work
>       sleep @sleep_time
>     end
>   end
>
> That seemed to work pretty well for the most part.

Yuck. Never use sleep() in new backgroundrb workers. :)

>
> So now, looking at the current version.  I'm guessing that it's
> probably not OK for me to use create() the way I was using do_work()
> in the past;  Presumably create() is supposed to return at some point.
>  Besides which, one of my workers needs to have multiple instances, so
> I have to pass in a job_key somewhere.  I'm guessing that I should
> call MiddleMan.new_worker(), telling it to run my main_work() method
> with some scheduling option, but I'm not sure which scheduling option
> to use.  I don't want to schedule it to run every 10 seconds, since
> each call to main_work could take anything up to several minutes, and
> I don't want the invocations piling up. The docs also seem to suggest
> that I could do things my old way by calling MiddleMan.ask_work() and
> letting it run forever, but it's not clear whether I can pass in a
> job_key there.  I'd love any suggestions here...

I am still unclear about whats your purpose of using bdrb? If you just
want to use it to schedule tasks, then you can specify your schedule
in config file, using either Cron on UNIX syntax and it will work. If
you are scheduling at in interval of 10 seconds and your actual tasks
takes longer than that, then also there is nothing to worry and bdrb
will schedule tasks as they are ready to run and worker is free. So
your tasks won't be piling up. create() does return at some point, but
at each iteration of reactor loop, tasks ready to run are executed.

Currently if you need more than one instance of same worker running at
the same time, its not supported via backgroundrb.yml file, but you
can create as many instances of workers as you want using different
job_keys using MiddleMan.new_worker() method from rails. Hell, I think
you can invoke MiddleMan.new_worker() from one of your workers to
start workers on demand and supply job_keys.

ask_work() is for doing one shot task execution. It can be only used
in a already runner worker. If you started a worker with a job_key
using new_worker() method, then you will have to use that job_key
along with worker name in ask_work() or ask_status() invocation.

>
> Next, is there any way from inside a worker to see what its job_key
> is?  With the old version I've been using the jobkey as a way to split
> up some of the work requests, so each instance of my worker class
> needs to know its own key in order to find what work is assigned to
> it.

Sad truth of life is that, you can't. Not from inside of a worker. But
since inside any worker you have full rails environment and hence
MiddleMan proxy object, you can query job_key using:

MiddleMan.worker_info(:worker => worker_name.to_sym)

Haven't tried, but it should work.

>
> Finally, it's not at all clear to me how the current version handles
> multiple instances and threads.  For example, if I call
> MiddleMan.new_worker(), does it actually create a new instance of my
> worker class (as the old version did), and if so, is that instance in
> a new thread of its own?  How about MiddleMan.ask_work() and
> MiddleMan.send_request()?

new_worker() creates a full forked process. ask_work(), send_request()
can only operate on workers which are already running.

And lastly, sorry for documentation, if you found it a bit out of
touch. *vows to improve it*


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From jnutting at gmail.com  Thu Jan 17 15:26:49 2008
From: jnutting at gmail.com (Jack Nutting)
Date: Thu, 17 Jan 2008 21:26:49 +0100
Subject: [Backgroundrb-devel] periodic scheduling
In-Reply-To: <fb283d5e0801170941l761b1855pc15bb208a524bfc9@mail.gmail.com>
References: <f84841020801170900v1dafbd99n2c8275aec3c195f@mail.gmail.com>
	<fb283d5e0801170941l761b1855pc15bb208a524bfc9@mail.gmail.com>
Message-ID: <f84841020801171226p3ac8c71r6a7053b880109409@mail.gmail.com>

On Jan 17, 2008 6:41 PM, hemant <gethemant at gmail.com> wrote:
> Yuck. Never use sleep() in new backgroundrb workers. :)

Yeah, I wasn't really 100% happy with that solution either...  I will
change my ways!

> I am still unclear about whats your purpose of using bdrb? If you just
> want to use it to schedule tasks, then you can specify your schedule
> in config file, using either Cron on UNIX syntax and it will work.

Good question!  At an earlier stage in my application's design, there
was a place for more standard backgroundrb usage, with some processing
that was triggered by user actions.  I liked the way it worked, and
was enough of a rails noob that I frankly didn't know any other way to
do background tasks that accessed activerecord, my model classes, etc,
so I started using backgroundrb for perpetually-running processes as
well;  eventually these also incorporated the functions that were
previously performed by user actions as well, and there I am today.
Maybe I should just stop using backgroundrb for now, until I have a
use that better matches its intended purpose.

> If
> you are scheduling at in interval of 10 seconds and your actual tasks
> takes longer than that, then also there is nothing to worry and bdrb
> will schedule tasks as they are ready to run and worker is free. So
> your tasks won't be piling up.

Ah, OK.  I picked up a different meaning from this text in the README,
the Cron Scheduling section:

  The method specified in the configuration
  file would be called periodically. You should accommodate
  for the fact that the time gap between periodic
  invocation of a method should be more than the time
  that is actually required to execute the method.
  If a method takes longer time than the time window
  specified, your method invocations will lag
  perpetually.

Doesn't that seem to be saying that requests will be piling up?

> ask_work() is for doing one shot task execution. It can be only used
> in a already runner worker. If you started a worker with a job_key
> using new_worker() method, then you will have to use that job_key
> along with worker name in ask_work() or ask_status() invocation.

OK, I didn't realize you could use job_key with ask_work().  Now it
makes sense to me.

> > Next, is there any way from inside a worker to see what its job_key
> > is?
>
> Sad truth of life is that, you can't. Not from inside of a worker. But
> since inside any worker you have full rails environment and hence
> MiddleMan proxy object, you can query job_key using:
>
> MiddleMan.worker_info(:worker => worker_name.to_sym)

It also occurred to me that I could pass the job_key in with the data
parameter when I create the worker.

> new_worker() creates a full forked process. ask_work(), send_request()
> can only operate on workers which are already running.

Thanks for the clarification.

> And lastly, sorry for documentation, if you found it a bit out of
> touch. *vows to improve it*

No problem, I appreciate that the documentation is there at all, and
most of all that you've picked up the backgroundrb ball and run with
it!  Keep up the good work!

-- 
// jack
// http://www.nuthole.com

From gethemant at gmail.com  Thu Jan 17 16:42:24 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 18 Jan 2008 03:12:24 +0530
Subject: [Backgroundrb-devel] periodic scheduling
In-Reply-To: <f84841020801171226p3ac8c71r6a7053b880109409@mail.gmail.com>
References: <f84841020801170900v1dafbd99n2c8275aec3c195f@mail.gmail.com>
	<fb283d5e0801170941l761b1855pc15bb208a524bfc9@mail.gmail.com>
	<f84841020801171226p3ac8c71r6a7053b880109409@mail.gmail.com>
Message-ID: <fb283d5e0801171342g73ff10aao51fec533c07b673c@mail.gmail.com>

Hi Jack,

On Jan 18, 2008 1:56 AM, Jack Nutting <jnutting at gmail.com> wrote:
> Good question!  At an earlier stage in my application's design, there
> was a place for more standard backgroundrb usage, with some processing
> that was triggered by user actions.  I liked the way it worked, and
> was enough of a rails noob that I frankly didn't know any other way to
> do background tasks that accessed activerecord, my model classes, etc,
> so I started using backgroundrb for perpetually-running processes as
> well;  eventually these also incorporated the functions that were
> previously performed by user actions as well, and there I am today.
> Maybe I should just stop using backgroundrb for now, until I have a
> use that better matches its intended purpose.

What I meant was, if you are using backgroundrb for periodic
scheduling of tasks, then you use cron or unix scheduling of tasks
provided by backgroundrb. I wouldn't ask you to go to crontab.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From felixclack at gmail.com  Thu Jan 17 21:21:13 2008
From: felixclack at gmail.com (felix clack)
Date: Fri, 18 Jan 2008 13:21:13 +1100
Subject: [Backgroundrb-devel] Backgroundrb 1.0.1: uninitialized constant
	Packet::ClassHelpers (NameError)
In-Reply-To: <fb283d5e0801170128s5e57685m3b67a2a115f7cc55@mail.gmail.com>
References: <85dc64750801161533q7f38fa45x4518ce4f214bef92@mail.gmail.com>
	<fb283d5e0801161826t37039738v75cc8ac696cfa1f1@mail.gmail.com>
	<85dc64750801161847o787b255epb4e456b1025add30@mail.gmail.com>
	<1200539326.6593.7.camel@shire>
	<85dc64750801161931y46a2f22bic9136d3031412d8a@mail.gmail.com>
	<fb283d5e0801170128s5e57685m3b67a2a115f7cc55@mail.gmail.com>
Message-ID: <85dc64750801171821i6e0e0c6ma1add3dfd9186f1c@mail.gmail.com>

Hi Hemant,

thanks for the help. I checked out a fresh copy of backgroundrb into a clean
rails project and moved it into my existing project and its all working fine
now. I must have made a mistake somewhere along the line with svn to prevent
it importing all the latest files.

Thanks again, this is a great piece of software.

Felix

On 17/01/2008, hemant <gethemant at gmail.com> wrote:
>
> Hi Felix,
>
> On Jan 17, 2008 9:01 AM, felix clack <felixclack at gmail.com> wrote:
> > Hi hemant,
> >
> > thanks for the help but unfortunately that hasn't worked either. Still
> the
> > same error message when attempting to start backgroundrb.
> >
> > The require line you suggested wasn't present in packet.rb so I added
> it,
> > however the same error message still appeared.
> >
> > I've also checked out a fresh copy of backgroundrb to ensure I haven't
> > introduced any errors in the code. Doesn't seem to have made any
> difference
> > to the error.
>
> There is something fishy going in your setup? Checkout this file:
>
> http://svn.devjavu.com/backgroundrb/tags/release-1.0.1/framework/packet.rb
>
> It has the line "require 'class_helpers'", so I am surprised to see
> that despite using 1.0.1 tag, you didn't had that line.
> Also, there could be a possibility that you have a file with same name
> elsewhere in your load_path. Can you check and tell me, if the
> class_helpers file I mentioned is actually being loaded or not?
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>



-- 
thanks,
Felix
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080118/549c819f/attachment-0001.html 

From mengkuan at gmail.com  Fri Jan 18 05:24:41 2008
From: mengkuan at gmail.com (Meng Kuan)
Date: Fri, 18 Jan 2008 18:24:41 +0800
Subject: [Backgroundrb-devel] mysterious crash of a particular worker
In-Reply-To: <fb283d5e0801160716y35c3e127n22e0f69836d1397d@mail.gmail.com>
References: <4345930A-FC93-4638-B2A4-ACFF2B35761E@gmail.com>
	<fb283d5e0801160716y35c3e127n22e0f69836d1397d@mail.gmail.com>
Message-ID: <D6F5250F-6BA8-49DA-8EC9-39430F6ABA68@gmail.com>

On 16 Jan 2008, at 11:16 PM, hemant wrote:
>
> Can you sync your plugin with trunk and see if its fixed? Looks like
> we had a bug in logger functionality.
>

Using trunk instead of release-1.0.1 seems to have fixed the problem.  
Ran the workers for 5 hours and no more mysterious disappearances so  
far. Thanks, hemant, and great work on the plugin!

From orion at mh2o.com  Fri Jan 18 18:12:13 2008
From: orion at mh2o.com (Orion Delwaterman)
Date: Fri, 18 Jan 2008 18:12:13 -0500
Subject: [Backgroundrb-devel] Query All Worker
Message-ID: <205576B004DDBD438F6560241E0751A0157F912E@sbsexc02.mh2onyc.com>

What does the method query_all_workers() do?  There is no documentation
on what this method returns (1.0.1 version).

 

Orion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080118/9f9b8112/attachment.html 

From gethemant at gmail.com  Sat Jan 19 06:57:57 2008
From: gethemant at gmail.com (hemant)
Date: Sat, 19 Jan 2008 17:27:57 +0530
Subject: [Backgroundrb-devel] Query All Worker
In-Reply-To: <205576B004DDBD438F6560241E0751A0157F912E@sbsexc02.mh2onyc.com>
References: <205576B004DDBD438F6560241E0751A0157F912E@sbsexc02.mh2onyc.com>
Message-ID: <fb283d5e0801190357j18b6f2edgee8ee9bff0ee2ccf@mail.gmail.com>

On Jan 19, 2008 4:42 AM, Orion Delwaterman <orion at mh2o.com> wrote:
>
>
>
>
> What does the method query_all_workers() do?  There is no documentation on
> what this method returns (1.0.1 version).

It returns status/result objects for all the available workers.

From dale at dalecook.us  Sat Jan 19 21:20:32 2008
From: dale at dalecook.us (Dale Cook)
Date: Sat, 19 Jan 2008 18:20:32 -0800
Subject: [Backgroundrb-devel] Upgrading to 2.0
Message-ID: <3b4d152e0801191820i6322a6eesc2595363c59b9261@mail.gmail.com>

We're currently using a really old version of BackgrounDRb in a small
production environment and we're thinking about upgrading to 2.0, primarily
because we want to push it a little harder and have it manage a few more
things. Problem is, we're a little confused on some changes that have been
made to the new version. Perhaps these are stupid questions, or perhaps they
have been answered elsewhere, but we can't find them in the mailing list
archive, and the current documentation doesn't seem to address them, so here
goes:

1. Auto-generated job keys.
In our current version we can do something like

@job_key = MiddleMan.new_worker(:class => :test_worker, :args => {"some args
go here"})

which will return back a automatically generated random job key that we can
then use to access the worker at a later stage. 2.0 doesn't seem to do this,
but instead requires us to generate a job key for it - is this right or are
we just missing something?

2. Accessing a particular working
In our version we can access a particular worker by doing something like:

my_worker = MiddleMan.get_worker(@job_key)

which, of course, returns our worker. There doesn't seem to be a similar
function for 2.0. We've seen the example for getting all the workers, but
not an example for getting one particular worker. Is this possible?

3. Accessing worker attributes
We have some workers that are defined something like this:

class TestWorker < BackgrounDRb::Rails

  attr_reader :progress
  attr_reader :total
  attr_reader :status

  def do_work(args = nil)

    @status = "started"
    @total = some_object.length

    #blah blah blah...

    @status = "working"
    @progress += 1

    #blah blah blah...

    @status = "complete"

  end

end

For long running tasks we'd grab the worker object similar to the example
given in 2 above and then query it's parameters like so

puts my_worker.status
puts my_worker.length

while(my_worker.progress < 100)
{
  puts my_worker.progress
}

Now, we realise that you can use something like MiddleMan.ask_status(:worker
=> :foo_worker) but it seems that it's possible to only have one attribute
per worker. Is it possible to have multiple attributes per worker and if so
how do we access them?

We've actually spent some time trying out a few of these things but we've
had very little success. I suspect that the new 2.0 system is so different
from the version that we're using that we're just having a difficult time
getting our heads around it. Any help here would be greatly appreciated.

Dale
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080119/077ce5a8/attachment.html 

From gethemant at gmail.com  Sat Jan 19 22:44:08 2008
From: gethemant at gmail.com (hemant)
Date: Sun, 20 Jan 2008 09:14:08 +0530
Subject: [Backgroundrb-devel] Upgrading to 2.0
In-Reply-To: <3b4d152e0801191820i6322a6eesc2595363c59b9261@mail.gmail.com>
References: <3b4d152e0801191820i6322a6eesc2595363c59b9261@mail.gmail.com>
Message-ID: <fb283d5e0801191944w5d1f45b3ka6885cbb6ef7ccb6@mail.gmail.com>

Hi,

On Jan 20, 2008 7:50 AM, Dale Cook <dale at dalecook.us> wrote:
> We're currently using a really old version of BackgrounDRb in a small
> production environment and we're thinking about upgrading to 2.0, primarily
> because we want to push it a little harder and have it manage a few more
> things. Problem is, we're a little confused on some changes that have been
> made to the new version. Perhaps these are stupid questions, or perhaps they
> have been answered elsewhere, but we can't find them in the mailing list
> archive, and the current documentation doesn't seem to address them, so here
> goes:
>
> 1. Auto-generated job keys.
> In our current version we can do something like
>
> @job_key = MiddleMan.new_worker(:class => :test_worker, :args => {"some args
> go here"})
>
> which will return back a automatically generated random job key that we can
> then use to access the worker at a later stage. 2.0 doesn't seem to do this,
> but instead requires us to generate a job key for it - is this right or are
> we just missing something?

Yes thats right, you will have to specify your job_key while calling the worker.
Even in older versions, you had the option to supply the job_key.

>
> 2. Accessing a particular working
> In our version we can access a particular worker by doing something like:
>
> my_worker = MiddleMan.get_worker(@job_key)
>
> which, of course, returns our worker. There doesn't seem to be a similar
> function for 2.0. We've seen the example for getting all the workers, but
> not an example for getting one particular worker. Is this possible?

There is a major difference that in older version you had access to
worker environment through DRb.
For stability sake, there are no threads and hence you don't have
direct access to worker environment.

If you want to execute a task, you have to use:

MiddleMan.ask_work(:worker => :foo_worker,:job_key => @job_key)

>
> 3. Accessing worker attributes
> We have some workers that are defined something like this:
>
> class TestWorker < BackgrounDRb::Rails
>
>   attr_reader :progress
>   attr_reader :total
>   attr_reader :status
>
>   def do_work(args = nil)
>
>     @status = "started"
>     @total = some_object.length
>
>     #blah blah blah...
>
>     @status = "working"
>     @progress += 1
>
>     #blah blah blah...
>
>     @status = "complete"
>
>   end
>
> end
>
> For long running tasks we'd grab the worker object similar to the example
> given in 2 above and then query it's parameters like so
>
> puts my_worker.status
> puts my_worker.length
>
> while(my_worker.progress < 100)
> {
>   puts my_worker.progress
> }
>
> Now, we realise that you can use something like MiddleMan.ask_status(:worker
> => :foo_worker) but it seems that it's possible to only have one attribute
> per worker. Is it possible to have multiple attributes per worker and if so
> how do we access them?
>
> We've actually spent some time trying out a few of these things but we've
> had very little success. I suspect that the new 2.0 system is so different
> from the version that we're using that we're just having a difficult time
> getting our heads around it. Any help here would be greatly appreciated.

Again remember you don't have direct access to worker environment
through MiddleMan. Its probably good and bad. Good because it reduces
complexity and bad because it sacrifices on flexibility. However, you
can easily compile your attributes in a hash and access it from rails.
For example:

class HelloWorker < BackgrounDRb::MetaWorker
  set_worker_name :hello_worker
  def create(args = nil)
    @worker_attributes = {:length => 0, :percentage => 0}
    register_status(@worker_status)
  end

   def start_work
     loop do
        # do some processing
     end
     @worker_attributes[:length] = 100
     @worker_attributes[:percentage] = 100
     register_status(@worker_attributes)
  end
end

Issue of not able to access worker attributes, is it a blocker for you?


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From dale at dalecook.us  Sun Jan 20 16:50:22 2008
From: dale at dalecook.us (Dale Cook)
Date: Sun, 20 Jan 2008 13:50:22 -0800
Subject: [Backgroundrb-devel] Upgrading to 2.0
In-Reply-To: <fb283d5e0801191944w5d1f45b3ka6885cbb6ef7ccb6@mail.gmail.com>
References: <3b4d152e0801191820i6322a6eesc2595363c59b9261@mail.gmail.com>
	<fb283d5e0801191944w5d1f45b3ka6885cbb6ef7ccb6@mail.gmail.com>
Message-ID: <3b4d152e0801201350g46ead192o7fd9f9de8f09103c@mail.gmail.com>

Thanks for the reply. It seems that there's certainly a fundamental
difference between our version and 2.0 - I think this has been causing our
confusion.

None of this is a show stopper for us, but let me see if I've got this
straight, let's say we had a worker that took a fairly long running task
(lets say it pulls down csv data from another server and processes it) and
was capable of updating a number of attributes so that we could report back
to the user that status and progress. Would it look something like this?

The worker code:

class CsvWorker < BackgrounDRb::MetaWorker

  set_worker_name :csv_worker

  def create(args = nil)
    @worker_attributes = {:status => 0, total => 0, :progress => 0}
    register_status(@worker_status)
  end

  def process_file(args = nil)

    #these values are necessary to login to the external csv server
    login = args[:login]
    password = args[:password]
    server_location = args[:server]

    @worker_attributes[:total] = 0
    @worker_attributes[:progress] = 0
    @worker_attributes[:status] = "contacting server"
    register_status(@worker_attributes)

    # not shown here: login to the server, download the file, load the lines
of the csv file into an array, etc....

    @worker_attributes[:total] = csv_array.length
    @worker_attributes[:status] = "processing"
    register_status(@worker_attributes)

    for record in csv_array

      #process the record, or whatever....

      @worker_attributes[:progress] += 1
      register_status(@worker_attributes)

    end

 end

end

Then our controller code would look something like:

class ProcessesController < ApplicationController

  def import

    #generate_job_key generates a new random key for us
    @job_key = generate_job_key

    MiddleMan.new_worker(:worker => :csv_worker, :job_key => @job_key)

    MiddleMan.ask_work(:worker => :csv_worker, :job_key => @job_key,
:worker_method => :process_file, :data => {:login => "mylogin", :password =>
"mypassword", :server => "192.168.22.55"})

    #send an Ajax.Request function back tot he page to make it access the
status method
    render :update do |page|

      page << remote_function(:url => {:action => :status, :controller =>
"processes", :id => @job_key})

    end

  end

  def status

    results = MiddleMan.ask_status(:worker => :csv_worker, :job_key =
params[:id])

    #update the page with relevent progress information
    render :update do |page|

      if(results[:status] = "processing" )

        page['report-progress'].replace_html "#{results[:progress]} records
processed out of a total of #{results[:total]"

        page << remote_function(:url => {:action => :status, :controller =>
"processes", :id => params[:id]})

      else

        page['report-progress'].replace_html results[:status]

      end

    end

  end

end

I won't include the actual view since it's just a simple page with a "start"
link and a div that provides the update information.

We've already tried something like this (actually this is a simplification
of the test code we're using) but we still haven't had much success -
perhaps there's simply something wrong here.

Anyway, I also wanted to say thanks to everyone who's worked on
BackgrounDRb, even if the new version doesn't work out for us (I suspect it
will) it's an amazing project and we're really thankful that people have
taken the time and energy to develop it.

Dale

On Jan 19, 2008 7:44 PM, hemant <gethemant at gmail.com> wrote:

> Hi,
>
> On Jan 20, 2008 7:50 AM, Dale Cook <dale at dalecook.us> wrote:
> > We're currently using a really old version of BackgrounDRb in a small
> > production environment and we're thinking about upgrading to 2.0,
> primarily
> > because we want to push it a little harder and have it manage a few more
> > things. Problem is, we're a little confused on some changes that have
> been
> > made to the new version. Perhaps these are stupid questions, or perhaps
> they
> > have been answered elsewhere, but we can't find them in the mailing list
> > archive, and the current documentation doesn't seem to address them, so
> here
> > goes:
> >
> > 1. Auto-generated job keys.
> > In our current version we can do something like
> >
> > @job_key = MiddleMan.new_worker(:class => :test_worker, :args => {"some
> args
> > go here"})
> >
> > which will return back a automatically generated random job key that we
> can
> > then use to access the worker at a later stage. 2.0 doesn't seem to do
> this,
> > but instead requires us to generate a job key for it - is this right or
> are
> > we just missing something?
>
> Yes thats right, you will have to specify your job_key while calling the
> worker.
> Even in older versions, you had the option to supply the job_key.
>
> >
> > 2. Accessing a particular working
> > In our version we can access a particular worker by doing something
> like:
> >
> > my_worker = MiddleMan.get_worker(@job_key)
> >
> > which, of course, returns our worker. There doesn't seem to be a similar
> > function for 2.0. We've seen the example for getting all the workers,
> but
> > not an example for getting one particular worker. Is this possible?
>
> There is a major difference that in older version you had access to
> worker environment through DRb.
> For stability sake, there are no threads and hence you don't have
> direct access to worker environment.
>
> If you want to execute a task, you have to use:
>
> MiddleMan.ask_work(:worker => :foo_worker,:job_key => @job_key)
>
> >
> > 3. Accessing worker attributes
> > We have some workers that are defined something like this:
> >
> > class TestWorker < BackgrounDRb::Rails
> >
> >   attr_reader :progress
> >   attr_reader :total
> >   attr_reader :status
> >
> >   def do_work(args = nil)
> >
> >     @status = "started"
> >     @total = some_object.length
> >
> >     #blah blah blah...
> >
> >     @status = "working"
> >     @progress += 1
> >
> >     #blah blah blah...
> >
> >     @status = "complete"
> >
> >   end
> >
> > end
> >
> > For long running tasks we'd grab the worker object similar to the
> example
> > given in 2 above and then query it's parameters like so
> >
> > puts my_worker.status
> > puts my_worker.length
> >
> > while(my_worker.progress < 100)
> > {
> >   puts my_worker.progress
> > }
> >
> > Now, we realise that you can use something like MiddleMan.ask_status
> (:worker
> > => :foo_worker) but it seems that it's possible to only have one
> attribute
> > per worker. Is it possible to have multiple attributes per worker and if
> so
> > how do we access them?
> >
> > We've actually spent some time trying out a few of these things but
> we've
> > had very little success. I suspect that the new 2.0 system is so
> different
> > from the version that we're using that we're just having a difficult
> time
> > getting our heads around it. Any help here would be greatly appreciated.
>
> Again remember you don't have direct access to worker environment
> through MiddleMan. Its probably good and bad. Good because it reduces
> complexity and bad because it sacrifices on flexibility. However, you
> can easily compile your attributes in a hash and access it from rails.
> For example:
>
> class HelloWorker < BackgrounDRb::MetaWorker
>  set_worker_name :hello_worker
>  def create(args = nil)
>    @worker_attributes = {:length => 0, :percentage => 0}
>    register_status(@worker_status)
>  end
>
>   def start_work
>     loop do
>        # do some processing
>     end
>     @worker_attributes[:length] = 100
>     @worker_attributes[:percentage] = 100
>     register_status(@worker_attributes)
>  end
> end
>
> Issue of not able to access worker attributes, is it a blocker for you?
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080120/5370ec97/attachment-0001.html 

From gethemant at gmail.com  Sun Jan 20 17:14:25 2008
From: gethemant at gmail.com (hemant)
Date: Mon, 21 Jan 2008 03:44:25 +0530
Subject: [Backgroundrb-devel] Upgrading to 2.0
In-Reply-To: <3b4d152e0801201350g46ead192o7fd9f9de8f09103c@mail.gmail.com>
References: <3b4d152e0801191820i6322a6eesc2595363c59b9261@mail.gmail.com>
	<fb283d5e0801191944w5d1f45b3ka6885cbb6ef7ccb6@mail.gmail.com>
	<3b4d152e0801201350g46ead192o7fd9f9de8f09103c@mail.gmail.com>
Message-ID: <fb283d5e0801201414j30ed7cdepa2c2dfdf1e15d506@mail.gmail.com>

Hi Dale,

On Jan 21, 2008 3:20 AM, Dale Cook <dale at dalecook.us> wrote:
> Thanks for the reply. It seems that there's certainly a fundamental
> difference between our version and 2.0 - I think this has been causing our
> confusion.
>
> None of this is a show stopper for us, but let me see if I've got this
> straight, let's say we had a worker that took a fairly long running task
> (lets say it pulls down csv data from another server and processes it) and
> was capable of updating a number of attributes so that we could report back
> to the user that status and progress. Would it look something like this?
>
> The worker code:
>
> class CsvWorker < BackgrounDRb::MetaWorker
>
>   set_worker_name :csv_worker
>
>   def create(args = nil)
>     @worker_attributes = {:status => 0, total => 0, :progress => 0}
>
>     register_status(@worker_status)
>   end
>
>   def process_file(args = nil)
>
>     #these values are necessary to login to the external csv server
>     login = args[:login]
>     password = args[:password]
>     server_location = args[:server]
>
>     @worker_attributes[:total] = 0
>      @worker_attributes[:progress] = 0
>      @worker_attributes[:status] = "contacting server"
>      register_status(@worker_attributes)
>
>     # not shown here: login to the server, download the file, load the lines
> of the csv file into an array, etc....
>
>     @worker_attributes[:total] = csv_array.length
>     @worker_attributes[:status] = "processing"
>     register_status(@worker_attributes)
>
>     for record in csv_array
>
>       #process the record, or whatever....
>
>        @worker_attributes[:progress] += 1
>
>        register_status(@worker_attributes)
>
>     end
>
>  end
>
> end
>
> Then our controller code would look something like:
>
> class ProcessesController < ApplicationController
>
>   def import
>
>     #generate_job_key generates a new random key for us
>     @job_key = generate_job_key
>
>     MiddleMan.new_worker(:worker => :csv_worker, :job_key => @job_key)
>
>     MiddleMan.ask_work(:worker => :csv_worker, :job_key => @job_key,
> :worker_method => :process_file, :data => {:login => "mylogin", :password =>
> "mypassword", :server => " 192.168.22.55"})
>
>     #send an Ajax.Request function back tot he page to make it access the
> status method
>     render :update do |page|
>
>       page << remote_function(:url => {:action => :status, :controller =>
> "processes", :id => @job_key})
>
>     end
>
>   end
>
>   def status
>
>     results = MiddleMan.ask_status(:worker => :csv_worker, :job_key =
> params[:id])
>
>     #update the page with relevent progress information
>     render :update do |page|
>
>       if(results[:status] = "processing" )
>
>         page['report-progress'].replace_html "#{results[:progress]} records
> processed out of a total of #{results[:total]"
>
>         page << remote_function(:url => {:action => :status, :controller =>
> "processes", :id => params[:id]})
>
>       else
>
>         page['report-progress'].replace_html results[:status]
>
>       end
>
>     end
>
>   end
>
> end
>
> I won't include the actual view since it's just a simple page with a "start"
> link and a div that provides the update information.
>
> We've already tried something like this (actually this is a simplification
> of the test code we're using) but we still haven't had much success -
> perhaps there's simply something wrong here.
>
> Anyway, I also wanted to say thanks to everyone who's worked on
> BackgrounDRb, even if the new version doesn't work out for us (I suspect it
> will) it's an amazing project and we're really thankful that people have
> taken the time and energy to develop it.
>

Yes your code looks alright, at least the part thats dealing with
BackgrounDRb. So whats the error you are getting? If you tell us about
the problem, we can surely straighten it out. Which part is not
working? Do you have errors in RAILS_ROOT/log/backgroundrb_debug.log ?
You can turn log messages in foreground and find out where its failing
and stuff like that.

From orion at mh2o.com  Mon Jan 21 10:11:50 2008
From: orion at mh2o.com (Orion Delwaterman)
Date: Mon, 21 Jan 2008 10:11:50 -0500
Subject: [Backgroundrb-devel] Query All Worker
Message-ID: <205576B004DDBD438F6560241E0751A0157F912F@sbsexc02.mh2onyc.com>

Maybe I am doing something wrong but query_all_workers() does not appear
to return any results from my workers.  Does the results object have to
be some kind of hash or is this just some weird bug?

Worker:

class OriontestWorker < BackgrounDRb::MetaWorker
  set_worker_name :oriontest_worker
#  set_no_auto_load true
  
  def create(args = nil)
    # this method is called, when worker is loaded for the first time
  end
  
  def get_random
    register_status(rand(200))
    register_result(RAILS_ENV)
    true
  end
  
end

IRB testing:
Loading development environment.
>> MiddleMan.query_all_workers
=> {:log_worker=>nil, :oriontest_worker=>nil}

-----Original Message-----
From: hemant [mailto:gethemant at gmail.com] 
Sent: Saturday, January 19, 2008 6:58 AM
To: Orion Delwaterman
Cc: backgroundrb-devel at rubyforge.org
Subject: Re: [Backgroundrb-devel] Query All Worker

On Jan 19, 2008 4:42 AM, Orion Delwaterman <orion at mh2o.com> wrote:
>
>
>
>
> What does the method query_all_workers() do?  There is no
documentation on
> what this method returns (1.0.1 version).

It returns status/result objects for all the available workers.

From orion at mh2o.com  Mon Jan 21 11:03:20 2008
From: orion at mh2o.com (Orion Delwaterman)
Date: Mon, 21 Jan 2008 11:03:20 -0500
Subject: [Backgroundrb-devel] Query All Worker
Message-ID: <205576B004DDBD438F6560241E0751A0157F9130@sbsexc02.mh2onyc.com>

Whoops mixup in my code here, I fixed it but I am still getting the same
issue where either register_result does not properly log the result or
query_all_workers() is not returning the result:

CODE:

class OriontestWorker < BackgrounDRb::MetaWorker
  set_worker_name :oriontest_worker
#  set_no_auto_load true
  
  def create(args = nil)
    # this method is called, when worker is loaded for the first time
    register_status(rand(200))
    res = RAILS_ENV
    log res
    register_result(res)
  end
  
  def get_methods
    self.methods.sort
  end
  
  def test_results
    res = rand(200)
    log res
    register_result(res)
    res
  end
  
  def get_msg_writer
    self.msg_writer.inspect
  end
  
end

IRB:

[orion at OrionFoot hydra]$ ruby script/console
Loading development environment.
>> MiddleMan.query_all_workers
=> {:log_worker=>nil, :oriontest_worker=>nil}
>> MiddleMan.send_request :worker => :oriontest_worker, :worker_method
=> :test_results
=> {:data=>73, :client_signature=>12, :type=>:response}
>> MiddleMan.query_all_workers
=> {:log_worker=>nil, :oriontest_worker=>nil}
>>

-----Original Message-----
From: Orion Delwaterman 
Sent: Monday, January 21, 2008 10:12 AM
To: 'hemant'
Cc: backgroundrb-devel at rubyforge.org
Subject: RE: [Backgroundrb-devel] Query All Worker

Maybe I am doing something wrong but query_all_workers() does not appear
to return any results from my workers.  Does the results object have to
be some kind of hash or is this just some weird bug?

Worker:

class OriontestWorker < BackgrounDRb::MetaWorker
  set_worker_name :oriontest_worker
#  set_no_auto_load true
  
  def create(args = nil)
    # this method is called, when worker is loaded for the first time
  end
  
  def get_random
    register_status(rand(200))
    register_result(RAILS_ENV)
    true
  end
  
end

IRB testing:
Loading development environment.
>> MiddleMan.query_all_workers
=> {:log_worker=>nil, :oriontest_worker=>nil}

-----Original Message-----
From: hemant [mailto:gethemant at gmail.com] 
Sent: Saturday, January 19, 2008 6:58 AM
To: Orion Delwaterman
Cc: backgroundrb-devel at rubyforge.org
Subject: Re: [Backgroundrb-devel] Query All Worker

On Jan 19, 2008 4:42 AM, Orion Delwaterman <orion at mh2o.com> wrote:
>
>
>
>
> What does the method query_all_workers() do?  There is no
documentation on
> what this method returns (1.0.1 version).

It returns status/result objects for all the available workers.

From dale at dalecook.us  Mon Jan 21 14:28:26 2008
From: dale at dalecook.us (Dale Cook)
Date: Mon, 21 Jan 2008 11:28:26 -0800
Subject: [Backgroundrb-devel] Upgrading to 2.0
In-Reply-To: <fb283d5e0801201414j30ed7cdepa2c2dfdf1e15d506@mail.gmail.com>
References: <3b4d152e0801191820i6322a6eesc2595363c59b9261@mail.gmail.com>
	<fb283d5e0801191944w5d1f45b3ka6885cbb6ef7ccb6@mail.gmail.com>
	<3b4d152e0801201350g46ead192o7fd9f9de8f09103c@mail.gmail.com>
	<fb283d5e0801201414j30ed7cdepa2c2dfdf1e15d506@mail.gmail.com>
Message-ID: <3b4d152e0801211128n323f5937g4973a34c1bea6748@mail.gmail.com>

Actually, thanks to your guidance, we played around with it the morning and
it's now working like a charm. I really think having a better understanding
of how it works now really helped - thanks a bunch.

Dale

On Jan 20, 2008 2:14 PM, hemant <gethemant at gmail.com> wrote:

> Hi Dale,
>
> On Jan 21, 2008 3:20 AM, Dale Cook <dale at dalecook.us> wrote:
> > Thanks for the reply. It seems that there's certainly a fundamental
> > difference between our version and 2.0 - I think this has been causing
> our
> > confusion.
> >
> > None of this is a show stopper for us, but let me see if I've got this
> > straight, let's say we had a worker that took a fairly long running task
> > (lets say it pulls down csv data from another server and processes it)
> and
> > was capable of updating a number of attributes so that we could report
> back
> > to the user that status and progress. Would it look something like this?
> >
> > The worker code:
> >
> > class CsvWorker < BackgrounDRb::MetaWorker
> >
> >   set_worker_name :csv_worker
> >
> >   def create(args = nil)
> >     @worker_attributes = {:status => 0, total => 0, :progress => 0}
> >
> >     register_status(@worker_status)
> >   end
> >
> >   def process_file(args = nil)
> >
> >     #these values are necessary to login to the external csv server
> >     login = args[:login]
> >     password = args[:password]
> >     server_location = args[:server]
> >
> >     @worker_attributes[:total] = 0
> >      @worker_attributes[:progress] = 0
> >      @worker_attributes[:status] = "contacting server"
> >      register_status(@worker_attributes)
> >
> >     # not shown here: login to the server, download the file, load the
> lines
> > of the csv file into an array, etc....
> >
> >     @worker_attributes[:total] = csv_array.length
> >     @worker_attributes[:status] = "processing"
> >     register_status(@worker_attributes)
> >
> >     for record in csv_array
> >
> >       #process the record, or whatever....
> >
> >        @worker_attributes[:progress] += 1
> >
> >        register_status(@worker_attributes)
> >
> >     end
> >
> >  end
> >
> > end
> >
> > Then our controller code would look something like:
> >
> > class ProcessesController < ApplicationController
> >
> >   def import
> >
> >     #generate_job_key generates a new random key for us
> >     @job_key = generate_job_key
> >
> >     MiddleMan.new_worker(:worker => :csv_worker, :job_key => @job_key)
> >
> >     MiddleMan.ask_work(:worker => :csv_worker, :job_key => @job_key,
> > :worker_method => :process_file, :data => {:login => "mylogin",
> :password =>
> > "mypassword", :server => " 192.168.22.55"})
> >
> >     #send an Ajax.Request function back tot he page to make it access
> the
> > status method
> >     render :update do |page|
> >
> >       page << remote_function(:url => {:action => :status, :controller
> =>
> > "processes", :id => @job_key})
> >
> >     end
> >
> >   end
> >
> >   def status
> >
> >     results = MiddleMan.ask_status(:worker => :csv_worker, :job_key =
> > params[:id])
> >
> >     #update the page with relevent progress information
> >     render :update do |page|
> >
> >       if(results[:status] = "processing" )
> >
> >         page['report-progress'].replace_html "#{results[:progress]}
> records
> > processed out of a total of #{results[:total]"
> >
> >         page << remote_function(:url => {:action => :status, :controller
> =>
> > "processes", :id => params[:id]})
> >
> >       else
> >
> >         page['report-progress'].replace_html results[:status]
> >
> >       end
> >
> >     end
> >
> >   end
> >
> > end
> >
> > I won't include the actual view since it's just a simple page with a
> "start"
> > link and a div that provides the update information.
> >
> > We've already tried something like this (actually this is a
> simplification
> > of the test code we're using) but we still haven't had much success -
> > perhaps there's simply something wrong here.
> >
> > Anyway, I also wanted to say thanks to everyone who's worked on
> > BackgrounDRb, even if the new version doesn't work out for us (I suspect
> it
> > will) it's an amazing project and we're really thankful that people have
> > taken the time and energy to develop it.
> >
>
> Yes your code looks alright, at least the part thats dealing with
> BackgrounDRb. So whats the error you are getting? If you tell us about
> the problem, we can surely straighten it out. Which part is not
> working? Do you have errors in RAILS_ROOT/log/backgroundrb_debug.log ?
> You can turn log messages in foreground and find out where its failing
> and stuff like that.
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080121/8b353a29/attachment.html 

From swertyui at gmail.com  Mon Jan 21 19:03:58 2008
From: swertyui at gmail.com (steef nobody)
Date: Mon, 21 Jan 2008 19:03:58 -0500
Subject: [Backgroundrb-devel] Newbie question on Scheduling
In-Reply-To: <3262a2db0801211601v6f355992xc4def9dbcc262f53@mail.gmail.com>
References: <3262a2db0801211601v6f355992xc4def9dbcc262f53@mail.gmail.com>
Message-ID: <3262a2db0801211603g24dac8a7x1021081c5ed5c1f1@mail.gmail.com>

I've scheduled cron jobs etc before, but I'm curious as to how to layout a
worker properly.  Here's a basic example of what one of my workers looks
like:

class FooWorker < BackgrounDRb::MetaWorker
set_worker_name :foo_worker
def create(args = nil)
logger.info("Starting test: #{Time.now.strftime("%T")}")
run_test
logger.info("Finished test: #{Time.now.strftime("%T")}")
exit
end
def run_test
logger.info("aaaa")
sleep(30)
logger.info("bbbb")
end
end

I'd set my workers up like this while I was testing things.  I'd modify them
and restart backgroundrb to see how things went.  I have things where I want
them to be now, and am trying to get them scheduled properly.

Let's say I wanted the above worker to run every day at 16:00.  How would
that look in my backgroundrb.yml file?  Should I keep things out of the
create method?  The documentation seems to suggest I create my own methods
and schedule each individually without calling exit to kill the worker.

- Steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080121/73782173/attachment-0001.html 

From james at howison.name  Mon Jan 21 22:57:48 2008
From: james at howison.name (James Howison)
Date: Mon, 21 Jan 2008 22:57:48 -0500
Subject: [Backgroundrb-devel] Unable to load Models which reference plugins?
Message-ID: <F87ADCB4-112D-4DF3-BE81-E7418F6B8F76@howison.name>

I'm using a plugin which adds a method to ActiveRecord::Base

http://similetimelinerailshelper.googlecode.com/svn/trunk/simile_timeline/

This adds an "acts_as" type declaration to the Models, via the file in  
vendor/plugins/simile_timeline/lib/simile_timeline.rb Rails is working  
with these declarations, and they are being used successfully in  
'regular Rails'.

When I run script/backgroundrb start, I get an error saying that  
method is not found (error follows sig).  Backgroundrb is able to  
start up with those acts_as declarations commented out.

Presumably I need to require the plugin file somewhere, but I wondered  
if backgroundrb might be meant to load the same things as Rails, by  
default.

Thanks,
James

[james at MinistryOfInformation2]$script/backgroundrb start
/Library/Ruby/Gems/1.8/gems/activerecord-2.0.2/lib/active_record/ 
base.rb:1532:in `method_missing': undefined method  
`acts_as_simile_timeline_event' for #<Class:0x11655fc> (NoMethodError)
	from /Users/james/Documents/University/Syracuse/Research/Dissertation/ 
fromSVN/Dissertation/genrebrowser/app/models/event.rb:9
	from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in  
`gem_original_require'
	from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in `require'
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/ 
active_support/dependencies.rb:496:in `require'
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/ 
active_support/dependencies.rb:342:in `new_constants_in'
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/ 
active_support/dependencies.rb:496:in `require'
	from /Users/james/Documents/University/Syracuse/Research/Dissertation/ 
fromSVN/Dissertation/genrebrowser/vendor/plugins/backgroundrb/ 
framework/packet.rb:10
	from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in  
`gem_original_require'
	from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in `require'
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/ 
active_support/dependencies.rb:496:in `require'
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/ 
active_support/dependencies.rb:342:in `new_constants_in'
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/ 
active_support/dependencies.rb:496:in `require'
	from script/backgroundrb:16


From gapeldoorn at wehkamp.nl  Tue Jan 22 03:29:42 2008
From: gapeldoorn at wehkamp.nl (Ger Apeldoorn)
Date: Tue, 22 Jan 2008 09:29:42 +0100
Subject: [Backgroundrb-devel] Cannot connect when spawning new workers on
	demand
Message-ID: <1200990582.9575.20.camel@localhost>

Hi!

I'm using the latest&greatest backgroundrb with rails 2.0.2 on ubuntu
dapper.

I've made an app that lets you run a query on several servers at once,
which are selected at runtime. Therefore, for each server that is
selected, I spawn a new worker and assign its work.
After everything is completed, the workers are deleted.

I often get this error:
BackgrounDRb::BdrbConnError in Multi com/indexController#uitvoeren 
Not able to connect

Here's the code that spawns the workers: (sorry for the linewraps)

  def uitvoeren
    # put selected servers in a session
    session[:selectedservers] = params[:servers]
    session[:running_cmd] = 1
    @selected_servers = System.find_all_by_id(params[:servers])
    @selected_servers.each do |svr|
      # spawn new process
      MiddleMan.new_worker(:worker => :remote_command_worker, :job_key
=> svr.name)
      MiddleMan.ask_work(:worker => :remote_command_worker, :job_key =>
svr.name, :worker_method => :do_work, :data => {:svr =>
svr.name, :command =>
MultiComCommand.find_by_id(params[:commando]).command})
    end
  end

When I select only a few servers, it works just fine! There is no
definable threshold, I've had failures with only 4 servers selected and
success with 12 servers selected.

After such a connection error, the worker processes are still running:
$ ps -ef | grep worker
09:08 ?        00:00:00 ruby log_worker               
09:08 ?        00:00:00 ruby remote_command_worker    
09:08 ?        00:00:00 ruby remote_command_worker    
09:08 ?        00:00:00 ruby remote_command_worker    
09:08 ?        00:00:00 ruby remote_command_worker    
09:08 ?        00:00:00 ruby remote_command_worker 

Your help is VERY MUCH appreciated!

Thanks,
Ger.



From gethemant at gmail.com  Tue Jan 22 03:59:04 2008
From: gethemant at gmail.com (hemant)
Date: Tue, 22 Jan 2008 14:29:04 +0530
Subject: [Backgroundrb-devel] Newbie question on Scheduling
In-Reply-To: <3262a2db0801211603g24dac8a7x1021081c5ed5c1f1@mail.gmail.com>
References: <3262a2db0801211601v6f355992xc4def9dbcc262f53@mail.gmail.com>
	<3262a2db0801211603g24dac8a7x1021081c5ed5c1f1@mail.gmail.com>
Message-ID: <fb283d5e0801220059k50015031t462ea8f4247ac90e@mail.gmail.com>

Hi,

On Jan 22, 2008 5:33 AM, steef nobody <swertyui at gmail.com> wrote:
> I've scheduled cron jobs etc before, but I'm curious as to how to layout a
> worker properly.  Here's a basic example of what one of my workers looks
> like:
>
> class FooWorker < BackgrounDRb::MetaWorker
>
> set_worker_name :foo_worker
> def create(args = nil)
> logger.info("Starting test: #{Time.now.strftime("%T")}")
> run_test
>  logger.info("Finished test: #{Time.now.strftime("%T")}")
> exit
> end
> def run_test
> logger.info("aaaa")
> sleep(30)
> logger.info("bbbb")
>  end
> end

Never use sleep() in workers, its calling for trouble.

>
> I'd set my workers up like this while I was testing things.  I'd modify them
> and restart backgroundrb to see how things went.  I have things where I want
> them to be now, and am trying to get them scheduled properly.
>
> Let's say I wanted the above worker to run every day at 16:00.  How would
> that look in my backgroundrb.yml file?  Should I keep things out of the
> create method?  The documentation seems to suggest I create my own methods
> and schedule each individually without calling exit to kill the worker.

You can use cron styled scheduling with trigger option:

0 0 16 * * * *

From gethemant at gmail.com  Tue Jan 22 04:03:44 2008
From: gethemant at gmail.com (hemant)
Date: Tue, 22 Jan 2008 14:33:44 +0530
Subject: [Backgroundrb-devel] Unable to load Models which reference
	plugins?
In-Reply-To: <F87ADCB4-112D-4DF3-BE81-E7418F6B8F76@howison.name>
References: <F87ADCB4-112D-4DF3-BE81-E7418F6B8F76@howison.name>
Message-ID: <fb283d5e0801220103i7fbea170hbf413edcccff3fec@mail.gmail.com>

Hi,

On Jan 22, 2008 9:27 AM, James Howison <james at howison.name> wrote:
> I'm using a plugin which adds a method to ActiveRecord::Base
>
> http://similetimelinerailshelper.googlecode.com/svn/trunk/simile_timeline/
>
> This adds an "acts_as" type declaration to the Models, via the file in
> vendor/plugins/simile_timeline/lib/simile_timeline.rb Rails is working
> with these declarations, and they are being used successfully in
> 'regular Rails'.
>
> When I run script/backgroundrb start, I get an error saying that
> method is not found (error follows sig).  Backgroundrb is able to
> start up with those acts_as declarations commented out.
>
> Presumably I need to require the plugin file somewhere, but I wondered
> if backgroundrb might be meant to load the same things as Rails, by
> default.
>
> Thanks,
> James
>
> [james at MinistryOfInformation2]$script/backgroundrb start
> /Library/Ruby/Gems/1.8/gems/activerecord-2.0.2/lib/active_record/
> base.rb:1532:in `method_missing': undefined method
> `acts_as_simile_timeline_event' for #<Class:0x11655fc> (NoMethodError)
>         from /Users/james/Documents/University/Syracuse/Research/Dissertation/
> fromSVN/Dissertation/genrebrowser/app/models/event.rb:9
>         from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in
> `gem_original_require'
>         from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in `require'
>         from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> active_support/dependencies.rb:496:in `require'
>         from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> active_support/dependencies.rb:342:in `new_constants_in'
>         from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> active_support/dependencies.rb:496:in `require'
>         from /Users/james/Documents/University/Syracuse/Research/Dissertation/
> fromSVN/Dissertation/genrebrowser/vendor/plugins/backgroundrb/
> framework/packet.rb:10
>         from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in
> `gem_original_require'
>         from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in `require'
>         from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> active_support/dependencies.rb:496:in `require'
>         from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> active_support/dependencies.rb:342:in `new_constants_in'
>         from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> active_support/dependencies.rb:496:in `require'
>         from script/backgroundrb:16

Which version of BackgrounDRb you are running? Use either trunk or 1.0.1.
I remember version 1.0 has aggresive loading of models and hence you
will get errors, if your model requires other plugins.
But moving to 1.0.1 and trunk, I removed aggresive loading and model
environment is loaded same as rails loads them.

Tell me the version, I will have a look at issue.


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Tue Jan 22 04:46:45 2008
From: gethemant at gmail.com (hemant)
Date: Tue, 22 Jan 2008 15:16:45 +0530
Subject: [Backgroundrb-devel] Query All Worker
In-Reply-To: <205576B004DDBD438F6560241E0751A0157F9130@sbsexc02.mh2onyc.com>
References: <205576B004DDBD438F6560241E0751A0157F9130@sbsexc02.mh2onyc.com>
Message-ID: <fb283d5e0801220146r5102f03exfeb12e15006b95fa@mail.gmail.com>

Hi Orion,

There was a bug with query_all_workers, I fixed it. So you can sync
with latest source code from trunk and get it working.

On Jan 21, 2008 9:33 PM, Orion Delwaterman <orion at mh2o.com> wrote:
> Whoops mixup in my code here, I fixed it but I am still getting the same
> issue where either register_result does not properly log the result or
> query_all_workers() is not returning the result:
>
> CODE:
>
> class OriontestWorker < BackgrounDRb::MetaWorker
>   set_worker_name :oriontest_worker
> #  set_no_auto_load true
>
>   def create(args = nil)
>     # this method is called, when worker is loaded for the first time
>     register_status(rand(200))
>     res = RAILS_ENV
>     log res
>     register_result(res)
>   end
>
>   def get_methods
>     self.methods.sort
>   end
>
>   def test_results
>     res = rand(200)
>     log res
>     register_result(res)
>     res
>   end
>
>   def get_msg_writer
>     self.msg_writer.inspect
>   end
>
> end
>
> IRB:
>
> [orion at OrionFoot hydra]$ ruby script/console
> Loading development environment.
> >> MiddleMan.query_all_workers
> => {:log_worker=>nil, :oriontest_worker=>nil}
> >> MiddleMan.send_request :worker => :oriontest_worker, :worker_method
> => :test_results
> => {:data=>73, :client_signature=>12, :type=>:response}
> >> MiddleMan.query_all_workers
> => {:log_worker=>nil, :oriontest_worker=>nil}
> >>
>
> -----Original Message-----
> From: Orion Delwaterman
> Sent: Monday, January 21, 2008 10:12 AM
> To: 'hemant'
> Cc: backgroundrb-devel at rubyforge.org
>
> Subject: RE: [Backgroundrb-devel] Query All Worker
>
> Maybe I am doing something wrong but query_all_workers() does not appear
> to return any results from my workers.  Does the results object have to
> be some kind of hash or is this just some weird bug?
>
> Worker:
>
> class OriontestWorker < BackgrounDRb::MetaWorker
>   set_worker_name :oriontest_worker
> #  set_no_auto_load true
>
>   def create(args = nil)
>     # this method is called, when worker is loaded for the first time
>   end
>
>   def get_random
>     register_status(rand(200))
>     register_result(RAILS_ENV)
>     true
>   end
>
> end
>
> IRB testing:
> Loading development environment.
> >> MiddleMan.query_all_workers
> => {:log_worker=>nil, :oriontest_worker=>nil}
>
> -----Original Message-----
> From: hemant [mailto:gethemant at gmail.com]
> Sent: Saturday, January 19, 2008 6:58 AM
> To: Orion Delwaterman
> Cc: backgroundrb-devel at rubyforge.org
> Subject: Re: [Backgroundrb-devel] Query All Worker
>
> On Jan 19, 2008 4:42 AM, Orion Delwaterman <orion at mh2o.com> wrote:
> >
> >
> >
> >
> > What does the method query_all_workers() do?  There is no
> documentation on
> > what this method returns (1.0.1 version).
>
> It returns status/result objects for all the available workers.
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Tue Jan 22 04:48:11 2008
From: gethemant at gmail.com (hemant)
Date: Tue, 22 Jan 2008 15:18:11 +0530
Subject: [Backgroundrb-devel] Cannot connect when spawning new workers
	on demand
In-Reply-To: <1200990582.9575.20.camel@localhost>
References: <1200990582.9575.20.camel@localhost>
Message-ID: <fb283d5e0801220148paaaaff6j6ef6bce8b85cabe0@mail.gmail.com>

Hi,

On Jan 22, 2008 1:59 PM, Ger Apeldoorn <gapeldoorn at wehkamp.nl> wrote:
> Hi!
>
> I'm using the latest&greatest backgroundrb with rails 2.0.2 on ubuntu
> dapper.
>
> I've made an app that lets you run a query on several servers at once,
> which are selected at runtime. Therefore, for each server that is
> selected, I spawn a new worker and assign its work.
> After everything is completed, the workers are deleted.
>
> I often get this error:
> BackgrounDRb::BdrbConnError in Multi com/indexController#uitvoeren
> Not able to connect
>
> Here's the code that spawns the workers: (sorry for the linewraps)
>
>   def uitvoeren
>     # put selected servers in a session
>     session[:selectedservers] = params[:servers]
>     session[:running_cmd] = 1
>     @selected_servers = System.find_all_by_id(params[:servers])
>     @selected_servers.each do |svr|
>       # spawn new process
>       MiddleMan.new_worker(:worker => :remote_command_worker, :job_key
> => svr.name)
>       MiddleMan.ask_work(:worker => :remote_command_worker, :job_key =>
> svr.name, :worker_method => :do_work, :data => {:svr =>
> svr.name, :command =>
> MultiComCommand.find_by_id(params[:commando]).command})
>     end
>   end
>
> When I select only a few servers, it works just fine! There is no
> definable threshold, I've had failures with only 4 servers selected and
> success with 12 servers selected.
>
> After such a connection error, the worker processes are still running:
> $ ps -ef | grep worker
> 09:08 ?        00:00:00 ruby log_worker
> 09:08 ?        00:00:00 ruby remote_command_worker
> 09:08 ?        00:00:00 ruby remote_command_worker
> 09:08 ?        00:00:00 ruby remote_command_worker
> 09:08 ?        00:00:00 ruby remote_command_worker
> 09:08 ?        00:00:00 ruby remote_command_worker
>
> Your help is VERY MUCH appreciated!

Did you check backgroundrb_debug.log ?
You can turn on debugging to foreground with:

:log: foreground

option to further debug this. Something is killing BackgrounDRb master
process. Can you sync with trunk and enable foreground logging and
check on this?

Also, if possible, can we see your worker code?

From gapeldoorn at wehkamp.nl  Tue Jan 22 05:03:11 2008
From: gapeldoorn at wehkamp.nl (Ger Apeldoorn)
Date: Tue, 22 Jan 2008 11:03:11 +0100
Subject: [Backgroundrb-devel] Cannot connect when spawning new
	workers	on demand
In-Reply-To: <fb283d5e0801220148paaaaff6j6ef6bce8b85cabe0@mail.gmail.com>
References: <1200990582.9575.20.camel@localhost>
	<fb283d5e0801220148paaaaff6j6ef6bce8b85cabe0@mail.gmail.com>
Message-ID: <1200996191.9575.27.camel@localhost>

Hi!

Thanks a lot for your reply!

> Did you check backgroundrb_debug.log ?
> You can turn on debugging to foreground with:
> 
> :log: foreground

Where should I put :log: foreground?

Anyway, this comes from log/backgroundrb_debug.log:

--------------------%<--------------------

$ tail -f log/backgroundrb_debug.log 
000000073:      type:start_worker:
                                   worker:remote_command_worker:

job_key"

server06
{:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server06"}
000000115{
:       type:
             do_work:
                     worker:remote_command_worker:     data{:
                                                              command"
megacommandheresvr"
server06:
        job_key:worker_method;
{:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server06", :command=>"megacommandhere"}, :worker=>:remote_command_worker, :job_key=>"server06"}
000000073:      type:start_worker:
                                   worker:remote_command_worker:

job_key"

server12
{:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server12"}
000000115{
:       type:
             do_work:
                     worker:remote_command_worker:     data{:
                                                              command"
megacommandheresvr"
server12:
        job_key:worker_method;
{:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server12", :command=>"megacommandhere"}, :worker=>:remote_command_worker, :job_key=>"server12"}

--------------------%<--------------------

> 
> option to further debug this. Something is killing BackgrounDRb master
> process. Can you sync with trunk and enable foreground logging and
> check on this?

Is there an easy way to update a plugin after it's been installed
besides removing it and checking it out again?

> Also, if possible, can we see your worker code?

Here's my worker:

class RemoteCommandWorker < BackgrounDRb::MetaWorker
  set_worker_name :remote_command_worker
  set_no_auto_load :true

  def create(args = nil)
    register_status("processing")
  end

  def do_work(args = nil)
    result = String.new
    result = "Result for #{args[:command]}:\n #{%
x[running_a_shell_script_here]}"
    register_status(result)
  end
end

Thanks,
Ger.

From james at howison.name  Tue Jan 22 10:14:53 2008
From: james at howison.name (James Howison)
Date: Tue, 22 Jan 2008 10:14:53 -0500
Subject: [Backgroundrb-devel] Unable to load Models which reference
	plugins?
In-Reply-To: <fb283d5e0801220103i7fbea170hbf413edcccff3fec@mail.gmail.com>
References: <F87ADCB4-112D-4DF3-BE81-E7418F6B8F76@howison.name>
	<fb283d5e0801220103i7fbea170hbf413edcccff3fec@mail.gmail.com>
Message-ID: <B46B4352-14CF-4B12-AC02-ACCD13FA678A@howison.name>


On Jan 22, 2008, at 4:03 AM, hemant wrote:

> Hi,
>
> On Jan 22, 2008 9:27 AM, James Howison <james at howison.name> wrote:
>> I'm using a plugin which adds a method to ActiveRecord::Base
>>
>> http://similetimelinerailshelper.googlecode.com/svn/trunk/simile_timeline/
>>
>> This adds an "acts_as" type declaration to the Models, via the file  
>> in
>> vendor/plugins/simile_timeline/lib/simile_timeline.rb Rails is  
>> working
>> with these declarations, and they are being used successfully in
>> 'regular Rails'.
>>
>> When I run script/backgroundrb start, I get an error saying that
>> method is not found (error follows sig).  Backgroundrb is able to
>> start up with those acts_as declarations commented out.
>>
>> Presumably I need to require the plugin file somewhere, but I  
>> wondered
>> if backgroundrb might be meant to load the same things as Rails, by
>> default.
>>
>> Thanks,
>> James
>>
>> [james at MinistryOfInformation2]$script/backgroundrb start
>> /Library/Ruby/Gems/1.8/gems/activerecord-2.0.2/lib/active_record/
>> base.rb:1532:in `method_missing': undefined method
>> `acts_as_simile_timeline_event' for #<Class:0x11655fc>  
>> (NoMethodError)
>>        from /Users/james/Documents/University/Syracuse/Research/ 
>> Dissertation/
>> fromSVN/Dissertation/genrebrowser/app/models/event.rb:9
>>        from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in
>> `gem_original_require'
>>        from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in  
>> `require'
>>        from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
>> active_support/dependencies.rb:496:in `require'
>>        from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
>> active_support/dependencies.rb:342:in `new_constants_in'
>>        from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
>> active_support/dependencies.rb:496:in `require'
>>        from /Users/james/Documents/University/Syracuse/Research/ 
>> Dissertation/
>> fromSVN/Dissertation/genrebrowser/vendor/plugins/backgroundrb/
>> framework/packet.rb:10
>>        from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in
>> `gem_original_require'
>>        from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in  
>> `require'
>>        from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
>> active_support/dependencies.rb:496:in `require'
>>        from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
>> active_support/dependencies.rb:342:in `new_constants_in'
>>        from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
>> active_support/dependencies.rb:496:in `require'
>>        from script/backgroundrb:16
>
> Which version of BackgrounDRb you are running? Use either trunk or  
> 1.0.1.
> I remember version 1.0 has aggresive loading of models and hence you
> will get errors, if your model requires other plugins.
> But moving to 1.0.1 and trunk, I removed aggresive loading and model
> environment is loaded same as rails loads them.
>
> Tell me the version, I will have a look at issue.

Thanks.  Initially I had used piston on the trunk, but now I've used  
svn:externals as in the README file:

[james at MinistryOfInformation2]$svn propedit svn:externals vendor/plugins

backgroundrb http://svn.devjavu.com/backgroundrb/trunk
<snip>
~
~
"svn-prop.tmp" 1L, 55C written
Set new value for property 'svn:externals' on 'vendor/plugins'
$~/Documents/University/Syracuse/Research/Dissertation/fromSVN/ 
Dissertation/genrebrowser
[james at MinistryOfInformation2]$svn up vendor/plugins

Fetching external item into 'vendor/plugins/backgroundrb'
A    vendor/plugins/backgroundrb/test
A    vendor/plugins/backgroundrb/test/meta_worker_test.rb
A    vendor/plugins/backgroundrb/test/backgroundrb_test.rb
A    vendor/plugins/backgroundrb/test/master_worker_test.rb
A    vendor/plugins/backgroundrb/test/bdrb_test_helper.rb
A    vendor/plugins/backgroundrb/cron_trigger.patch
A    vendor/plugins/backgroundrb/Rakefile
A    vendor/plugins/backgroundrb/LICENSE
A    vendor/plugins/backgroundrb/tasks
A    vendor/plugins/backgroundrb/tasks/backgroundrb_tasks.rake
A    vendor/plugins/backgroundrb/TODO.org
A    vendor/plugins/backgroundrb/framework
A    vendor/plugins/backgroundrb/framework/packet_guid.rb
A    vendor/plugins/backgroundrb/framework/cpu_worker.rb
A    vendor/plugins/backgroundrb/framework/event.rb
A    vendor/plugins/backgroundrb/framework/packet_master.rb
A    vendor/plugins/backgroundrb/framework/bin_parser.rb
A    vendor/plugins/backgroundrb/framework/nbio.rb
A    vendor/plugins/backgroundrb/framework/class_helpers.rb
A    vendor/plugins/backgroundrb/framework/core.rb
A    vendor/plugins/backgroundrb/framework/buftok.rb
A    vendor/plugins/backgroundrb/framework/deferrable.rb
A    vendor/plugins/backgroundrb/framework/callback.rb
A    vendor/plugins/backgroundrb/framework/double_keyed_hash.rb
A    vendor/plugins/backgroundrb/framework/connection.rb
A    vendor/plugins/backgroundrb/framework/periodic_event.rb
A    vendor/plugins/backgroundrb/framework/io_worker.rb
A    vendor/plugins/backgroundrb/framework/disconnect_error.rb
A    vendor/plugins/backgroundrb/framework/pimp.rb
A    vendor/plugins/backgroundrb/framework/packet.rb
A    vendor/plugins/backgroundrb/framework/worker.rb
A    vendor/plugins/backgroundrb/framework/meta_pimp.rb
A    vendor/plugins/backgroundrb/README
A    vendor/plugins/backgroundrb/script
A    vendor/plugins/backgroundrb/script/backgroundrb
A    vendor/plugins/backgroundrb/script/bdrb_test_helper.rb
A    vendor/plugins/backgroundrb/init.rb
A    vendor/plugins/backgroundrb/config
A    vendor/plugins/backgroundrb/config/backgroundrb.yml
A    vendor/plugins/backgroundrb/server
A    vendor/plugins/backgroundrb/server/meta_worker.rb
A    vendor/plugins/backgroundrb/server/trigger.rb
A    vendor/plugins/backgroundrb/server/cron_trigger.rb
A    vendor/plugins/backgroundrb/server/invalid_dump_error.rb
A    vendor/plugins/backgroundrb/server/master_worker.rb
A    vendor/plugins/backgroundrb/server/log_worker.rb
A    vendor/plugins/backgroundrb/lib
A    vendor/plugins/backgroundrb/lib/bdrb_conn_error.rb
A    vendor/plugins/backgroundrb/lib/backgroundrb.rb
A    vendor/plugins/backgroundrb/install.rb
A    vendor/plugins/backgroundrb/generators
A    vendor/plugins/backgroundrb/generators/worker
A    vendor/plugins/backgroundrb/generators/worker/worker_generator.rb
A    vendor/plugins/backgroundrb/generators/worker/USAGE
A    vendor/plugins/backgroundrb/generators/worker/templates
A    vendor/plugins/backgroundrb/generators/worker/templates/ 
unit_test.rb
A    vendor/plugins/backgroundrb/generators/worker/templates/worker.rb
A    vendor/plugins/backgroundrb/MIT-LICENSE
A    vendor/plugins/backgroundrb/uninstall.rb
A    vendor/plugins/backgroundrb/examples
A    vendor/plugins/backgroundrb/examples/god_worker.rb
A    vendor/plugins/backgroundrb/examples/foo_controller.rb
A    vendor/plugins/backgroundrb/examples/worker_tests
A    vendor/plugins/backgroundrb/examples/worker_tests/ 
god_worker_test.rb
A    vendor/plugins/backgroundrb/examples/workers
A    vendor/plugins/backgroundrb/examples/workers/xmpp_worker.rb
A    vendor/plugins/backgroundrb/examples/workers/server_worker.rb
A    vendor/plugins/backgroundrb/examples/workers/rss_worker.rb
A    vendor/plugins/backgroundrb/examples/workers/world_worker.rb
A    vendor/plugins/backgroundrb/examples/workers/error_worker.rb
A    vendor/plugins/backgroundrb/examples/workers/god_worker.rb
A    vendor/plugins/backgroundrb/examples/workers/renewal_worker.rb
A    vendor/plugins/backgroundrb/examples/workers/model_worker.rb
A    vendor/plugins/backgroundrb/examples/workers/foo_worker.rb
Updated external to revision 312.

Updated to revision 339.
$~/Documents/University/Syracuse/Research/Dissertation/fromSVN/ 
Dissertation/genrebrowser
[james at MinistryOfInformation2]$svn commit -m 'added backgroundrb using  
externals'
Sending        genrebrowser/app/models/event.rb
Sending        genrebrowser/vendor/plugins
Transmitting file data .
Committed revision 340.
$~/Documents/University/Syracuse/Research/Dissertation/fromSVN/ 
Dissertation/genrebrowser
[james at MinistryOfInformation2]$svn st
?      tmp
?      test/bdrb_test_helper.rb
?      script/backgroundrb
?      config/backgroundrb.yml
?      lib/workers
X      vendor/plugins/backgroundrb
?      vendor/plugins/acts_as_list

Performing status on external item at 'vendor/plugins/backgroundrb'
$~/Documents/University/Syracuse/Research/Dissertation/fromSVN/ 
Dissertation/genrebrowser
[james at MinistryOfInformation2]$rake backgroundrb:setup
(in /Users/james/Documents/University/Syracuse/Research/Dissertation/ 
fromSVN/Dissertation/genrebrowser)
$~/Documents/University/Syracuse/Research/Dissertation/fromSVN/ 
Dissertation/genrebrowser
[james at MinistryOfInformation2]$script/backgroundrb start
/Library/Ruby/Gems/1.8/gems/activerecord-2.0.2/lib/active_record/ 
base.rb:1532:in `method_missing': undefined method  
`acts_as_simile_timeline_event' for #<Class:0x125bfd8> (NoMethodError)
	from /Users/james/Documents/University/Syracuse/Research/Dissertation/ 
fromSVN/Dissertation/genrebrowser/app/models/event.rb:9
	from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in  
`gem_original_require'
	from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in `require'
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/ 
active_support/dependencies.rb:496:in `require'
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/ 
active_support/dependencies.rb:342:in `new_constants_in'
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/ 
active_support/dependencies.rb:496:in `require'
	from /Users/james/Documents/University/Syracuse/Research/Dissertation/ 
fromSVN/Dissertation/genrebrowser/vendor/plugins/backgroundrb/ 
framework/packet.rb:10
	from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in  
`gem_original_require'
	from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in `require'
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/ 
active_support/dependencies.rb:496:in `require'
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/ 
active_support/dependencies.rb:342:in `new_constants_in'
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/ 
active_support/dependencies.rb:496:in `require'
	from script/backgroundrb:16
$~/Documents/University/Syracuse/Research/Dissertation/fromSVN/ 
Dissertation/genrebrowser

I get the same error using the "acts_as_list" plugin, installed using  
script/plugin.

Thanks,
James

ps. Perhaps the README file for installation via piston should  
indicate that the command is meant to be run from the vendor/plugins  
folder? I initially had a backgroundrb folder in my RAILS_ROOT folder.

From orion at mh2o.com  Tue Jan 22 12:31:59 2008
From: orion at mh2o.com (Orion Delwaterman)
Date: Tue, 22 Jan 2008 12:31:59 -0500
Subject: [Backgroundrb-devel] Query All Worker
Message-ID: <205576B004DDBD438F6560241E0751A0157F913A@sbsexc02.mh2onyc.com>

Hi Hemant,

I just checked out the trunk into my project and ran my test again with
the same results.  I double checked that I have the latest code from the
trunk:

Worker:
class OriontestWorker < BackgrounDRb::MetaWorker
  set_worker_name :oriontest_worker
#  set_no_auto_load true
  
  def create(args = nil)
    # this method is called, when worker is loaded for the first time
    register_status(rand(200))
    res = RAILS_ENV
    log res
    register_result(res)
  end
  
  def get_methods
    self.methods.sort
  end
  
  def test_results
    res = rand(200)
    log res
    register_result(res)
    res
  end
  
  def get_msg_writer
    self.msg_writer.inspect
  end
  
  def get_worker_proxy
    self.class.worker_proxy.inspect
  end
  
end

IRB:
[orion at OrionFoot hydra]$ ruby script/console
Loading development environment.
>> MiddleMan.query_all_workers
=> {:log_worker=>nil, :oriontest_worker=>nil}
>> MiddleMan.send_request :worker => :oriontest_worker, :worker_method
=> :test_results
=> {:data=>69, :type=>:response, :client_signature=>14}
>> MiddleMan.query_all_workers
=> {:log_worker=>nil, :oriontest_worker=>nil}

I've taken a look through the source code and found it somewhat
confusing.  I saw the bug fix in server/master_worker.rb (line 67):

    def query_all_worker_status(p_data)
      dumpable_status = { }
      reactor.live_workers.each { |key,value| dumpable_status[key] =
value.worker_status }
      send_object(dumpable_status)
    end 
   
was changed to

   def query_all_worker_status(p_data)
      dumpable_status = { }
      reactor.live_workers.each { |key,value| dumpable_status[key] =
reactor.result_hash[key] }
      send_object(dumpable_status)
    end


However if you take a look at framework/meta_pimp.rb (line 37) you can
see that the results aren't saved on the reactor:

  def save_worker_result(data_options = { })
    @worker_result = data_options[:data]
  end

and if you look just below that on line 41 you see:

  def save_worker_status(data_options = { })
    # @worker_status = data_options[:data]
    reactor.update_result(worker_key,data_options[:data])
  end

It's confusing to see the save_worker_status() use the reactor's result
object to store its status, and also have a method save_worker_result().

This is where I get really confused about the method
'query_all_workers()'?  Previous you said that '[query_all_workers()]
returns status/result objects for all the available workers.'  This is a
little confusing because the workers have two methods: register_status()
(which I presume is supposed to record what the worker is currently
doing e.g. idle, processing, completed, etc.), and register_results
(which I presume is supposed to hold the results of the worker).

Is the results object of the workers a holdover from the 0.2.X versions?
Is the results object and the status object supposed to be the same? Or
are they different and the nomenclature is just a bit confusing, and
query_all_workers() is supposed to return the status of the worker?

Also I just wanted to say, I know how hard it can be to take over an
opensource project.  I applaud you for what you are doing, I am just
trying to clarify the situation.  I am also happy to try and resolve
this bug and give you a diff, but I'd like to know what you want to do
with results vs. status thing I mentioned above.

Orion


-----Original Message-----
From: hemant [mailto:gethemant at gmail.com] 
Sent: Tuesday, January 22, 2008 4:47 AM
To: Orion Delwaterman
Cc: backgroundrb-devel at rubyforge.org
Subject: Re: [Backgroundrb-devel] Query All Worker

Hi Orion,

There was a bug with query_all_workers, I fixed it. So you can sync
with latest source code from trunk and get it working.

On Jan 21, 2008 9:33 PM, Orion Delwaterman <orion at mh2o.com> wrote:
> Whoops mixup in my code here, I fixed it but I am still getting the
same
> issue where either register_result does not properly log the result or
> query_all_workers() is not returning the result:
>
> CODE:
>
> class OriontestWorker < BackgrounDRb::MetaWorker
>   set_worker_name :oriontest_worker
> #  set_no_auto_load true
>
>   def create(args = nil)
>     # this method is called, when worker is loaded for the first time
>     register_status(rand(200))
>     res = RAILS_ENV
>     log res
>     register_result(res)
>   end
>
>   def get_methods
>     self.methods.sort
>   end
>
>   def test_results
>     res = rand(200)
>     log res
>     register_result(res)
>     res
>   end
>
>   def get_msg_writer
>     self.msg_writer.inspect
>   end
>
> end
>
> IRB:
>
> [orion at OrionFoot hydra]$ ruby script/console
> Loading development environment.
> >> MiddleMan.query_all_workers
> => {:log_worker=>nil, :oriontest_worker=>nil}
> >> MiddleMan.send_request :worker => :oriontest_worker, :worker_method
> => :test_results
> => {:data=>73, :client_signature=>12, :type=>:response}
> >> MiddleMan.query_all_workers
> => {:log_worker=>nil, :oriontest_worker=>nil}
> >>
>
> -----Original Message-----
> From: Orion Delwaterman
> Sent: Monday, January 21, 2008 10:12 AM
> To: 'hemant'
> Cc: backgroundrb-devel at rubyforge.org
>
> Subject: RE: [Backgroundrb-devel] Query All Worker
>
> Maybe I am doing something wrong but query_all_workers() does not
appear
> to return any results from my workers.  Does the results object have
to
> be some kind of hash or is this just some weird bug?
>
> Worker:
>
> class OriontestWorker < BackgrounDRb::MetaWorker
>   set_worker_name :oriontest_worker
> #  set_no_auto_load true
>
>   def create(args = nil)
>     # this method is called, when worker is loaded for the first time
>   end
>
>   def get_random
>     register_status(rand(200))
>     register_result(RAILS_ENV)
>     true
>   end
>
> end
>
> IRB testing:
> Loading development environment.
> >> MiddleMan.query_all_workers
> => {:log_worker=>nil, :oriontest_worker=>nil}
>
> -----Original Message-----
> From: hemant [mailto:gethemant at gmail.com]
> Sent: Saturday, January 19, 2008 6:58 AM
> To: Orion Delwaterman
> Cc: backgroundrb-devel at rubyforge.org
> Subject: Re: [Backgroundrb-devel] Query All Worker
>
> On Jan 19, 2008 4:42 AM, Orion Delwaterman <orion at mh2o.com> wrote:
> >
> >
> >
> >
> > What does the method query_all_workers() do?  There is no
> documentation on
> > what this method returns (1.0.1 version).
>
> It returns status/result objects for all the available workers.
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gobigdave at gmail.com  Tue Jan 22 16:43:37 2008
From: gobigdave at gmail.com (Dave Dupre)
Date: Tue, 22 Jan 2008 16:43:37 -0500
Subject: [Backgroundrb-devel] Threadpool and queuing of tasks
Message-ID: <258cc3f50801221343kb68d84as2d393a3bd805c2c9@mail.gmail.com>

I recently switched over to v1.0, and things are rolling along pretty well.
However, one thing that has always been a little confusing to me is knowing
when to use thread_pool.  Since most of my bgrb workers are called from my
web app to process rather than being scheduled, I'm using the thread_pool
for every call.  Unfortunately, that means that I have to split up workers
by how many threads I can have.  It would be great if one worker could
partition a single thread pool among the methods.  I want to avoid too many
workers to keep the process count down.

I'm now working on a new scheme that pushes this example.  Basically, I have
some long running, saved searches that are triggered by various events
throughout the site.  All I want my site to do is update a status that the
job is queued and have it picked up from there.  Here is where I run into
trouble, possibly because I've built too many systems like this that use
real queuing packages.  Here is what I want:

Dispatch method (usually one thread is necessary):
1. Find the oldest 'queued' record (make sure to find with :lock => true)
1a. If none, goto step 5
2. Update status to 'processing'
3. Send to search method
4. Repeat 1
5. Done

Search method (many threads):
1. Perform the search
2. Update status to 'complete'
3. Done

The easy answer is to split these into two workers.  Set the pool_size of
Dispatch to 1, and Search to 5 or 10.  However, eating two processes (master
and worker) for something so simple as Dispatch seems like serious overkill
to me.  Since I currently run on one server, the extra processes cut into
the memory the main site wants.

A related question is how to implement Dispatch without polling.  Call me
anal, but I feel dirty whenever I using polling, especially something that I
want to be picked up immediately.  Is there a way I can trigger it to run if
it isn't already?  The old bgrb had a singleton that let me do something
like that.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080122/6f2e1e03/attachment.html 

From gethemant at gmail.com  Tue Jan 22 17:37:12 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 23 Jan 2008 04:07:12 +0530
Subject: [Backgroundrb-devel] Query All Worker
In-Reply-To: <205576B004DDBD438F6560241E0751A0157F913A@sbsexc02.mh2onyc.com>
References: <205576B004DDBD438F6560241E0751A0157F913A@sbsexc02.mh2onyc.com>
Message-ID: <fb283d5e0801221437u38136861rb904653fabba1bb2@mail.gmail.com>

Hi Orion,

In trunk or new version of bdrb there is no register_result and hence
you are getting nil objects when quering the workers using
query_all_workers. I know perhaps its a bit confusing, but in new
version status and result objects are the same,

Also worker status is stored in reactor object, which is generally
updated through meta_pimp. I am glad that you poked through the code.

I know nomenclature is a bit confusing, but just accept the fact that
in new version, there is NO register_result method and status objects
have replaced result objects.

On Jan 22, 2008 11:01 PM, Orion Delwaterman <orion at mh2o.com> wrote:
> Hi Hemant,
>
> I just checked out the trunk into my project and ran my test again with
> the same results.  I double checked that I have the latest code from the
> trunk:
>
> Worker:
> class OriontestWorker < BackgrounDRb::MetaWorker
>  set_worker_name :oriontest_worker
> #  set_no_auto_load true
>
>  def create(args = nil)
>    # this method is called, when worker is loaded for the first time
>    register_status(rand(200))
>    res = RAILS_ENV
>    log res
>    register_result(res)
>  end
>
>  def get_methods
>    self.methods.sort
>  end
>
>  def test_results
>    res = rand(200)
>    log res
>    register_result(res)
>    res
>  end
>
>  def get_msg_writer
>    self.msg_writer.inspect
>  end
>
>  def get_worker_proxy
>    self.class.worker_proxy.inspect
>  end
>
> end
>
> IRB:
> [orion at OrionFoot hydra]$ ruby script/console
> Loading development environment.
> >> MiddleMan.query_all_workers
> => {:log_worker=>nil, :oriontest_worker=>nil}
> >> MiddleMan.send_request :worker => :oriontest_worker, :worker_method
> => :test_results
> => {:data=>69, :type=>:response, :client_signature=>14}
> >> MiddleMan.query_all_workers
> => {:log_worker=>nil, :oriontest_worker=>nil}

Yeah, so change register_result to register_status and everything will be fine.

>
> I've taken a look through the source code and found it somewhat
> confusing.  I saw the bug fix in server/master_worker.rb (line 67):
>
>    def query_all_worker_status(p_data)
>      dumpable_status = { }
>      reactor.live_workers.each { |key,value| dumpable_status[key] =
> value.worker_status }
>      send_object(dumpable_status)
>    end
>
> was changed to
>
>   def query_all_worker_status(p_data)
>      dumpable_status = { }
>      reactor.live_workers.each { |key,value| dumpable_status[key] =
> reactor.result_hash[key] }
>      send_object(dumpable_status)
>    end
>
>
> However if you take a look at framework/meta_pimp.rb (line 37) you can
> see that the results aren't saved on the reactor:
>
>  def save_worker_result(data_options = { })
>    @worker_result = data_options[:data]
>  end
>
> and if you look just below that on line 41 you see:
>
>  def save_worker_status(data_options = { })
>    # @worker_status = data_options[:data]
>    reactor.update_result(worker_key,data_options[:data])
>  end
>
> It's confusing to see the save_worker_status() use the reactor's result
> object to store its status, and also have a method save_worker_result().
>
Hmm, may be it needs a cleanup. I will think it through.

> This is where I get really confused about the method
> 'query_all_workers()'?  Previous you said that '[query_all_workers()]
> returns status/result objects for all the available workers.'  This is a
> little confusing because the workers have two methods: register_status()
> (which I presume is supposed to record what the worker is currently
> doing e.g. idle, processing, completed, etc.), and register_results
> (which I presume is supposed to hold the results of the worker).

Again, new version result and status are the same.
>
> Is the results object of the workers a holdover from the 0.2.X versions?
> Is the results object and the status object supposed to be the same? Or
> are they different and the nomenclature is just a bit confusing, and
> query_all_workers() is supposed to return the status of the worker?

query_all_workers() is supposed to return status objects of all the
workers that has been registered through register_status.
>
> Also I just wanted to say, I know how hard it can be to take over an
> opensource project.  I applaud you for what you are doing, I am just
> trying to clarify the situation.  I am also happy to try and resolve
> this bug and give you a diff, but I'd like to know what you want to do
> with results vs. status thing I mentioned above.

You are welcome.
>
> Orion
>
>
> -----Original Message-----
> From: hemant [mailto:gethemant at gmail.com]
>
> Sent: Tuesday, January 22, 2008 4:47 AM
> To: Orion Delwaterman
> Cc: backgroundrb-devel at rubyforge.org
> Subject: Re: [Backgroundrb-devel] Query All Worker
>
> Hi Orion,
>
> There was a bug with query_all_workers, I fixed it. So you can sync
> with latest source code from trunk and get it working.
>
> On Jan 21, 2008 9:33 PM, Orion Delwaterman <orion at mh2o.com> wrote:
> > Whoops mixup in my code here, I fixed it but I am still getting the
> same
> > issue where either register_result does not properly log the result or
> > query_all_workers() is not returning the result:
> >
> > CODE:
> >
> > class OriontestWorker < BackgrounDRb::MetaWorker
> >   set_worker_name :oriontest_worker
> > #  set_no_auto_load true
> >
> >   def create(args = nil)
> >     # this method is called, when worker is loaded for the first time
> >     register_status(rand(200))
> >     res = RAILS_ENV
> >     log res
> >     register_result(res)
> >   end
> >
> >   def get_methods
> >     self.methods.sort
> >   end
> >
> >   def test_results
> >     res = rand(200)
> >     log res
> >     register_result(res)
> >     res
> >   end
> >
> >   def get_msg_writer
> >     self.msg_writer.inspect
> >   end
> >
> > end
> >
> > IRB:
> >
> > [orion at OrionFoot hydra]$ ruby script/console
> > Loading development environment.
> > >> MiddleMan.query_all_workers
> > => {:log_worker=>nil, :oriontest_worker=>nil}
> > >> MiddleMan.send_request :worker => :oriontest_worker, :worker_method
> > => :test_results
> > => {:data=>73, :client_signature=>12, :type=>:response}
> > >> MiddleMan.query_all_workers
> > => {:log_worker=>nil, :oriontest_worker=>nil}
> > >>
> >
> > -----Original Message-----
> > From: Orion Delwaterman
> > Sent: Monday, January 21, 2008 10:12 AM
> > To: 'hemant'
> > Cc: backgroundrb-devel at rubyforge.org
> >
> > Subject: RE: [Backgroundrb-devel] Query All Worker
> >
> > Maybe I am doing something wrong but query_all_workers() does not
> appear
> > to return any results from my workers.  Does the results object have
> to
> > be some kind of hash or is this just some weird bug?
> >
> > Worker:
> >
> > class OriontestWorker < BackgrounDRb::MetaWorker
> >   set_worker_name :oriontest_worker
> > #  set_no_auto_load true
> >
> >   def create(args = nil)
> >     # this method is called, when worker is loaded for the first time
> >   end
> >
> >   def get_random
> >     register_status(rand(200))
> >     register_result(RAILS_ENV)
> >     true
> >   end
> >
> > end
> >
> > IRB testing:
> > Loading development environment.
> > >> MiddleMan.query_all_workers
> > => {:log_worker=>nil, :oriontest_worker=>nil}
> >
> > -----Original Message-----
> > From: hemant [mailto:gethemant at gmail.com]
> > Sent: Saturday, January 19, 2008 6:58 AM
> > To: Orion Delwaterman
> > Cc: backgroundrb-devel at rubyforge.org
> > Subject: Re: [Backgroundrb-devel] Query All Worker
> >
> > On Jan 19, 2008 4:42 AM, Orion Delwaterman <orion at mh2o.com> wrote:
> > >
> > >
> > >
> > >
> > > What does the method query_all_workers() do?  There is no
> > documentation on
> > > what this method returns (1.0.1 version).
> >
> > It returns status/result objects for all the available workers.
> >
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From orion at mh2o.com  Tue Jan 22 17:48:07 2008
From: orion at mh2o.com (Orion Delwaterman)
Date: Tue, 22 Jan 2008 17:48:07 -0500
Subject: [Backgroundrb-devel] Query All Worker
Message-ID: <205576B004DDBD438F6560241E0751A0157F913B@sbsexc02.mh2onyc.com>

Hi Hemant,

I am fine with removing the idea of results from BackgrounDRb.  I'll
adjust my code to work with that.

I just updated my BackgrounDRb but I the method register_result() is
still in the trunk (repository:
http://svn.devjavu.com/backgroundrb/trunk at revision 312). Do I have
the wrong repository or is something else going on with the subversion
server?

Orion

-----Original Message-----
From: hemant [mailto:gethemant at gmail.com] 
Sent: Tuesday, January 22, 2008 5:37 PM
To: Orion Delwaterman
Cc: backgroundrb-devel at rubyforge.org
Subject: Re: [Backgroundrb-devel] Query All Worker

Hi Orion,

In trunk or new version of bdrb there is no register_result and hence
you are getting nil objects when quering the workers using
query_all_workers. I know perhaps its a bit confusing, but in new
version status and result objects are the same,

Also worker status is stored in reactor object, which is generally
updated through meta_pimp. I am glad that you poked through the code.

I know nomenclature is a bit confusing, but just accept the fact that
in new version, there is NO register_result method and status objects
have replaced result objects.

On Jan 22, 2008 11:01 PM, Orion Delwaterman <orion at mh2o.com> wrote:
> Hi Hemant,
>
> I just checked out the trunk into my project and ran my test again
with
> the same results.  I double checked that I have the latest code from
the
> trunk:
>
> Worker:
> class OriontestWorker < BackgrounDRb::MetaWorker
>  set_worker_name :oriontest_worker
> #  set_no_auto_load true
>
>  def create(args = nil)
>    # this method is called, when worker is loaded for the first time
>    register_status(rand(200))
>    res = RAILS_ENV
>    log res
>    register_result(res)
>  end
>
>  def get_methods
>    self.methods.sort
>  end
>
>  def test_results
>    res = rand(200)
>    log res
>    register_result(res)
>    res
>  end
>
>  def get_msg_writer
>    self.msg_writer.inspect
>  end
>
>  def get_worker_proxy
>    self.class.worker_proxy.inspect
>  end
>
> end
>
> IRB:
> [orion at OrionFoot hydra]$ ruby script/console
> Loading development environment.
> >> MiddleMan.query_all_workers
> => {:log_worker=>nil, :oriontest_worker=>nil}
> >> MiddleMan.send_request :worker => :oriontest_worker, :worker_method
> => :test_results
> => {:data=>69, :type=>:response, :client_signature=>14}
> >> MiddleMan.query_all_workers
> => {:log_worker=>nil, :oriontest_worker=>nil}

Yeah, so change register_result to register_status and everything will
be fine.

>
> I've taken a look through the source code and found it somewhat
> confusing.  I saw the bug fix in server/master_worker.rb (line 67):
>
>    def query_all_worker_status(p_data)
>      dumpable_status = { }
>      reactor.live_workers.each { |key,value| dumpable_status[key] =
> value.worker_status }
>      send_object(dumpable_status)
>    end
>
> was changed to
>
>   def query_all_worker_status(p_data)
>      dumpable_status = { }
>      reactor.live_workers.each { |key,value| dumpable_status[key] =
> reactor.result_hash[key] }
>      send_object(dumpable_status)
>    end
>
>
> However if you take a look at framework/meta_pimp.rb (line 37) you can
> see that the results aren't saved on the reactor:
>
>  def save_worker_result(data_options = { })
>    @worker_result = data_options[:data]
>  end
>
> and if you look just below that on line 41 you see:
>
>  def save_worker_status(data_options = { })
>    # @worker_status = data_options[:data]
>    reactor.update_result(worker_key,data_options[:data])
>  end
>
> It's confusing to see the save_worker_status() use the reactor's
result
> object to store its status, and also have a method
save_worker_result().
>
Hmm, may be it needs a cleanup. I will think it through.

> This is where I get really confused about the method
> 'query_all_workers()'?  Previous you said that '[query_all_workers()]
> returns status/result objects for all the available workers.'  This is
a
> little confusing because the workers have two methods:
register_status()
> (which I presume is supposed to record what the worker is currently
> doing e.g. idle, processing, completed, etc.), and register_results
> (which I presume is supposed to hold the results of the worker).

Again, new version result and status are the same.
>
> Is the results object of the workers a holdover from the 0.2.X
versions?
> Is the results object and the status object supposed to be the same?
Or
> are they different and the nomenclature is just a bit confusing, and
> query_all_workers() is supposed to return the status of the worker?

query_all_workers() is supposed to return status objects of all the
workers that has been registered through register_status.
>
> Also I just wanted to say, I know how hard it can be to take over an
> opensource project.  I applaud you for what you are doing, I am just
> trying to clarify the situation.  I am also happy to try and resolve
> this bug and give you a diff, but I'd like to know what you want to do
> with results vs. status thing I mentioned above.

You are welcome.
>
> Orion
>
>
> -----Original Message-----
> From: hemant [mailto:gethemant at gmail.com]
>
> Sent: Tuesday, January 22, 2008 4:47 AM
> To: Orion Delwaterman
> Cc: backgroundrb-devel at rubyforge.org
> Subject: Re: [Backgroundrb-devel] Query All Worker
>
> Hi Orion,
>
> There was a bug with query_all_workers, I fixed it. So you can sync
> with latest source code from trunk and get it working.
>
> On Jan 21, 2008 9:33 PM, Orion Delwaterman <orion at mh2o.com> wrote:
> > Whoops mixup in my code here, I fixed it but I am still getting the
> same
> > issue where either register_result does not properly log the result
or
> > query_all_workers() is not returning the result:
> >
> > CODE:
> >
> > class OriontestWorker < BackgrounDRb::MetaWorker
> >   set_worker_name :oriontest_worker
> > #  set_no_auto_load true
> >
> >   def create(args = nil)
> >     # this method is called, when worker is loaded for the first
time
> >     register_status(rand(200))
> >     res = RAILS_ENV
> >     log res
> >     register_result(res)
> >   end
> >
> >   def get_methods
> >     self.methods.sort
> >   end
> >
> >   def test_results
> >     res = rand(200)
> >     log res
> >     register_result(res)
> >     res
> >   end
> >
> >   def get_msg_writer
> >     self.msg_writer.inspect
> >   end
> >
> > end
> >
> > IRB:
> >
> > [orion at OrionFoot hydra]$ ruby script/console
> > Loading development environment.
> > >> MiddleMan.query_all_workers
> > => {:log_worker=>nil, :oriontest_worker=>nil}
> > >> MiddleMan.send_request :worker => :oriontest_worker,
:worker_method
> > => :test_results
> > => {:data=>73, :client_signature=>12, :type=>:response}
> > >> MiddleMan.query_all_workers
> > => {:log_worker=>nil, :oriontest_worker=>nil}
> > >>
> >
> > -----Original Message-----
> > From: Orion Delwaterman
> > Sent: Monday, January 21, 2008 10:12 AM
> > To: 'hemant'
> > Cc: backgroundrb-devel at rubyforge.org
> >
> > Subject: RE: [Backgroundrb-devel] Query All Worker
> >
> > Maybe I am doing something wrong but query_all_workers() does not
> appear
> > to return any results from my workers.  Does the results object have
> to
> > be some kind of hash or is this just some weird bug?
> >
> > Worker:
> >
> > class OriontestWorker < BackgrounDRb::MetaWorker
> >   set_worker_name :oriontest_worker
> > #  set_no_auto_load true
> >
> >   def create(args = nil)
> >     # this method is called, when worker is loaded for the first
time
> >   end
> >
> >   def get_random
> >     register_status(rand(200))
> >     register_result(RAILS_ENV)
> >     true
> >   end
> >
> > end
> >
> > IRB testing:
> > Loading development environment.
> > >> MiddleMan.query_all_workers
> > => {:log_worker=>nil, :oriontest_worker=>nil}
> >
> > -----Original Message-----
> > From: hemant [mailto:gethemant at gmail.com]
> > Sent: Saturday, January 19, 2008 6:58 AM
> > To: Orion Delwaterman
> > Cc: backgroundrb-devel at rubyforge.org
> > Subject: Re: [Backgroundrb-devel] Query All Worker
> >
> > On Jan 19, 2008 4:42 AM, Orion Delwaterman <orion at mh2o.com> wrote:
> > >
> > >
> > >
> > >
> > > What does the method query_all_workers() do?  There is no
> > documentation on
> > > what this method returns (1.0.1 version).
> >
> > It returns status/result objects for all the available workers.
> >
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Tue Jan 22 18:00:19 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 23 Jan 2008 04:30:19 +0530
Subject: [Backgroundrb-devel] Query All Worker
In-Reply-To: <205576B004DDBD438F6560241E0751A0157F913B@sbsexc02.mh2onyc.com>
References: <205576B004DDBD438F6560241E0751A0157F913B@sbsexc02.mh2onyc.com>
Message-ID: <fb283d5e0801221500q631f6423j2a2b8702a21f31f9@mail.gmail.com>

Hi,

On Jan 23, 2008 4:18 AM, Orion Delwaterman <orion at mh2o.com> wrote:
> Hi Hemant,
>
> I am fine with removing the idea of results from BackgrounDRb.  I'll
> adjust my code to work with that.
>
> I just updated my BackgrounDRb but I the method register_result() is
> still in the trunk (repository:
> http://svn.devjavu.com/backgroundrb/trunk at revision 312). Do I have
> the wrong repository or is something else going on with the subversion
> server?

Yeah, thats a little sad, its there but its not functional. In fact,
its not needed, because status objects do the same job.


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Tue Jan 22 18:27:55 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 23 Jan 2008 04:57:55 +0530
Subject: [Backgroundrb-devel] Unable to load Models which reference
	plugins?
In-Reply-To: <B46B4352-14CF-4B12-AC02-ACCD13FA678A@howison.name>
References: <F87ADCB4-112D-4DF3-BE81-E7418F6B8F76@howison.name>
	<fb283d5e0801220103i7fbea170hbf413edcccff3fec@mail.gmail.com>
	<B46B4352-14CF-4B12-AC02-ACCD13FA678A@howison.name>
Message-ID: <fb283d5e0801221527r561df349tcd3ffb885b9164d5@mail.gmail.com>

Hi,

On Jan 22, 2008 8:44 PM, James Howison <james at howison.name> wrote:
>
>
> On Jan 22, 2008, at 4:03 AM, hemant wrote:
>
> > Hi,
> >
> > On Jan 22, 2008 9:27 AM, James Howison <james at howison.name> wrote:
> >> I'm using a plugin which adds a method to ActiveRecord::Base
> >>
> >> http://similetimelinerailshelper.googlecode.com/svn/trunk/simile_timeline/
> >>
> >> This adds an "acts_as" type declaration to the Models, via the file
> >> in
> >> vendor/plugins/simile_timeline/lib/simile_timeline.rb Rails is
> >> working
> >> with these declarations, and they are being used successfully in
> >> 'regular Rails'.
> >>
> >> When I run script/backgroundrb start, I get an error saying that
> >> method is not found (error follows sig).  Backgroundrb is able to
> >> start up with those acts_as declarations commented out.
> >>
> >> Presumably I need to require the plugin file somewhere, but I
> >> wondered
> >> if backgroundrb might be meant to load the same things as Rails, by
> >> default.
> >>
> >> Thanks,
> >> James
> >>
> >> [james at MinistryOfInformation2]$script/backgroundrb start
> >> /Library/Ruby/Gems/1.8/gems/activerecord-2.0.2/lib/active_record/
> >> base.rb:1532:in `method_missing': undefined method
> >> `acts_as_simile_timeline_event' for #<Class:0x11655fc>
> >> (NoMethodError)
> >>        from /Users/james/Documents/University/Syracuse/Research/
> >> Dissertation/
> >> fromSVN/Dissertation/genrebrowser/app/models/event.rb:9
> >>        from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in
> >> `gem_original_require'
> >>        from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in
> >> `require'
> >>        from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> >> active_support/dependencies.rb:496:in `require'
> >>        from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> >> active_support/dependencies.rb:342:in `new_constants_in'
> >>        from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> >> active_support/dependencies.rb:496:in `require'
> >>        from /Users/james/Documents/University/Syracuse/Research/
> >> Dissertation/
> >> fromSVN/Dissertation/genrebrowser/vendor/plugins/backgroundrb/
> >> framework/packet.rb:10
> >>        from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in
> >> `gem_original_require'
> >>        from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in
> >> `require'
> >>        from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> >> active_support/dependencies.rb:496:in `require'
> >>        from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> >> active_support/dependencies.rb:342:in `new_constants_in'
> >>        from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> >> active_support/dependencies.rb:496:in `require'
> >>        from script/backgroundrb:16
> >
> > Which version of BackgrounDRb you are running? Use either trunk or
> > 1.0.1.
> > I remember version 1.0 has aggresive loading of models and hence you
> > will get errors, if your model requires other plugins.
> > But moving to 1.0.1 and trunk, I removed aggresive loading and model
> > environment is loaded same as rails loads them.
> >
> > Tell me the version, I will have a look at issue.
>
> Thanks.  Initially I had used piston on the trunk, but now I've used
> svn:externals as in the README file:
>
> [james at MinistryOfInformation2]$svn propedit svn:externals vendor/plugins
>
> backgroundrb http://svn.devjavu.com/backgroundrb/trunk
> <snip>
> ~
> ~
> "svn-prop.tmp" 1L, 55C written
> Set new value for property 'svn:externals' on 'vendor/plugins'
> $~/Documents/University/Syracuse/Research/Dissertation/fromSVN/
> Dissertation/genrebrowser
> [james at MinistryOfInformation2]$svn up vendor/plugins
>
> Fetching external item into 'vendor/plugins/backgroundrb'
> A    vendor/plugins/backgroundrb/test
> A    vendor/plugins/backgroundrb/test/meta_worker_test.rb
> A    vendor/plugins/backgroundrb/test/backgroundrb_test.rb
> A    vendor/plugins/backgroundrb/test/master_worker_test.rb
> A    vendor/plugins/backgroundrb/test/bdrb_test_helper.rb
> A    vendor/plugins/backgroundrb/cron_trigger.patch
> A    vendor/plugins/backgroundrb/Rakefile
> A    vendor/plugins/backgroundrb/LICENSE
> A    vendor/plugins/backgroundrb/tasks
> A    vendor/plugins/backgroundrb/tasks/backgroundrb_tasks.rake
> A    vendor/plugins/backgroundrb/TODO.org
> A    vendor/plugins/backgroundrb/framework
> A    vendor/plugins/backgroundrb/framework/packet_guid.rb
> A    vendor/plugins/backgroundrb/framework/cpu_worker.rb
> A    vendor/plugins/backgroundrb/framework/event.rb
> A    vendor/plugins/backgroundrb/framework/packet_master.rb
> A    vendor/plugins/backgroundrb/framework/bin_parser.rb
> A    vendor/plugins/backgroundrb/framework/nbio.rb
> A    vendor/plugins/backgroundrb/framework/class_helpers.rb
> A    vendor/plugins/backgroundrb/framework/core.rb
> A    vendor/plugins/backgroundrb/framework/buftok.rb
> A    vendor/plugins/backgroundrb/framework/deferrable.rb
> A    vendor/plugins/backgroundrb/framework/callback.rb
> A    vendor/plugins/backgroundrb/framework/double_keyed_hash.rb
> A    vendor/plugins/backgroundrb/framework/connection.rb
> A    vendor/plugins/backgroundrb/framework/periodic_event.rb
> A    vendor/plugins/backgroundrb/framework/io_worker.rb
> A    vendor/plugins/backgroundrb/framework/disconnect_error.rb
> A    vendor/plugins/backgroundrb/framework/pimp.rb
> A    vendor/plugins/backgroundrb/framework/packet.rb
> A    vendor/plugins/backgroundrb/framework/worker.rb
> A    vendor/plugins/backgroundrb/framework/meta_pimp.rb
> A    vendor/plugins/backgroundrb/README
> A    vendor/plugins/backgroundrb/script
> A    vendor/plugins/backgroundrb/script/backgroundrb
> A    vendor/plugins/backgroundrb/script/bdrb_test_helper.rb
> A    vendor/plugins/backgroundrb/init.rb
> A    vendor/plugins/backgroundrb/config
> A    vendor/plugins/backgroundrb/config/backgroundrb.yml
> A    vendor/plugins/backgroundrb/server
> A    vendor/plugins/backgroundrb/server/meta_worker.rb
> A    vendor/plugins/backgroundrb/server/trigger.rb
> A    vendor/plugins/backgroundrb/server/cron_trigger.rb
> A    vendor/plugins/backgroundrb/server/invalid_dump_error.rb
> A    vendor/plugins/backgroundrb/server/master_worker.rb
> A    vendor/plugins/backgroundrb/server/log_worker.rb
> A    vendor/plugins/backgroundrb/lib
> A    vendor/plugins/backgroundrb/lib/bdrb_conn_error.rb
> A    vendor/plugins/backgroundrb/lib/backgroundrb.rb
> A    vendor/plugins/backgroundrb/install.rb
> A    vendor/plugins/backgroundrb/generators
> A    vendor/plugins/backgroundrb/generators/worker
> A    vendor/plugins/backgroundrb/generators/worker/worker_generator.rb
> A    vendor/plugins/backgroundrb/generators/worker/USAGE
> A    vendor/plugins/backgroundrb/generators/worker/templates
> A    vendor/plugins/backgroundrb/generators/worker/templates/
> unit_test.rb
> A    vendor/plugins/backgroundrb/generators/worker/templates/worker.rb
> A    vendor/plugins/backgroundrb/MIT-LICENSE
> A    vendor/plugins/backgroundrb/uninstall.rb
> A    vendor/plugins/backgroundrb/examples
> A    vendor/plugins/backgroundrb/examples/god_worker.rb
> A    vendor/plugins/backgroundrb/examples/foo_controller.rb
> A    vendor/plugins/backgroundrb/examples/worker_tests
> A    vendor/plugins/backgroundrb/examples/worker_tests/
> god_worker_test.rb
> A    vendor/plugins/backgroundrb/examples/workers
> A    vendor/plugins/backgroundrb/examples/workers/xmpp_worker.rb
> A    vendor/plugins/backgroundrb/examples/workers/server_worker.rb
> A    vendor/plugins/backgroundrb/examples/workers/rss_worker.rb
> A    vendor/plugins/backgroundrb/examples/workers/world_worker.rb
> A    vendor/plugins/backgroundrb/examples/workers/error_worker.rb
> A    vendor/plugins/backgroundrb/examples/workers/god_worker.rb
> A    vendor/plugins/backgroundrb/examples/workers/renewal_worker.rb
> A    vendor/plugins/backgroundrb/examples/workers/model_worker.rb
> A    vendor/plugins/backgroundrb/examples/workers/foo_worker.rb
> Updated external to revision 312.
>
> Updated to revision 339.
> $~/Documents/University/Syracuse/Research/Dissertation/fromSVN/
> Dissertation/genrebrowser
> [james at MinistryOfInformation2]$svn commit -m 'added backgroundrb using
> externals'
> Sending        genrebrowser/app/models/event.rb
> Sending        genrebrowser/vendor/plugins
> Transmitting file data .
> Committed revision 340.
> $~/Documents/University/Syracuse/Research/Dissertation/fromSVN/
> Dissertation/genrebrowser
> [james at MinistryOfInformation2]$svn st
> ?      tmp
> ?      test/bdrb_test_helper.rb
> ?      script/backgroundrb
> ?      config/backgroundrb.yml
> ?      lib/workers
> X      vendor/plugins/backgroundrb
> ?      vendor/plugins/acts_as_list
>
> Performing status on external item at 'vendor/plugins/backgroundrb'
> $~/Documents/University/Syracuse/Research/Dissertation/fromSVN/
> Dissertation/genrebrowser
> [james at MinistryOfInformation2]$rake backgroundrb:setup
> (in /Users/james/Documents/University/Syracuse/Research/Dissertation/
> fromSVN/Dissertation/genrebrowser)
> $~/Documents/University/Syracuse/Research/Dissertation/fromSVN/
> Dissertation/genrebrowser
> [james at MinistryOfInformation2]$script/backgroundrb start
> /Library/Ruby/Gems/1.8/gems/activerecord-2.0.2/lib/active_record/
> base.rb:1532:in `method_missing': undefined method
> `acts_as_simile_timeline_event' for #<Class:0x125bfd8> (NoMethodError)
>         from /Users/james/Documents/University/Syracuse/Research/Dissertation/
> fromSVN/Dissertation/genrebrowser/app/models/event.rb:9
>         from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in
> `gem_original_require'
>         from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in `require'
>         from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> active_support/dependencies.rb:496:in `require'
>         from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> active_support/dependencies.rb:342:in `new_constants_in'
>         from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> active_support/dependencies.rb:496:in `require'
>         from /Users/james/Documents/University/Syracuse/Research/Dissertation/
> fromSVN/Dissertation/genrebrowser/vendor/plugins/backgroundrb/
> framework/packet.rb:10
>         from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in
> `gem_original_require'
>         from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:27:in `require'
>         from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> active_support/dependencies.rb:496:in `require'
>         from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> active_support/dependencies.rb:342:in `new_constants_in'
>         from /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/
> active_support/dependencies.rb:496:in `require'
>         from script/backgroundrb:16
> $~/Documents/University/Syracuse/Research/Dissertation/fromSVN/
> Dissertation/genrebrowser
>
> I get the same error using the "acts_as_list" plugin, installed using
> script/plugin.
>


Looks like you are having a model called event and backgroundrb has a
class called Event and hence when bdrb is trying to load event class
of its own, its loading your model and its failing.

Sadly to overcome this, you will have to perhaps rename your model.

From yem_backgroundrb-devel at filter.yve.net  Tue Jan 22 18:51:23 2008
From: yem_backgroundrb-devel at filter.yve.net (Yves-Eric Martin)
Date: Wed, 23 Jan 2008 08:51:23 +0900
Subject: [Backgroundrb-devel] Unable to load Models which
	reference	plugins?
In-Reply-To: <fb283d5e0801221527r561df349tcd3ffb885b9164d5@mail.gmail.com>
References: <F87ADCB4-112D-4DF3-BE81-E7418F6B8F76@howison.name>	<fb283d5e0801220103i7fbea170hbf413edcccff3fec@mail.gmail.com>	<B46B4352-14CF-4B12-AC02-ACCD13FA678A@howison.name>
	<fb283d5e0801221527r561df349tcd3ffb885b9164d5@mail.gmail.com>
Message-ID: <4796817B.9000606@filter.yve.net>

hemant wrote:
> Looks like you are having a model called event and backgroundrb has a
> class called Event and hence when bdrb is trying to load event class
> of its own, its loading your model and its failing.
>
> Sadly to overcome this, you will have to perhaps rename your model
Could changing "Event" to "Packet::Event" help bdrb load the correct class?

-- 
Yves-Eric

From james at howison.name  Tue Jan 22 20:21:15 2008
From: james at howison.name (James Howison)
Date: Tue, 22 Jan 2008 20:21:15 -0500
Subject: [Backgroundrb-devel] Unable to load Models which
	reference	plugins?
In-Reply-To: <4796817B.9000606@filter.yve.net>
References: <F87ADCB4-112D-4DF3-BE81-E7418F6B8F76@howison.name>	<fb283d5e0801220103i7fbea170hbf413edcccff3fec@mail.gmail.com>	<B46B4352-14CF-4B12-AC02-ACCD13FA678A@howison.name>
	<fb283d5e0801221527r561df349tcd3ffb885b9164d5@mail.gmail.com>
	<4796817B.9000606@filter.yve.net>
Message-ID: <61C98FF0-D04B-4439-9665-9B6FF56EDE48@howison.name>


On Jan 22, 2008, at 6:51 PM, Yves-Eric Martin wrote:

> hemant wrote:
>> Looks like you are having a model called event and backgroundrb has a
>> class called Event and hence when bdrb is trying to load event class
>> of its own, its loading your model and its failing.
>>
>> Sadly to overcome this, you will have to perhaps rename your model

I'm certainly able to change my Model names, since it's early in the  
game for this application, but seems that there is a more general  
issue with namespaces here.  Presumably the same thing would happen if  
I had a file matching any of the requires in packet.rb (including  
pimp ;)

> Could changing "Event" to "Packet::Event" help bdrb load the correct  
> class?

Seems that the issue comes from the "require 'event'" line at bdrb's  
packet.rb:10 file.  That line is picking up my RAILS_ROOT/app/model/ 
event.rb file.

Changing that line to:

require './vendor/plugins/backgroundrb/framework/event.rb'

seems to work, at least bdrb is able to start.

I'm very far from really understanding Ruby's namespaces, but the call  
to dbrb's Event.new on core.rb:215 is inside a Packet module ...  
perhaps that will protect it. Perhaps better, though, if bdrb could be  
very pedantic about the namespacing it uses (either includes or full  
Package names for it's internal classes, as suggested above).  Maybe  
even a top level BDRB package for everything?

Thx,
James

ps. Is there any smart automated way to change a Rails Model's name,  
it is scattered all over the place, in files and filenames, not to  
mention the database?

From yem_backgroundrb-devel at filter.yve.net  Tue Jan 22 21:20:17 2008
From: yem_backgroundrb-devel at filter.yve.net (Yves-Eric Martin)
Date: Wed, 23 Jan 2008 11:20:17 +0900
Subject: [Backgroundrb-devel] Unable to load Models
	which	reference	plugins?
In-Reply-To: <61C98FF0-D04B-4439-9665-9B6FF56EDE48@howison.name>
References: <F87ADCB4-112D-4DF3-BE81-E7418F6B8F76@howison.name>	<fb283d5e0801220103i7fbea170hbf413edcccff3fec@mail.gmail.com>	<B46B4352-14CF-4B12-AC02-ACCD13FA678A@howison.name>	<fb283d5e0801221527r561df349tcd3ffb885b9164d5@mail.gmail.com>	<4796817B.9000606@filter.yve.net>
	<61C98FF0-D04B-4439-9665-9B6FF56EDE48@howison.name>
Message-ID: <4796A461.7050408@filter.yve.net>

James Howison wrote:
> ps. Is there any smart automated way to change a Rails Model's name,  
> it is scattered all over the place, in files and filenames, not to  
> mention the database?
>   
Yes, it can be a real pain to rename a model. That is why the "rename 
your model" solution made me wince...

As you pointed out, plugins should be very verbose with their 
namespace/paths in order to avoid conflicting with the main application. 
Now a small problem with

require './vendor/plugins/backgroundrb/framework/event.rb'

is that the path is hardcoded. It could be something else (I could have 
checked out the plugin code into ./vendor/plugins/BackgrounDRb). I went 
through some other plugins code to look for best practices, and it looks 
like a better way to do it would be:

require File.dirname(__FILE__) + '/event'

Can you check and see if that works?

-- 
Yves-Eric


From james at howison.name  Tue Jan 22 21:45:29 2008
From: james at howison.name (James Howison)
Date: Tue, 22 Jan 2008 21:45:29 -0500
Subject: [Backgroundrb-devel] Unable to load Models
	which	reference	plugins?
In-Reply-To: <4796A461.7050408@filter.yve.net>
References: <F87ADCB4-112D-4DF3-BE81-E7418F6B8F76@howison.name>	<fb283d5e0801220103i7fbea170hbf413edcccff3fec@mail.gmail.com>	<B46B4352-14CF-4B12-AC02-ACCD13FA678A@howison.name>	<fb283d5e0801221527r561df349tcd3ffb885b9164d5@mail.gmail.com>	<4796817B.9000606@filter.yve.net>
	<61C98FF0-D04B-4439-9665-9B6FF56EDE48@howison.name>
	<4796A461.7050408@filter.yve.net>
Message-ID: <A72E5883-4178-4ABA-85B1-19AF28FD88F8@howison.name>


On Jan 22, 2008, at 9:20 PM, Yves-Eric Martin wrote:

> James Howison wrote:
>> ps. Is there any smart automated way to change a Rails Model's  
>> name,  it is scattered all over the place, in files and filenames,  
>> not to  mention the database?
>
> Yes, it can be a real pain to rename a model. That is why the  
> "rename your model" solution made me wince...
>
> As you pointed out, plugins should be very verbose with their  
> namespace/paths in order to avoid conflicting with the main  
> application. Now a small problem with
>
> require './vendor/plugins/backgroundrb/framework/event.rb'
>
> is that the path is hardcoded. It could be something else (I could  
> have checked out the plugin code into ./vendor/plugins/BackgrounDRb).

Indeed.

> I went through some other plugins code to look for best practices,  
> and it looks like a better way to do it would be:
>
> require File.dirname(__FILE__) + '/event'
>
> Can you check and see if that works?

Yes, that seems to work (bdrb starts ok), nice one.  Does that take  
care of potential namespace issues, though, maybe still need a unique  
surrounding package?

--J

From gethemant at gmail.com  Wed Jan 23 00:03:39 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 23 Jan 2008 10:33:39 +0530
Subject: [Backgroundrb-devel] Unable to load Models which reference
	plugins?
In-Reply-To: <4796A461.7050408@filter.yve.net>
References: <F87ADCB4-112D-4DF3-BE81-E7418F6B8F76@howison.name>
	<fb283d5e0801220103i7fbea170hbf413edcccff3fec@mail.gmail.com>
	<B46B4352-14CF-4B12-AC02-ACCD13FA678A@howison.name>
	<fb283d5e0801221527r561df349tcd3ffb885b9164d5@mail.gmail.com>
	<4796817B.9000606@filter.yve.net>
	<61C98FF0-D04B-4439-9665-9B6FF56EDE48@howison.name>
	<4796A461.7050408@filter.yve.net>
Message-ID: <fb283d5e0801222103h5e8ba3ao54d45b882a44f973@mail.gmail.com>

Hi,

On Jan 23, 2008 7:50 AM, Yves-Eric Martin
<yem_backgroundrb-devel at filter.yve.net> wrote:
> James Howison wrote:
> > ps. Is there any smart automated way to change a Rails Model's name,
> > it is scattered all over the place, in files and filenames, not to
> > mention the database?
> >
> Yes, it can be a real pain to rename a model. That is why the "rename
> your model" solution made me wince...

Yeah, I know it sucks to rename a model, just for using a plugin.

>
> As you pointed out, plugins should be very verbose with their
> namespace/paths in order to avoid conflicting with the main application.
> Now a small problem with
>
> require './vendor/plugins/backgroundrb/framework/event.rb'
>
> is that the path is hardcoded. It could be something else (I could have
> checked out the plugin code into ./vendor/plugins/BackgrounDRb). I went
> through some other plugins code to look for best practices, and it looks
> like a better way to do it would be:
>
> require File.dirname(__FILE__) + '/event'
>

Okay, Its fixed now. You shouldn't get any conflict now. You don't
need to rename the model, you don't need need to hack the plugin.
Just sync with latest trunk.

I guess this solves all the problems.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Wed Jan 23 00:05:55 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 23 Jan 2008 10:35:55 +0530
Subject: [Backgroundrb-devel] Unable to load Models which reference
	plugins?
In-Reply-To: <61C98FF0-D04B-4439-9665-9B6FF56EDE48@howison.name>
References: <F87ADCB4-112D-4DF3-BE81-E7418F6B8F76@howison.name>
	<fb283d5e0801220103i7fbea170hbf413edcccff3fec@mail.gmail.com>
	<B46B4352-14CF-4B12-AC02-ACCD13FA678A@howison.name>
	<fb283d5e0801221527r561df349tcd3ffb885b9164d5@mail.gmail.com>
	<4796817B.9000606@filter.yve.net>
	<61C98FF0-D04B-4439-9665-9B6FF56EDE48@howison.name>
Message-ID: <fb283d5e0801222105t37eb9019ob176b88d3c0de7e7@mail.gmail.com>

Hi James,

On Jan 23, 2008 6:51 AM, James Howison <james at howison.name> wrote:
>
> On Jan 22, 2008, at 6:51 PM, Yves-Eric Martin wrote:
>
> > hemant wrote:
> >> Looks like you are having a model called event and backgroundrb has a
> >> class called Event and hence when bdrb is trying to load event class
> >> of its own, its loading your model and its failing.
> >>
> >> Sadly to overcome this, you will have to perhaps rename your model
>
> I'm certainly able to change my Model names, since it's early in the
> game for this application, but seems that there is a more general
> issue with namespaces here.  Presumably the same thing would happen if
> I had a file matching any of the requires in packet.rb (including
> pimp ;)
>
> > Could changing "Event" to "Packet::Event" help bdrb load the correct
> > class?
>
> Seems that the issue comes from the "require 'event'" line at bdrb's
> packet.rb:10 file.  That line is picking up my RAILS_ROOT/app/model/
> event.rb file.
>
> Changing that line to:
>
> require './vendor/plugins/backgroundrb/framework/event.rb'
>
> seems to work, at least bdrb is able to start.

Get rid of your existing copy and sync with trunk. bdrb will no longer
conflict with any of your model files.

>
> I'm very far from really understanding Ruby's namespaces, but the call
> to dbrb's Event.new on core.rb:215 is inside a Packet module ...
> perhaps that will protect it. Perhaps better, though, if bdrb could be
> very pedantic about the namespacing it uses (either includes or full
> Package names for it's internal classes, as suggested above).  Maybe
> even a top level BDRB package for everything?
>
> Thx,
> James
>
> ps. Is there any smart automated way to change a Rails Model's name,
> it is scattered all over the place, in files and filenames, not to
> mention the database?

Unless you use netbeans or something, I guess no. And please don't
rename the model if you have lots of code written around it.


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Wed Jan 23 00:25:27 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 23 Jan 2008 10:55:27 +0530
Subject: [Backgroundrb-devel] Unable to load Models which reference
	plugins?
In-Reply-To: <A72E5883-4178-4ABA-85B1-19AF28FD88F8@howison.name>
References: <F87ADCB4-112D-4DF3-BE81-E7418F6B8F76@howison.name>
	<fb283d5e0801220103i7fbea170hbf413edcccff3fec@mail.gmail.com>
	<B46B4352-14CF-4B12-AC02-ACCD13FA678A@howison.name>
	<fb283d5e0801221527r561df349tcd3ffb885b9164d5@mail.gmail.com>
	<4796817B.9000606@filter.yve.net>
	<61C98FF0-D04B-4439-9665-9B6FF56EDE48@howison.name>
	<4796A461.7050408@filter.yve.net>
	<A72E5883-4178-4ABA-85B1-19AF28FD88F8@howison.name>
Message-ID: <fb283d5e0801222125p36eaec9cs4ea506a9710dac70@mail.gmail.com>

On Jan 23, 2008 8:15 AM, James Howison <james at howison.name> wrote:
>
> On Jan 22, 2008, at 9:20 PM, Yves-Eric Martin wrote:
>
> > James Howison wrote:
> >> ps. Is there any smart automated way to change a Rails Model's
> >> name,  it is scattered all over the place, in files and filenames,
> >> not to  mention the database?
> >
> > Yes, it can be a real pain to rename a model. That is why the
> > "rename your model" solution made me wince...
> >
> > As you pointed out, plugins should be very verbose with their
> > namespace/paths in order to avoid conflicting with the main
> > application. Now a small problem with
> >
> > require './vendor/plugins/backgroundrb/framework/event.rb'
> >
> > is that the path is hardcoded. It could be something else (I could
> > have checked out the plugin code into ./vendor/plugins/BackgrounDRb).
>
> Indeed.
>
> > I went through some other plugins code to look for best practices,
> > and it looks like a better way to do it would be:
> >
> > require File.dirname(__FILE__) + '/event'
> >
> > Can you check and see if that works?
>
> Yes, that seems to work (bdrb starts ok), nice one.  Does that take
> care of potential namespace issues, though, maybe still need a unique
> surrounding package?

We didn't had namespace issues, Event class was already inside packet
model so properly namespaced.

From gethemant at gmail.com  Wed Jan 23 00:30:17 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 23 Jan 2008 11:00:17 +0530
Subject: [Backgroundrb-devel] Threadpool and queuing of tasks
In-Reply-To: <258cc3f50801221343kb68d84as2d393a3bd805c2c9@mail.gmail.com>
References: <258cc3f50801221343kb68d84as2d393a3bd805c2c9@mail.gmail.com>
Message-ID: <fb283d5e0801222130t17f3b690xcb5f96b2b43769b@mail.gmail.com>

Hi Dave,

On Jan 23, 2008 3:13 AM, Dave Dupre <gobigdave at gmail.com> wrote:
> I recently switched over to v1.0, and things are rolling along pretty well.

Switch to trunk or 1.0.1. Preferably trunk, we fixed lots of issues in past.

> However, one thing that has always been a little confusing to me is knowing
> when to use thread_pool.  Since most of my bgrb workers are called from my
> web app to process rather than being scheduled, I'm using the thread_pool
> for every call.  Unfortunately, that means that I have to split up workers
> by how many threads I can have.  It would be great if one worker could
> partition a single thread pool among the methods.  I want to avoid too many
> workers to keep the process count down.

I don't think I follow you here. Since a worker comes with thread pool
of size 20, you should be good to go. But obviously Ruby green threads
don't offer you any parallel execution.
Thread pools has been designed to run concurrent tasks (not parallel),
it wouldn't be useful to have the ability to partition thread pool
among methods.
In fact, I don't follow that notion at all, you mean you want to
assign number of threads for each method that you are invoking from
rails?

Can you just clarify things a bit?


>
> I'm now working on a new scheme that pushes this example.  Basically, I have
> some long running, saved searches that are triggered by various events
> throughout the site.  All I want my site to do is update a status that the
> job is queued and have it picked up from there.  Here is where I run into
> trouble, possibly because I've built too many systems like this that use
> real queuing packages.  Here is what I want:
>
> Dispatch method (usually one thread is necessary):
> 1. Find the oldest 'queued' record (make sure to find with :lock => true)
> 1a. If none, goto step 5
> 2. Update status to 'processing'
>  3. Send to search method
> 4. Repeat 1
> 5. Done
>
> Search method (many threads):
> 1. Perform the search
> 2. Update status to 'complete'
> 3. Done
>
> The easy answer is to split these into two workers.  Set the pool_size of
> Dispatch to 1, and Search to 5 or 10.  However, eating two processes (master
> and worker) for something so simple as Dispatch seems like serious overkill
> to me.  Since I currently run on one server, the extra processes cut into
> the memory the main site wants.

Again, I didn't quite follow you there, so let me just rephrase the
thing that you want and if I understood it correctly.
Basically you want to implement a queue, so when a task is submitted
from rails the task gets queued.
Now dispatch worker, finds the latest(or oldest?) task, and updates
the status of the task as taken and hands over the tasks to search
worker.
Is that right?

You can use Queue class for that purpose. And you don't need to poll
on a queue, because if Queue is empty all the threads that reading
from the queue are automatically blocked.
bdrb thread pool implementation makes use of that. So, whats wrong
with a thread pool of size 10 or 20 and add each task to that queue,
when a task is removed from the queue, mark the status as processing
and just go on with processing of the task. You don't need two
processes for this.

>
> A related question is how to implement Dispatch without polling.  Call me
> anal, but I feel dirty whenever I using polling, especially something that I
> want to be picked up immediately.  Is there a way I can trigger it to run if
> it isn't already?  The old bgrb had a singleton that let me do something
> like that.
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gapeldoorn at wehkamp.nl  Wed Jan 23 09:10:17 2008
From: gapeldoorn at wehkamp.nl (Ger Apeldoorn)
Date: Wed, 23 Jan 2008 15:10:17 +0100
Subject: [Backgroundrb-devel] Cannot connect when spawning
	new	workers	on demand
In-Reply-To: <1200996191.9575.27.camel@localhost>
References: <1200990582.9575.20.camel@localhost>
	<fb283d5e0801220148paaaaff6j6ef6bce8b85cabe0@mail.gmail.com>
	<1200996191.9575.27.camel@localhost>
Message-ID: <1201097417.5709.17.camel@localhost>

Hi again,

Can I perhaps provide more information to help solve this problem? If
so, I'd be happy to give it!

Thanks,
Ger.

On Tue, 2008-01-22 at 11:03 +0100, Ger Apeldoorn wrote:
> Hi!
> 
> Thanks a lot for your reply!
> 
> > Did you check backgroundrb_debug.log ?
> > You can turn on debugging to foreground with:
> > 
> > :log: foreground
> 
> Where should I put :log: foreground?
> 
> Anyway, this comes from log/backgroundrb_debug.log:
> 
> --------------------%<--------------------
> 
> $ tail -f log/backgroundrb_debug.log 
> 000000073:      type:start_worker:
>                                    worker:remote_command_worker:
> 
> job_key"
> 
> server06
> {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server06"}
> 000000115{
> :       type:
>              do_work:
>                      worker:remote_command_worker:     data{:
>                                                               command"
> megacommandheresvr"
> server06:
>         job_key:worker_method;
> {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server06", :command=>"megacommandhere"}, :worker=>:remote_command_worker, :job_key=>"server06"}
> 000000073:      type:start_worker:
>                                    worker:remote_command_worker:
> 
> job_key"
> 
> server12
> {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server12"}
> 000000115{
> :       type:
>              do_work:
>                      worker:remote_command_worker:     data{:
>                                                               command"
> megacommandheresvr"
> server12:
>         job_key:worker_method;
> {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server12", :command=>"megacommandhere"}, :worker=>:remote_command_worker, :job_key=>"server12"}
> 
> --------------------%<--------------------
> 
> > 
> > option to further debug this. Something is killing BackgrounDRb master
> > process. Can you sync with trunk and enable foreground logging and
> > check on this?
> 
> Is there an easy way to update a plugin after it's been installed
> besides removing it and checking it out again?
> 
> > Also, if possible, can we see your worker code?
> 
> Here's my worker:
> 
> class RemoteCommandWorker < BackgrounDRb::MetaWorker
>   set_worker_name :remote_command_worker
>   set_no_auto_load :true
> 
>   def create(args = nil)
>     register_status("processing")
>   end
> 
>   def do_work(args = nil)
>     result = String.new
>     result = "Result for #{args[:command]}:\n #{%
> x[running_a_shell_script_here]}"
>     register_status(result)
>   end
> end
> 
> Thanks,
> Ger.
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

From gobigdave at gmail.com  Wed Jan 23 09:28:12 2008
From: gobigdave at gmail.com (Dave Dupre)
Date: Wed, 23 Jan 2008 09:28:12 -0500
Subject: [Backgroundrb-devel] Threadpool and queuing of tasks
In-Reply-To: <fb283d5e0801222130t17f3b690xcb5f96b2b43769b@mail.gmail.com>
References: <258cc3f50801221343kb68d84as2d393a3bd805c2c9@mail.gmail.com>
	<fb283d5e0801222130t17f3b690xcb5f96b2b43769b@mail.gmail.com>
Message-ID: <258cc3f50801230628p5012a47co1c39137f3228e1a6@mail.gmail.com>

I wasn't specific.  I am using the trunk version.

Yes, I want to limit methods to the number of threads they run.  For
example, I have a worker with many methods that I use for several longer
running tasks.  All methods are called from the web application so I use the
thread_pool to queue up requests.  However, during some processes, I may
queue up a few hundred calls to a couple of methods.  These methods use a
3rd party web service to gather information, and at a pool_size of 20 I
overload the 3rd party server.  Through trial and error, 2-3 is about as
many as I can run.  The problem now is that I have to reduce the threads for
every method in the worker or create another process.

Regarding the queue, are you saying just do something like:

def create(args = nil)
  # Fire everyone back up if the process is restarted
  jobs = Job.find(:all, :conditions => "state = 'queued' or state =
'processing'", :order => 'created_at ')
  jobs.each  { |j| my_worker(j.id)
end

def my_worker(args = nil)
  thread_pool.defer(args) do |job_id|
    job = Job.find(job_id, :lock => true)  # Lock the row till we update the
state
    job.process!  # assuming acts_as_state_machine (this will change the
state and process the job)
  end
end

This I get, and in fact I started with an implementation just like this.
The reason I added the dispatcher is because of the way my app works.  My
app makes use of saved searches to connect users based on several
properties. The searches are very complex and are automatically kicked off
based on several triggers.  The problem with the above is that multiple
quick triggers will result in multiple calls to my_worker, and an expensive
operation runs multiple times.  However, if all the triggers do is update
the databased to change the state to 'queued' and wake up the dispatcher, I
will only have one call to my_worker (acts_as_state_machine will only change
the state to 'queued' if the current state is 'ready').  Obviously, I can
add a lot of logic to handle this, but using the persistent, database queue
seems so much simpler and safer.

Also, another reason for the dispatcher to allow for future scale. I got in
the habit of adding a dispatcher in cases like this to make it very simple
to have the dispatcher send tasks to multiple processing queues, often on
separate servers.  One dispatcher process can easily handle dozens of
processing servers (I've had as many as 75 processing servers without a
problem).

Dave

On Jan 23, 2008 12:30 AM, hemant <gethemant at gmail.com> wrote:

> Hi Dave,
>
> On Jan 23, 2008 3:13 AM, Dave Dupre <gobigdave at gmail.com> wrote:
> > I recently switched over to v1.0, and things are rolling along pretty
> well.
>
> Switch to trunk or 1.0.1. Preferably trunk, we fixed lots of issues in
> past.
>
> > However, one thing that has always been a little confusing to me is
> knowing
> > when to use thread_pool.  Since most of my bgrb workers are called from
> my
> > web app to process rather than being scheduled, I'm using the
> thread_pool
> > for every call.  Unfortunately, that means that I have to split up
> workers
> > by how many threads I can have.  It would be great if one worker could
> > partition a single thread pool among the methods.  I want to avoid too
> many
> > workers to keep the process count down.
>
> I don't think I follow you here. Since a worker comes with thread pool
> of size 20, you should be good to go. But obviously Ruby green threads
> don't offer you any parallel execution.
> Thread pools has been designed to run concurrent tasks (not parallel),
> it wouldn't be useful to have the ability to partition thread pool
> among methods.
> In fact, I don't follow that notion at all, you mean you want to
> assign number of threads for each method that you are invoking from
> rails?
>
> Can you just clarify things a bit?
>
>
> >
> > I'm now working on a new scheme that pushes this example.  Basically, I
> have
> > some long running, saved searches that are triggered by various events
> > throughout the site.  All I want my site to do is update a status that
> the
> > job is queued and have it picked up from there.  Here is where I run
> into
> > trouble, possibly because I've built too many systems like this that use
> > real queuing packages.  Here is what I want:
> >
> > Dispatch method (usually one thread is necessary):
> > 1. Find the oldest 'queued' record (make sure to find with :lock =>
> true)
> > 1a. If none, goto step 5
> > 2. Update status to 'processing'
> >  3. Send to search method
> > 4. Repeat 1
> > 5. Done
> >
> > Search method (many threads):
> > 1. Perform the search
> > 2. Update status to 'complete'
> > 3. Done
> >
> > The easy answer is to split these into two workers.  Set the pool_size
> of
> > Dispatch to 1, and Search to 5 or 10.  However, eating two processes
> (master
> > and worker) for something so simple as Dispatch seems like serious
> overkill
> > to me.  Since I currently run on one server, the extra processes cut
> into
> > the memory the main site wants.
>
> Again, I didn't quite follow you there, so let me just rephrase the
> thing that you want and if I understood it correctly.
> Basically you want to implement a queue, so when a task is submitted
> from rails the task gets queued.
> Now dispatch worker, finds the latest(or oldest?) task, and updates
> the status of the task as taken and hands over the tasks to search
> worker.
> Is that right?
>
> You can use Queue class for that purpose. And you don't need to poll
> on a queue, because if Queue is empty all the threads that reading
> from the queue are automatically blocked.
> bdrb thread pool implementation makes use of that. So, whats wrong
> with a thread pool of size 10 or 20 and add each task to that queue,
> when a task is removed from the queue, mark the status as processing
> and just go on with processing of the task. You don't need two
> processes for this.
>
> >
> > A related question is how to implement Dispatch without polling.  Call
> me
> > anal, but I feel dirty whenever I using polling, especially something
> that I
> > want to be picked up immediately.  Is there a way I can trigger it to
> run if
> > it isn't already?  The old bgrb had a singleton that let me do something
> > like that.
> >
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080123/09a26c56/attachment-0001.html 

From gethemant at gmail.com  Wed Jan 23 11:54:22 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 23 Jan 2008 22:24:22 +0530
Subject: [Backgroundrb-devel] Cannot connect when spawning new workers
	on demand
In-Reply-To: <1201097417.5709.17.camel@localhost>
References: <1200990582.9575.20.camel@localhost>
	<fb283d5e0801220148paaaaff6j6ef6bce8b85cabe0@mail.gmail.com>
	<1200996191.9575.27.camel@localhost>
	<1201097417.5709.17.camel@localhost>
Message-ID: <fb283d5e0801230854s6d52504ejaf53eb4777022ba2@mail.gmail.com>

On Jan 23, 2008 7:40 PM, Ger Apeldoorn <gapeldoorn at wehkamp.nl> wrote:
> Hi again,
>
> Can I perhaps provide more information to help solve this problem? If
> so, I'd be happy to give it!

You should put that in backgroundrb.yml file. Its described in README file.



>
> Thanks,
> Ger.
>
>
> On Tue, 2008-01-22 at 11:03 +0100, Ger Apeldoorn wrote:
> > Hi!
> >
> > Thanks a lot for your reply!
> >
> > > Did you check backgroundrb_debug.log ?
> > > You can turn on debugging to foreground with:
> > >
> > > :log: foreground
> >
> > Where should I put :log: foreground?
> >
> > Anyway, this comes from log/backgroundrb_debug.log:
> >
> > --------------------%<--------------------
> >
> > $ tail -f log/backgroundrb_debug.log
> > 000000073:      type: start_worker:
> >                                    worker: remote_command_worker:
> >
> > job_key"
> >
> > server06
> > {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server06"}
> > 000000115{
> > :       type:
> >              do_work:
> >                      worker: remote_command_worker:     data{:
> >                                                               command"
> > megacommandheresvr"
> > server06:
> >         job_key: worker_method;
> > {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server06", :command=>"megacommandhere"}, :worker=>:remote_command_worker, :job_key=>"server06"}
> > 000000073:      type: start_worker:
> >                                    worker: remote_command_worker:
> >
> > job_key"
> >
> > server12
> > {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server12"}
> > 000000115{
> > :       type:
> >              do_work:
> >                      worker: remote_command_worker:     data{:
> >                                                               command"
> > megacommandheresvr"
> > server12:
> >         job_key: worker_method;
> > {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server12", :command=>"megacommandhere"}, :worker=>:remote_command_worker, :job_key=>"server12"}
> >
> > --------------------%<--------------------
> >
> > >
> > > option to further debug this. Something is killing BackgrounDRb master
> > > process. Can you sync with trunk and enable foreground logging and
> > > check on this?
> >
> > Is there an easy way to update a plugin after it's been installed
> > besides removing it and checking it out again?
> >
> > > Also, if possible, can we see your worker code?
> >
> > Here's my worker:
> >
> > class RemoteCommandWorker < BackgrounDRb::MetaWorker
> >   set_worker_name :remote_command_worker
> >   set_no_auto_load :true
> >
> >   def create(args = nil)
> >     register_status("processing")
> >   end
> >
> >   def do_work(args = nil)
> >     result = String.new
> >     result = "Result for #{args[:command]}:\n #{%
> > x[running_a_shell_script_here]}"
> >     register_status(result)
> >   end
> > end
> >
> > Thanks,
> > Ger.
>
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From james at howison.name  Wed Jan 23 13:56:37 2008
From: james at howison.name (James Howison)
Date: Wed, 23 Jan 2008 13:56:37 -0500
Subject: [Backgroundrb-devel] Unable to load Models which reference
	plugins?
In-Reply-To: <fb283d5e0801222105t37eb9019ob176b88d3c0de7e7@mail.gmail.com>
References: <F87ADCB4-112D-4DF3-BE81-E7418F6B8F76@howison.name>
	<fb283d5e0801220103i7fbea170hbf413edcccff3fec@mail.gmail.com>
	<B46B4352-14CF-4B12-AC02-ACCD13FA678A@howison.name>
	<fb283d5e0801221527r561df349tcd3ffb885b9164d5@mail.gmail.com>
	<4796817B.9000606@filter.yve.net>
	<61C98FF0-D04B-4439-9665-9B6FF56EDE48@howison.name>
	<fb283d5e0801222105t37eb9019ob176b88d3c0de7e7@mail.gmail.com>
Message-ID: <42D10FD8-305D-4DE6-843F-1D99B10E0BA5@howison.name>


On Jan 23, 2008, at 12:05 AM, hemant wrote:

> Hi James,
>
> On Jan 23, 2008 6:51 AM, James Howison <james at howison.name> wrote:
>>
>> On Jan 22, 2008, at 6:51 PM, Yves-Eric Martin wrote:
>>
>>> hemant wrote:
>>>> Looks like you are having a model called event and backgroundrb  
>>>> has a
>>>> class called Event and hence when bdrb is trying to load event  
>>>> class
>>>> of its own, its loading your model and its failing.
>>>>
>>>> Sadly to overcome this, you will have to perhaps rename your model
>>
>> I'm certainly able to change my Model names, since it's early in the
>> game for this application, but seems that there is a more general
>> issue with namespaces here.  Presumably the same thing would happen  
>> if
>> I had a file matching any of the requires in packet.rb (including
>> pimp ;)
>>
>>> Could changing "Event" to "Packet::Event" help bdrb load the correct
>>> class?
>>
>> Seems that the issue comes from the "require 'event'" line at bdrb's
>> packet.rb:10 file.  That line is picking up my RAILS_ROOT/app/model/
>> event.rb file.
>>
>> Changing that line to:
>>
>> require './vendor/plugins/backgroundrb/framework/event.rb'
>>
>> seems to work, at least bdrb is able to start.
>
> Get rid of your existing copy and sync with trunk. bdrb will no longer
> conflict with any of your model files.

Great, the new directory structure looks to have solved the problem.   
Thanks.

James

From gapeldoorn at wehkamp.nl  Thu Jan 24 05:47:42 2008
From: gapeldoorn at wehkamp.nl (Ger Apeldoorn)
Date: Thu, 24 Jan 2008 11:47:42 +0100
Subject: [Backgroundrb-devel] Cannot connect when spawning new
	workers	on demand
In-Reply-To: <fb283d5e0801230854s6d52504ejaf53eb4777022ba2@mail.gmail.com>
References: <1200990582.9575.20.camel@localhost>
	<fb283d5e0801220148paaaaff6j6ef6bce8b85cabe0@mail.gmail.com>
	<1200996191.9575.27.camel@localhost>
	<1201097417.5709.17.camel@localhost>
	<fb283d5e0801230854s6d52504ejaf53eb4777022ba2@mail.gmail.com>
Message-ID: <1201171662.5709.26.camel@localhost>

Hi,

Here's output from the debug log: 
Btw; There were a lot of control characters in there which I've removed.

-------------------%<--------------------

{:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server06"}
000000123{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server06:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server06", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server06"}
000000075{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server06
{:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server06"}
000000123{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server06:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server06", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server06"}
000000073{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server08
{:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server08"}
000000121{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server08:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server08", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server08"}
000000074{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server06
{:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server06"}
000000122{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server06:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server06", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server06"}
000000074{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server04
{:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server04"}
000000122{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server04:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server04", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server04"}
000000073{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server04
{:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server04"}
000000121{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server04:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server04", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server04"}
000000074{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server07
{:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server07"}
000000122{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server07:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server07", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server07"}
000000073{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server07
{:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server07"}
000000121{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server07:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server07", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server07"}
000000075{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server05
{:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server05"}
000000123{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server05:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server05", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server05"}
000000075{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server07
{:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server07"}
000000123{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server07:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server07", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server07"}

--------------------%<--------------------

Any help very much appreciated!!

Ger.

From gethemant at gmail.com  Thu Jan 24 06:32:38 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 24 Jan 2008 17:02:38 +0530
Subject: [Backgroundrb-devel] Cannot connect when spawning new workers
	on demand
In-Reply-To: <1201171662.5709.26.camel@localhost>
References: <1200990582.9575.20.camel@localhost>
	<fb283d5e0801220148paaaaff6j6ef6bce8b85cabe0@mail.gmail.com>
	<1200996191.9575.27.camel@localhost>
	<1201097417.5709.17.camel@localhost>
	<fb283d5e0801230854s6d52504ejaf53eb4777022ba2@mail.gmail.com>
	<1201171662.5709.26.camel@localhost>
Message-ID: <fb283d5e0801240332k4c3ae2a9mab7f9883a50626b9@mail.gmail.com>

Hi Ger,

On Jan 24, 2008 4:17 PM, Ger Apeldoorn <gapeldoorn at wehkamp.nl> wrote:
> Hi,
>
> Here's output from the debug log:
> Btw; There were a lot of control characters in there which I've removed.
>
> -------------------%<--------------------
>
> {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server06"}
> 000000123{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server06:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server06", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server06"}
> 000000075{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server06
> {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server06"}
> 000000123{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server06:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server06", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server06"}
> 000000073{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server08
> {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server08"}
> 000000121{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server08:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server08", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server08"}
> 000000074{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server06
> {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server06"}
> 000000122{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server06:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server06", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server06"}
> 000000074{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server04
> {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server04"}
> 000000122{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server04:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server04", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server04"}
> 000000073{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server04
> {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server04"}
> 000000121{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server04:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server04", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server04"}
> 000000074{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server07
> {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server07"}
> 000000122{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server07:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server07", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server07"}
> 000000073{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server07
> {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server07"}
> 000000121{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server07:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server07", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server07"}
> 000000075{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server05
> {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server05"}
> 000000123{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server05:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server05", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server05"}
> 000000075{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server07
> {:type=>:start_worker, :worker=>:remote_command_worker, :job_key=>"server07"}
> 000000123{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server07:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work, :data=>{:svr=>"server07", :command=>"magiccommand"}, :worker=>:remote_command_worker, :job_key=>"server07"}
>
> --------------------%<--------------------
>

I am looking for some exceptions, when the master process dies! I
can't see any so far, in your log output.
Thats the reason, I have been asking you to run bdrb in foreground mode using:

:log: foreground

option and start backgroundrb with:

./script/backgroundrb

This way, when master crashed message will be logged to console.


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gapeldoorn at wehkamp.nl  Thu Jan 24 06:58:13 2008
From: gapeldoorn at wehkamp.nl (Ger Apeldoorn)
Date: Thu, 24 Jan 2008 12:58:13 +0100
Subject: [Backgroundrb-devel] Cannot connect when spawning new
	workers	on demand
In-Reply-To: <fb283d5e0801240332k4c3ae2a9mab7f9883a50626b9@mail.gmail.com>
References: <1200990582.9575.20.camel@localhost>
	<fb283d5e0801220148paaaaff6j6ef6bce8b85cabe0@mail.gmail.com>
	<1200996191.9575.27.camel@localhost>
	<1201097417.5709.17.camel@localhost>
	<fb283d5e0801230854s6d52504ejaf53eb4777022ba2@mail.gmail.com>
	<1201171662.5709.26.camel@localhost>
	<fb283d5e0801240332k4c3ae2a9mab7f9883a50626b9@mail.gmail.com>
Message-ID: <1201175893.5709.36.camel@localhost>

Hi,

My apologies, as I didnt get any output from the ./script/backgroundrb
in the console, I thought the debug log was the place to go.... (even
with the :log: foreground in place)

That said, the backgroundrb master process doesn't crash. After I get
the 'cannot connect' error, I can push 'back' on my browser and try
again with less servers successfully. Even an F5 so that all POST
parameters are submitted again can succeed. (all without restarting
anything)

I hope this can shed some light.. :)

Thanks,
Ger.



> 
> I am looking for some exceptions, when the master process dies! I
> can't see any so far, in your log output.
> Thats the reason, I have been asking you to run bdrb in foreground mode using:
> 
> :log: foreground
> 
> option and start backgroundrb with:
> 
> ./script/backgroundrb
> 
> This way, when master crashed message will be logged to console.
> 
> 

From gethemant at gmail.com  Thu Jan 24 08:41:18 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 24 Jan 2008 19:11:18 +0530
Subject: [Backgroundrb-devel] Cannot connect when spawning new workers
	on demand
In-Reply-To: <1201175893.5709.36.camel@localhost>
References: <1200990582.9575.20.camel@localhost>
	<fb283d5e0801220148paaaaff6j6ef6bce8b85cabe0@mail.gmail.com>
	<1200996191.9575.27.camel@localhost>
	<1201097417.5709.17.camel@localhost>
	<fb283d5e0801230854s6d52504ejaf53eb4777022ba2@mail.gmail.com>
	<1201171662.5709.26.camel@localhost>
	<fb283d5e0801240332k4c3ae2a9mab7f9883a50626b9@mail.gmail.com>
	<1201175893.5709.36.camel@localhost>
Message-ID: <fb283d5e0801240541n34ce2cbif50997632072c466@mail.gmail.com>

On Jan 24, 2008 5:28 PM, Ger Apeldoorn <gapeldoorn at wehkamp.nl> wrote:
> Hi,
>
> My apologies, as I didnt get any output from the ./script/backgroundrb
> in the console, I thought the debug log was the place to go.... (even
> with the :log: foreground in place)
>
> That said, the backgroundrb master process doesn't crash. After I get
> the 'cannot connect' error, I can push 'back' on my browser and try
> again with less servers successfully. Even an F5 so that all POST
> parameters are submitted again can succeed. (all without restarting
> anything)
>
> I hope this can shed some light.. :)

Okay, I further debugged this issue. The first Question is, are you
deleting the workers? In the first and second mail that you pasted, I
can't see any calls to either "exit" or MiddleMan.delete_worker().

What I found that, if you have too many workers running, even if
master doesn't die, bdrb master may refuse the connection. For me,
threshold was around 50 parallel workers on the same machine. I
couldn't find any fault in bdrb code, since I have 1 GB memory on this
machine and 50 parallel workers mean,  50*30 = 1500MB of memory.

Assuming other tasks running, available memory is around 600 MB or
something, and swapping brings my machine to a grinding hault.
So, its very important that you delete your worker. In worst case, if
you still can't solve the issue, just archive the your app and send me
( assuming its not too confidential, and you have reasonable amount of
trust on me).





-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From GApeldoorn at Wehkamp.nl  Thu Jan 24 10:15:08 2008
From: GApeldoorn at Wehkamp.nl (Apeldoorn, Ger)
Date: Thu, 24 Jan 2008 16:15:08 +0100
Subject: [Backgroundrb-devel] Cannot connect when spawning new workerson
	demand
In-Reply-To: <fb283d5e0801240541n34ce2cbif50997632072c466@mail.gmail.com>
References: <1200990582.9575.20.camel@localhost>
	<fb283d5e0801220148paaaaff6j6ef6bce8b85cabe0@mail.gmail.com>
	<1200996191.9575.27.camel@localhost>
	<1201097417.5709.17.camel@localhost>
	<fb283d5e0801230854s6d52504ejaf53eb4777022ba2@mail.gmail.com>
	<1201171662.5709.26.camel@localhost>
	<fb283d5e0801240332k4c3ae2a9mab7f9883a50626b9@mail.gmail.com>
	<1201175893.5709.36.camel@localhost>
	<fb283d5e0801240541n34ce2cbif50997632072c466@mail.gmail.com>
Message-ID: <1201187708.5709.39.camel@localhost>

Hi,

> Okay, I further debugged this issue. The first Question is, are you
> deleting the workers? In the first and second mail that you pasted, I
> can't see any calls to either "exit" or MiddleMan.delete_worker().

No, I am deleting the workers in another function when all the work is
done.

I have access to another machine with rails on it and more memory, I'll
try if it is a problem there. If I cannot figure it out, I'll send you
the application.

Thanks a lot for your time and effort!
Ger.

> What I found that, if you have too many workers running, even if
> master doesn't die, bdrb master may refuse the connection. For me,
> threshold was around 50 parallel workers on the same machine. I
> couldn't find any fault in bdrb code, since I have 1 GB memory on this
> machine and 50 parallel workers mean,  50*30 = 1500MB of memory.
> 
> Assuming other tasks running, available memory is around 600 MB or
> something, and swapping brings my machine to a grinding hault.
> So, its very important that you delete your worker. In worst case, if
> you still can't solve the issue, just archive the your app and send me
> ( assuming its not too confidential, and you have reasonable amount of
> trust on me).
> 
> 
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080124/042efecd/attachment-0001.html 

From mengkuan at gmail.com  Fri Jan 25 04:45:09 2008
From: mengkuan at gmail.com (Meng Kuan)
Date: Fri, 25 Jan 2008 17:45:09 +0800
Subject: [Backgroundrb-devel] worker and sleep
Message-ID: <3126B3CF-33A7-407F-B07A-82BD923ACA36@gmail.com>

Hi Hermant,

I have seen some earlier messages in the list where you warned people  
never to use sleep() in workers. However, currently I have a need for  
a worker to issue a command to an external host and then keep  
checking that external host for a status change. Other than sleep,  
what else can be used to make a worker wait some time (say 1 second)  
between checks?

cheers,
mengkuan

From gethemant at gmail.com  Fri Jan 25 05:27:52 2008
From: gethemant at gmail.com (hemant)
Date: Fri, 25 Jan 2008 15:57:52 +0530
Subject: [Backgroundrb-devel] worker and sleep
In-Reply-To: <3126B3CF-33A7-407F-B07A-82BD923ACA36@gmail.com>
References: <3126B3CF-33A7-407F-B07A-82BD923ACA36@gmail.com>
Message-ID: <fb283d5e0801250227k5b27591cr243c9630efbfe463@mail.gmail.com>

On Fri, Jan 25, 2008 at 3:15 PM, Meng Kuan <mengkuan at gmail.com> wrote:
> Hi Hermant,
>
>  I have seen some earlier messages in the list where you warned people
>  never to use sleep() in workers. However, currently I have a need for
>  a worker to issue a command to an external host and then keep
>  checking that external host for a status change. Other than sleep,
>  what else can be used to make a worker wait some time (say 1 second)
>  between checks?

add_periodic_timer(1) { check_status_change }



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From scott at shefield.com  Fri Jan 25 17:32:50 2008
From: scott at shefield.com (Scott Ward)
Date: Fri, 25 Jan 2008 14:32:50 -0800
Subject: [Backgroundrb-devel] Can schedules START a worker?
Message-ID: <20080125224522.0395F1858646@rubyforge.org>

Hi there,

I'm just working on getting backgroundrb (1.0.1) up and running.  I'm no
expert with ruby, etc, so I still don't quite get a few things.

I have scheduled workers that run every few minutes.  That works fine.  I
have other workers that run once per day or even once per week.  It doesn't
make sense to have them running all the time.  I'd rather they just fire up
when the time comes and shut down again.  

I thought I could "set_auto_load true" and that they would be started up
when the schedule dictated.  Then I'd use "exit" at the end to stop them
again.

But apparently, the schedules only work on running workers.

What am I missing?  What's the best way to handle workers that only need to
take action once per day and therefore don't need to be running all the
time?

Thanks,

Scott


From gethemant at gmail.com  Fri Jan 25 18:19:34 2008
From: gethemant at gmail.com (hemant)
Date: Sat, 26 Jan 2008 04:49:34 +0530
Subject: [Backgroundrb-devel] Can schedules START a worker?
In-Reply-To: <20080125224522.0395F1858646@rubyforge.org>
References: <20080125224522.0395F1858646@rubyforge.org>
Message-ID: <fb283d5e0801251519sa44c083o38006e050517776e@mail.gmail.com>

Hi Scott,

On Jan 26, 2008 4:02 AM, Scott Ward <scott at shefield.com> wrote:
> Hi there,
>
> I'm just working on getting backgroundrb (1.0.1) up and running.  I'm no
> expert with ruby, etc, so I still don't quite get a few things.
>
> I have scheduled workers that run every few minutes.  That works fine.  I
> have other workers that run once per day or even once per week.  It doesn't
> make sense to have them running all the time.  I'd rather they just fire up
> when the time comes and shut down again.
>
> I thought I could "set_auto_load true" and that they would be started up
> when the schedule dictated.  Then I'd use "exit" at the end to stop them
> again.
>
> But apparently, the schedules only work on running workers.
>
> What am I missing?  What's the best way to handle workers that only need to
> take action once per day and therefore don't need to be running all the
> time?

Currently you can't. You will have to let the worker running, but if
you can bear with us for few weeks, I am working on getting a new
release of bdrb out of door and it will have this feature.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From meechman at gmail.com  Fri Jan 25 23:56:06 2008
From: meechman at gmail.com (Meech)
Date: Fri, 25 Jan 2008 23:56:06 -0500
Subject: [Backgroundrb-devel] Connections
Message-ID: <313751190801252056t61502487ofd270ba5888bca3@mail.gmail.com>

I'm checking out backgroundrb for a calculation process I estimate will take
2-3 minutes and be ran on-demand only via Rails.  Very easy and slick.

But I have a question.

The connections never close, if I do a netstat they just keep piling up.

Do I need to do some housekeeping somewhere?   I noticed the @
connection.close statements are commented out in favor of a # FIXME type
deal.

Even if I store read_from_bdrb in an intermediate variable, call @
connection.close then return, they still sit in a CLOSE_WAIT status forever
-- long, long after mongrel has cleaned up.

I'm not a communications guy, so if I'm doing something wrong, let me know.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080125/f2beeb76/attachment.html 

From subimage at gmail.com  Sat Jan 26 00:14:34 2008
From: subimage at gmail.com (subimage interactive)
Date: Fri, 25 Jan 2008 21:14:34 -0800
Subject: [Backgroundrb-devel] Connections
In-Reply-To: <313751190801252056t61502487ofd270ba5888bca3@mail.gmail.com>
References: <313751190801252056t61502487ofd270ba5888bca3@mail.gmail.com>
Message-ID: <7aff9b4c0801252114q59f93506rfbcc4bbe406579d6@mail.gmail.com>

'ps -ef | grep worker' on your box. If you find there are a bunch hanging
around - there's your problem. I'm running an ooold version of BDRB (0.2.1),
and I have to kill the workers from rails, or call 'exit!' at the bottom of
my do_work method to get things behaving properly.

You might be encountering something similar. Not sure what the latest and
greatest method of achieving this is...TTL perhaps anyone?

On Jan 25, 2008 8:56 PM, Meech <meechman at gmail.com> wrote:

> I'm checking out backgroundrb for a calculation process I estimate will
> take 2-3 minutes and be ran on-demand only via Rails.  Very easy and slick.
>
>
> But I have a question.
>
> The connections never close, if I do a netstat they just keep piling up.
>
> Do I need to do some housekeeping somewhere?   I noticed the @
> connection.close statements are commented out in favor of a # FIXME type
> deal.
>
> Even if I store read_from_bdrb in an intermediate variable, call @
> connection.close then return, they still sit in a CLOSE_WAIT status
> forever -- long, long after mongrel has cleaned up.
>
> I'm not a communications guy, so if I'm doing something wrong, let me
> know.
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
--------------------
seth at subimage interactive
-----
http://sublog.subimage.com
-----
Cashboard - Estimates, invoices, and time tracking software - for free!
http://www.getcashboard.com
-----
Substruct - Open source RoR e-commerce software.
http://dev.subimage.com/projects/substruct
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080125/d8be0891/attachment.html 

From gethemant at gmail.com  Sat Jan 26 00:34:16 2008
From: gethemant at gmail.com (hemant)
Date: Sat, 26 Jan 2008 11:04:16 +0530
Subject: [Backgroundrb-devel] Connections
In-Reply-To: <313751190801252056t61502487ofd270ba5888bca3@mail.gmail.com>
References: <313751190801252056t61502487ofd270ba5888bca3@mail.gmail.com>
Message-ID: <fb283d5e0801252134j14fa073egc22b0f6c8ec15803@mail.gmail.com>

On Sat, Jan 26, 2008 at 10:26 AM, Meech <meechman at gmail.com> wrote:
> I'm checking out backgroundrb for a calculation process I estimate will take
> 2-3 minutes and be ran on-demand only via Rails.  Very easy and slick.
>
> But I have a question.
>
> The connections never close, if I do a netstat they just keep piling up.
>
> Do I need to do some housekeeping somewhere?   I noticed the
> @connection.close statements are commented out in favor of a # FIXME type
> deal.
>
> Even if I store read_from_bdrb in an intermediate variable, call
> @connection.close then return, they still sit in a CLOSE_WAIT status forever
> -- long, long after mongrel has cleaned up.

That shouldn't be a problem, although initially @connection.close
wasn't invoked to circumvent a bug with socket write, but I perhaps
understand your concern ( running out of open file descriptor? ), file
a ticket, if possible here:

http://backgroundrb.devjavu.com/

I will fix this in next release.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Sat Jan 26 00:35:09 2008
From: gethemant at gmail.com (hemant)
Date: Sat, 26 Jan 2008 11:05:09 +0530
Subject: [Backgroundrb-devel] Can schedules START a worker?
In-Reply-To: <fb283d5e0801251519sa44c083o38006e050517776e@mail.gmail.com>
References: <20080125224522.0395F1858646@rubyforge.org>
	<fb283d5e0801251519sa44c083o38006e050517776e@mail.gmail.com>
Message-ID: <fb283d5e0801252135jf97c33eo216847853c56bfa0@mail.gmail.com>

On Sat, Jan 26, 2008 at 4:49 AM, hemant <gethemant at gmail.com> wrote:
> Hi Scott,
>
>
>  On Jan 26, 2008 4:02 AM, Scott Ward <scott at shefield.com> wrote:
>  > Hi there,
>  >
>  > I'm just working on getting backgroundrb (1.0.1) up and running.  I'm no
>  > expert with ruby, etc, so I still don't quite get a few things.
>  >
>  > I have scheduled workers that run every few minutes.  That works fine.  I
>  > have other workers that run once per day or even once per week.  It doesn't
>  > make sense to have them running all the time.  I'd rather they just fire up
>  > when the time comes and shut down again.
>  >
>  > I thought I could "set_auto_load true" and that they would be started up
>  > when the schedule dictated.  Then I'd use "exit" at the end to stop them
>  > again.
>  >
>  > But apparently, the schedules only work on running workers.
>  >
>  > What am I missing?  What's the best way to handle workers that only need to
>  > take action once per day and therefore don't need to be running all the
>  > time?
>
>  Currently you can't. You will have to let the worker running, but if
>  you can bear with us for few weeks, I am working on getting a new
>  release of bdrb out of door and it will have this feature.

Scott, can you file a ticket regarding this, on :

http://backgroundrb.devjavu.com/

?/

From meechman at gmail.com  Sat Jan 26 09:55:26 2008
From: meechman at gmail.com (Meech)
Date: Sat, 26 Jan 2008 09:55:26 -0500
Subject: [Backgroundrb-devel] Connections
In-Reply-To: <fb283d5e0801252134j14fa073egc22b0f6c8ec15803@mail.gmail.com>
References: <313751190801252056t61502487ofd270ba5888bca3@mail.gmail.com>
	<fb283d5e0801252134j14fa073egc22b0f6c8ec15803@mail.gmail.com>
Message-ID: <313751190801260655m7e7e129cy558e2e83499caf97@mail.gmail.com>

SI,

I thought you did not _have_ to exit your workers, that they could just hang
out and process requests.   And most of my stale, open connections are the
result of ask_status calls,  not sure these hit (or need) a worker?    I'll
try it though.


Hemant,

Thanks.  I haven't really run into any barriers yet, I'm just used to
watching netstat to make sure all the various pieces and parts are awake and
listening.    My work rig is Windows, and I find myself checking netstat a
lot because backgroundrb (under cygwin) starts successfully (or rather
without any errors) but doesn't always really start.    Don't have this
problem on my home 'nix box.

My application dynamically shows and hides links, actions, etc and my 1st
revision polled MiddleMan directly everytime -- rather than caching it in an
@instance variable.   So for each request MiddleMan.ask_status could get hit
10-15 times depending on records, auth, etc.

After refreshing the page a few times, I see 50+ connections just hanging
around and that's just with me.   I'm just wondering what this will be like
when it hits production.

I'll put in a ticket...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080126/132596b0/attachment-0001.html 

From thomas.a.wood at uconn.edu  Sat Jan 26 10:24:05 2008
From: thomas.a.wood at uconn.edu (Tom Wood)
Date: Sat, 26 Jan 2008 10:24:05 -0500
Subject: [Backgroundrb-devel] thanks,
	and a question about week day scheduling
Message-ID: <8C81AA7D3B12F4408C6B3359AEB001CC04037601@LIB-EMarks.library.lib.uconn.edu>

I'm a big "fan" of BackgroundRB ... 1.0 came along just when my project
needed it, and I've been very happy. Thanks much for this very helpful
piece of software.

 

My question:  I have a worker scheduled cron-style as follows:

 

:notification_worker:

    :notify:

      :trigger_args: 0 0 8 * * 1,3,5 *

 

I.e., I want to run this particular worker method at 8:00a on
Monday/Wednesday/Friday.

 

According to the logs, however, this worker ran at 8:00a this morning,
which happens to be Saturday!  I have checked the system date, and it is
correct.

 

Am I misinterpreting the weekday numbering convention (0 = Sun, 1 = Mon,
2 = Tue, etc., like cron), or did I find a bug?

 

BTW, this is not the only worker in the system.

 

Thanks.

 

-Tom W.

 

Tom Wood

thomas.a.wood at uconn.edu <mailto:thomas.a.wood at uconn.edu> 

ITS Applications Developer

University of Connecticut Libraries

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080126/fa303fb3/attachment.html 

From gethemant at gmail.com  Sat Jan 26 21:03:19 2008
From: gethemant at gmail.com (hemant)
Date: Sun, 27 Jan 2008 07:33:19 +0530
Subject: [Backgroundrb-devel] Connections
In-Reply-To: <313751190801260655m7e7e129cy558e2e83499caf97@mail.gmail.com>
References: <313751190801252056t61502487ofd270ba5888bca3@mail.gmail.com>
	<fb283d5e0801252134j14fa073egc22b0f6c8ec15803@mail.gmail.com>
	<313751190801260655m7e7e129cy558e2e83499caf97@mail.gmail.com>
Message-ID: <fb283d5e0801261803q31692eeardccd602b8b5fd73d@mail.gmail.com>

On Sat, Jan 26, 2008 at 8:25 PM, Meech <meechman at gmail.com> wrote:
> SI,
>
> I thought you did not _have_ to exit your workers, that they could just hang
> out and process requests.   And most of my stale, open connections are the
> result of ask_status calls,  not sure these hit (or need) a worker?    I'll
> try it though.
>
>
> Hemant,
>
> Thanks.  I haven't really run into any barriers yet, I'm just used to
> watching netstat to make sure all the various pieces and parts are awake and
> listening.    My work rig is Windows, and I find myself checking netstat a
> lot because backgroundrb (under cygwin) starts successfully (or rather
> without any errors) but doesn't always really start.    Don't have this
> problem on my home 'nix box.
>
> My application dynamically shows and hides links, actions, etc and my 1st
> revision polled MiddleMan directly everytime -- rather than caching it in an
> @instance variable.   So for each request MiddleMan.ask_status could get hit
> 10-15 times depending on records, auth, etc.
>
> After refreshing the page a few times, I see 50+ connections just hanging
> around and that's just with me.   I'm just wondering what this will be like
> when it hits production.
>
> I'll put in a ticket...

Don't worry about "what happens in production in issue?". I have
rarely let any known/reproducible bug open for more than a week. So,
we will this straightened.

From gethemant at gmail.com  Sat Jan 26 21:37:03 2008
From: gethemant at gmail.com (hemant)
Date: Sun, 27 Jan 2008 08:07:03 +0530
Subject: [Backgroundrb-devel] thanks,
	and a question about week day scheduling
In-Reply-To: <8C81AA7D3B12F4408C6B3359AEB001CC04037601@LIB-EMarks.library.lib.uconn.edu>
References: <8C81AA7D3B12F4408C6B3359AEB001CC04037601@LIB-EMarks.library.lib.uconn.edu>
Message-ID: <fb283d5e0801261837n1c45bdb7qbf6a6984a9113b04@mail.gmail.com>

On Sat, Jan 26, 2008 at 8:54 PM, Tom Wood <thomas.a.wood at uconn.edu> wrote:
>
>
>
>
> I'm a big "fan" of BackgroundRB ? 1.0 came along just when my project needed
> it, and I've been very happy. Thanks much for this very helpful piece of
> software.
>
>
>
> My question:  I have a worker scheduled cron-style as follows:
>
>
>
> :notification_worker:
>
>     :notify:
>
>       :trigger_args: 0 0 8 * * 1,3,5 *
>
> I.e., I want to run this particular worker method at 8:00a on
> Monday/Wednesday/Friday.
>
> According to the logs, however, this worker ran at 8:00a this morning, which
> happens to be Saturday!  I have checked the system date, and it is correct.
>
> Am I misinterpreting the weekday numbering convention (0 = Sun, 1 = Mon, 2 =
> Tue, etc., like cron), or did I find a bug?

There was a fix done in 1.0.1 and trunk for week day, are you using
these versions? If not, upgrade.




-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From scott at shefield.com  Mon Jan 28 14:32:21 2008
From: scott at shefield.com (Scott Ward)
Date: Mon, 28 Jan 2008 11:32:21 -0800
Subject: [Backgroundrb-devel] Scheduling same worker/method at different
	times with different args
Message-ID: <20080128193112.8609C185866D@rubyforge.org>

I need to run the same worker's method twice per day with different
arguments.  Unfortunately, only the second entry in the schedule is firing.

I created an experimental worker to verify this:

Worker:
class ExperimentWorker < BackgrounDRb::MetaWorker
  set_worker_name :experiment_worker
  def create(args = nil)
    # this method is called, when worker is loaded for the first time
  end
  
  def experiment(args = {})
    logger.info "#{Time.now} - Experiment fired at #{args[:repeat_second]}
second mark."
  end
end


In backgroundrb.yml:
:schedules:
  :experiment_worker:
    :experiment:
      :trigger_args: "0 * * * * * *"
      :data:
        :repeat_second: zero
    :experiment:
      :trigger_args: "30 * * * * * *"
      :data:
        :repeat_second: thirty


In the log, I only see "Experiment fired at thirty second mark."  Why isn't
the first entry for experiment worker firing?  Perhaps, I have this set up
wrong?

Thanks,

Scott


From gethemant at gmail.com  Mon Jan 28 15:15:52 2008
From: gethemant at gmail.com (hemant)
Date: Tue, 29 Jan 2008 01:45:52 +0530
Subject: [Backgroundrb-devel] Scheduling same worker/method at different
	times with different args
In-Reply-To: <20080128193112.8609C185866D@rubyforge.org>
References: <20080128193112.8609C185866D@rubyforge.org>
Message-ID: <fb283d5e0801281215u6371183g1365813088e75d58@mail.gmail.com>

On Tue, Jan 29, 2008 at 1:02 AM, Scott Ward <scott at shefield.com> wrote:
> I need to run the same worker's method twice per day with different
>  arguments.  Unfortunately, only the second entry in the schedule is firing.
>
>  I created an experimental worker to verify this:
>
>  Worker:
>  class ExperimentWorker < BackgrounDRb::MetaWorker
>   set_worker_name :experiment_worker
>   def create(args = nil)
>     # this method is called, when worker is loaded for the first time
>   end
>
>   def experiment(args = {})
>     logger.info "#{Time.now} - Experiment fired at #{args[:repeat_second]}
>  second mark."
>   end
>  end
>
>
>  In backgroundrb.yml:
>  :schedules:
>   :experiment_worker:
>     :experiment:
>       :trigger_args: "0 * * * * * *"
>       :data:
>         :repeat_second: zero
>     :experiment:
>       :trigger_args: "30 * * * * * *"
>       :data:
>         :repeat_second: thirty
>
>
>  In the log, I only see "Experiment fired at thirty second mark."  Why isn't
>  the first entry for experiment worker firing?  Perhaps, I have this set up
>  wrong?


Because it needs different method names for such a firing. As you
might have noticed, currently you have two values with same key in
configuration file.
-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From andre at myavalaunch.com  Mon Jan 28 19:13:22 2008
From: andre at myavalaunch.com (Andre Borrelly)
Date: Mon, 28 Jan 2008 17:13:22 -0700
Subject: [Backgroundrb-devel] Setting up release-1.0.1
Message-ID: <d76343c4338d06e21ee507263bcc9085@localhost>



 Hello,  

 I am unable to get the backgroundrb server to run.     

 Initially I got:  


/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:263:in
`load_missing_constant': uninitialized constant
BackgrounDRb::MasterProxy (NameError)
         from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:453:in
`const_missing'
         from ./script/backgroundrb:41
 Then after doing this:  

 Index: script/backgroundrb
 ===================================================================
 --- script/backgroundrb (revision 313)
 +++ script/backgroundrb (working copy)
 @@ -6,8 +6,10 @@
  WORKER_ROOT = rails_root + "/lib/workers"
  SERVER_LOGGER = rails_root + "/log/backgroundrb_server.log"
 -["server","framework","lib"].each { |x|
$LOAD_PATH.unshift(PACKET_APP + "/#{x}")}
 -$LOAD_PATH.unshift(WORKER_ROOT)
 +#["server","framework","lib"].each { |x|
$LOAD_PATH.unshift(PACKET_APP + "/#{x}")}
 +["server","framework","lib"].each { |x| $:.push(PACKET_APP +
"/#{x}")}
 +#$LOAD_PATH.unshift(WORKER_ROOT)
 +$:.push(WORKER_ROOT)
  require RAILS_HOME + '/config/boot.rb'
  require "active_record"
 and adding in "environment.rb"  

   config.load_paths +=
Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/server"]
   config.load_paths +=
Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/framework"]
 I get this:  

  
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:249:in
`load_missing_constant': Expected
RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb to
define MasterWorker (LoadError)
   from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:453:in
`const_missing'
   from
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:465:in
`const_missing'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:96:in
`const_get'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:96:in
`load_workers'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:93:in
`each'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:93:in
`load_workers'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in
`run'
   from
RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:163:in
`initialize'
   from ./script/backgroundrb:62:in `new'
   from ./script/backgroundrb:62

RAILS_HOME/vendor/plugins/backgroundrb/lib/../framework/nbio.rb:23:in
`read_data': Packet::DisconnectError (Packet::DisconnectError)
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/worker.rb:47:in
`handle_internal_messages'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:158:in
`start_reactor'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:156:in
`each'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:156:in
`start_reactor'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:147:in
`loop'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:147:in
`start_reactor'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/worker.rb:21:in
`start_worker'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:134:in
`fork_and_load'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:115:in
`start_worker'
   from
RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:165:in
`initialize'
   from
RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:18:in
`run'
   from
RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:163:in
`initialize'
   from ./script/backgroundrb:62:in `new'
   from ./script/backgroundrb:62  

 Thanks,  

 Andre  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080128/54430c98/attachment.html 

From normelton at gmail.com  Mon Jan 28 20:59:00 2008
From: normelton at gmail.com (Norman Elton)
Date: Mon, 28 Jan 2008 20:59:00 -0500
Subject: [Backgroundrb-devel] Authoritative Documentation
Message-ID: <9DD5416C-5A15-4D35-849B-4692A9570683@gmail.com>

I've been using backgroundrb for... about an hour now. I must say, I'm  
impressed.

But I'm a bit stumped on where to find the "official" documentation.  
There's an API online at backgroundrb.rubyforge.org, but I don't see  
the methods available for MiddleMan, etc.

Right now, I'm trying to figure out how to get a list of all running  
workers. Poking through the source code, I see a function entitled  
"get_worker_by_class", but there is little to no documentation  
available. The documentation does refer to query_all_workers, but I  
can't find it anywhere in the source code.

I understand that cool technologies often race ahead of the  
documentation writers. But am I misunderstanding something? Perhaps  
I'm looking in the wrong place?

Thanks,

Norman Elton

From gethemant at gmail.com  Mon Jan 28 21:12:32 2008
From: gethemant at gmail.com (hemant)
Date: Tue, 29 Jan 2008 07:42:32 +0530
Subject: [Backgroundrb-devel] Authoritative Documentation
In-Reply-To: <9DD5416C-5A15-4D35-849B-4692A9570683@gmail.com>
References: <9DD5416C-5A15-4D35-849B-4692A9570683@gmail.com>
Message-ID: <fb283d5e0801281812x7c66368fw88a4dc43cfc938f8@mail.gmail.com>

On Tue, Jan 29, 2008 at 7:29 AM, Norman Elton <normelton at gmail.com> wrote:
> I've been using backgroundrb for... about an hour now. I must say, I'm
>  impressed.
>
>  But I'm a bit stumped on where to find the "official" documentation.
>  There's an API online at backgroundrb.rubyforge.org, but I don't see
>  the methods available for MiddleMan, etc.
>
>  Right now, I'm trying to figure out how to get a list of all running
>  workers. Poking through the source code, I see a function entitled
>  "get_worker_by_class", but there is little to no documentation
>  available. The documentation does refer to query_all_workers, but I
>  can't find it anywhere in the source code.
>
>  I understand that cool technologies often race ahead of the
>  documentation writers. But am I misunderstanding something? Perhaps
>  I'm looking in the wrong place?

I am quite aware of Documentation problem and I intend to fix it soon
( any help is more than welcome! )

What you are looking for is: MiddleMan.all_worker_info()



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Mon Jan 28 21:15:49 2008
From: gethemant at gmail.com (hemant)
Date: Tue, 29 Jan 2008 07:45:49 +0530
Subject: [Backgroundrb-devel] Setting up release-1.0.1
In-Reply-To: <d76343c4338d06e21ee507263bcc9085@localhost>
References: <d76343c4338d06e21ee507263bcc9085@localhost>
Message-ID: <fb283d5e0801281815q15205debyd4fa6ca96c1cc44d@mail.gmail.com>

On Tue, Jan 29, 2008 at 5:43 AM, Andre Borrelly <andre at myavalaunch.com> wrote:
>
>
>  Hello,
>
>  I am unable to get the backgroundrb server to run.
>
>  Initially I got:
>
>
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:263:in
> `load_missing_constant': uninitialized constant BackgrounDRb::MasterProxy
> (NameError)
>    from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:453:in
> `const_missing'
>    from ./script/backgroundrb:41
>
>  Then after doing this:
>
>  Index: script/backgroundrb
>  ===================================================================
>  --- script/backgroundrb (revision 313)
>  +++ script/backgroundrb (working copy)
>  @@ -6,8 +6,10 @@
>   WORKER_ROOT = rails_root + "/lib/workers"
>   SERVER_LOGGER = rails_root + "/log/backgroundrb_server.log"
>
>  -["server","framework","lib"].each { |x| $LOAD_PATH.unshift(PACKET_APP +
> "/#{x}")}
>  -$LOAD_PATH.unshift(WORKER_ROOT)
>  +#["server","framework","lib"].each { |x| $LOAD_PATH.unshift(PACKET_APP +
> "/#{x}")}
>  +["server","framework","lib"].each { |x| $:.push(PACKET_APP + "/#{x}")}
>  +#$LOAD_PATH.unshift(WORKER_ROOT)
>  +$:.push(WORKER_ROOT)
>
>   require RAILS_HOME + '/config/boot.rb'
>   require "active_record"
>
>
>  and adding in "environment.rb"
>
>    config.load_paths +=
> Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/server"]
>    config.load_paths +=
> Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/framework"]
>
>  I get this:
>
>
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:249:in
> `load_missing_constant': Expected
> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb to define
> MasterWorker (LoadError)
>    from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:453:in
> `const_missing'
>    from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:465:in
> `const_missing'
>    from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:96:in
> `const_get'
>    from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:96:in
> `load_workers'
>    from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:93:in
> `each'
>    from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:93:in
> `load_workers'
>    from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in
> `run'
>    from
> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:163:in
> `initialize'
>    from ./script/backgroundrb:62:in `new'
>    from ./script/backgroundrb:62
>  RAILS_HOME/vendor/plugins/backgroundrb/lib/../framework/nbio.rb:23:in
> `read_data': Packet::DisconnectError (Packet::DisconnectError)
>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/worker.rb:47:in
> `handle_internal_messages'
>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:158:in
> `start_reactor'
>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:156:in
> `each'
>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:156:in
> `start_reactor'
>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:147:in
> `loop'
>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:147:in
> `start_reactor'
>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/worker.rb:21:in
> `start_worker'
>    from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:134:in
> `fork_and_load'
>    from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:115:in
> `start_worker'
>    from
> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:165:in
> `initialize'
>    from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:18:in
> `run'
>    from
> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:163:in
> `initialize'
>    from ./script/backgroundrb:62:in `new'
>    from ./script/backgroundrb:62
>

Did you actually delete your old, script/backgroundrb file and ran

rake backgroundrb:setup

Often you get start problems with new releases, because your plugin is
upgraded, but start/stop script is not.
Which OS you are on?

From mengkuan at gmail.com  Mon Jan 28 21:34:00 2008
From: mengkuan at gmail.com (Meng Kuan)
Date: Tue, 29 Jan 2008 10:34:00 +0800
Subject: [Backgroundrb-devel] Scheduling same worker/method at different
	times with different args
In-Reply-To: <fb283d5e0801281215u6371183g1365813088e75d58@mail.gmail.com>
References: <20080128193112.8609C185866D@rubyforge.org>
	<fb283d5e0801281215u6371183g1365813088e75d58@mail.gmail.com>
Message-ID: <9771E750-7B6C-4FE6-86AD-2AAD27E2888B@gmail.com>


On 29 Jan 2008, at 4:15 AM, hemant wrote:

> On Tue, Jan 29, 2008 at 1:02 AM, Scott Ward <scott at shefield.com>  
> wrote:
>> I need to run the same worker's method twice per day with different
>>  arguments.  Unfortunately, only the second entry in the schedule  
>> is firing.
>>
>>  I created an experimental worker to verify this:
>>
>>  Worker:
>>  class ExperimentWorker < BackgrounDRb::MetaWorker
>>   set_worker_name :experiment_worker
>>   def create(args = nil)
>>     # this method is called, when worker is loaded for the first time
>>   end
>>
>>   def experiment(args = {})
>>     logger.info "#{Time.now} - Experiment fired at #{args 
>> [:repeat_second]}
>>  second mark."
>>   end
>>  end
>>
>>
>>  In backgroundrb.yml:
>>  :schedules:
>>   :experiment_worker:
>>     :experiment:
>>       :trigger_args: "0 * * * * * *"
>>       :data:
>>         :repeat_second: zero
>>     :experiment:
>>       :trigger_args: "30 * * * * * *"
>>       :data:
>>         :repeat_second: thirty
>>
>>
>>  In the log, I only see "Experiment fired at thirty second mark."   
>> Why isn't
>>  the first entry for experiment worker firing?  Perhaps, I have  
>> this set up
>>  wrong?
>
>
> Because it needs different method names for such a firing. As you
> might have noticed, currently you have two values with same key in
> configuration file.
>

You can try using alias_method to get a different method name:

   alias_method :experiment2, :experiment

Note: the alias_method call should come _after_ the actual  
"experiment" method definition.


Then your backgroundrb.yml becomes:

  :schedules:
   :experiment_worker:
     :experiment:
       :trigger_args: "0 * * * * * *"
       :data:
         :repeat_second: zero
     :experiment2:
       :trigger_args: "30 * * * * * *"
       :data:
         :repeat_second: thirty



From andre at myavalaunch.com  Mon Jan 28 21:40:33 2008
From: andre at myavalaunch.com (andre)
Date: Mon, 28 Jan 2008 18:40:33 -0800
Subject: [Backgroundrb-devel] Setting up release-1.0.1
In-Reply-To: <fb283d5e0801281815q15205debyd4fa6ca96c1cc44d@mail.gmail.com>
References: <d76343c4338d06e21ee507263bcc9085@localhost>
	<fb283d5e0801281815q15205debyd4fa6ca96c1cc44d@mail.gmail.com>
Message-ID: <479E9221.3090409@myavalaunch.com>

hemant wrote:
> On Tue, Jan 29, 2008 at 5:43 AM, Andre Borrelly <andre at myavalaunch.com> wrote:
>   
>>  Hello,
>>
>>  I am unable to get the backgroundrb server to run.
>>
>>  Initially I got:
>>
>>
>> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:263:in
>> `load_missing_constant': uninitialized constant BackgrounDRb::MasterProxy
>> (NameError)
>>    from
>> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:453:in
>> `const_missing'
>>    from ./script/backgroundrb:41
>>
>>  Then after doing this:
>>
>>  Index: script/backgroundrb
>>  ===================================================================
>>  --- script/backgroundrb (revision 313)
>>  +++ script/backgroundrb (working copy)
>>  @@ -6,8 +6,10 @@
>>   WORKER_ROOT = rails_root + "/lib/workers"
>>   SERVER_LOGGER = rails_root + "/log/backgroundrb_server.log"
>>
>>  -["server","framework","lib"].each { |x| $LOAD_PATH.unshift(PACKET_APP +
>> "/#{x}")}
>>  -$LOAD_PATH.unshift(WORKER_ROOT)
>>  +#["server","framework","lib"].each { |x| $LOAD_PATH.unshift(PACKET_APP +
>> "/#{x}")}
>>  +["server","framework","lib"].each { |x| $:.push(PACKET_APP + "/#{x}")}
>>  +#$LOAD_PATH.unshift(WORKER_ROOT)
>>  +$:.push(WORKER_ROOT)
>>
>>   require RAILS_HOME + '/config/boot.rb'
>>   require "active_record"
>>
>>
>>  and adding in "environment.rb"
>>
>>    config.load_paths +=
>> Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/server"]
>>    config.load_paths +=
>> Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/framework"]
>>
>>  I get this:
>>
>>
>> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:249:in
>> `load_missing_constant': Expected
>> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb to define
>> MasterWorker (LoadError)
>>    from
>> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:453:in
>> `const_missing'
>>    from
>> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:465:in
>> `const_missing'
>>    from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:96:in
>> `const_get'
>>    from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:96:in
>> `load_workers'
>>    from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:93:in
>> `each'
>>    from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:93:in
>> `load_workers'
>>    from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in
>> `run'
>>    from
>> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:163:in
>> `initialize'
>>    from ./script/backgroundrb:62:in `new'
>>    from ./script/backgroundrb:62
>>  RAILS_HOME/vendor/plugins/backgroundrb/lib/../framework/nbio.rb:23:in
>> `read_data': Packet::DisconnectError (Packet::DisconnectError)
>>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/worker.rb:47:in
>> `handle_internal_messages'
>>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:158:in
>> `start_reactor'
>>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:156:in
>> `each'
>>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:156:in
>> `start_reactor'
>>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:147:in
>> `loop'
>>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:147:in
>> `start_reactor'
>>    from RAILS_HOME/vendor/plugins/backgroundrb/framework/worker.rb:21:in
>> `start_worker'
>>    from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:134:in
>> `fork_and_load'
>>    from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:115:in
>> `start_worker'
>>    from
>> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:165:in
>> `initialize'
>>    from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:18:in
>> `run'
>>    from
>> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:163:in
>> `initialize'
>>    from ./script/backgroundrb:62:in `new'
>>    from ./script/backgroundrb:62
>>
>>     
>
> Did you actually delete your old, script/backgroundrb file and ran
>
> rake backgroundrb:setup
>
> Often you get start problems with new releases, because your plugin is
> upgraded, but start/stop script is not.
> Which OS you are on?
>   
Yes I have just followed those steps, but still no go.
I get:
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:266:in 
`load_missing_constant': uninitialized constant MasterWorker (NameError)

If I add this in the environment.rb
  config.load_paths += 
Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/server"]
  config.load_paths += 
Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/framework"]
I get
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:249:in 
`load_missing_constant': Expected 
RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb to define 
MasterWorker (LoadError)


I'm running:
 Kubuntu 7.10: the Gutsy Gibbon Release.
 ruby 1.8.6 (2007-03-13 patchlevel 0) [i686-linux]
actionmailer (2.0.2, 2.0.1, 1.3.5, 1.3.3)
actionpack (2.0.2, 2.0.1, 1.13.6, 1.13.5, 1.13.3)
actionwebservice (1.2.6, 1.2.5, 1.2.3)
activeldap (0.9.0)
activerecord (2.0.2, 2.0.1, 1.15.6, 1.15.5, 1.15.3)
activeresource (2.0.2, 2.0.1)
activesupport (2.0.2, 2.0.1, 1.4.4, 1.4.2)
bleak_house (3.7, 3.5.1, 3.2)
ccsv (0.1, 0.0.2)
cgi_multipart_eof_fix (2.5.0, 2.3)
daemons (1.0.9, 1.0.8)
fastthread (1.0.1, 1.0)
gem_plugin (0.2.3, 0.2.2)
hoe (1.4.0, 1.3.0)
httpclient (2.1.2)
log4r (1.0.5)
mailtrap (0.2.1)
memcache-client (1.5.0)
mongrel (1.1.3, 1.1.1, 1.0.1)
mysql (2.7)
rails (2.0.2, 2.0.1, 1.2.5, 1.2.3)
rake (0.8.1, 0.7.3)
rcov (0.8.1.2.0, 0.8.0.2)
RedCloth (3.0.4)
rspec (1.1.2, 1.0.8)
ruby-debug-base (0.10.0, 0.9.3)
ruby-debug-ide (0.1.9, 0.1.8)
rubyforge (0.4.4)
rubygems-update (1.0.1, 0.9.5, 0.9.4)
sources (0.0.1)
transaction-simple (1.4.0)
trollop (1.7.2, 1.7)
tzinfo (0.3.6, 0.3.5, 0.3.4)
ZenTest (3.8.0, 3.7.2)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080128/f8f1d0ff/attachment.html 

From gethemant at gmail.com  Mon Jan 28 22:50:33 2008
From: gethemant at gmail.com (hemant)
Date: Tue, 29 Jan 2008 09:20:33 +0530
Subject: [Backgroundrb-devel] Scheduling same worker/method at different
	times with different args
In-Reply-To: <9771E750-7B6C-4FE6-86AD-2AAD27E2888B@gmail.com>
References: <20080128193112.8609C185866D@rubyforge.org>
	<fb283d5e0801281215u6371183g1365813088e75d58@mail.gmail.com>
	<9771E750-7B6C-4FE6-86AD-2AAD27E2888B@gmail.com>
Message-ID: <fb283d5e0801281950i6aab524ap48d2f59f2119280b@mail.gmail.com>

On Tue, Jan 29, 2008 at 8:04 AM, Meng Kuan <mengkuan at gmail.com> wrote:
>
>
>  On 29 Jan 2008, at 4:15 AM, hemant wrote:
>
>  > On Tue, Jan 29, 2008 at 1:02 AM, Scott Ward <scott at shefield.com>
>  > wrote:
>  >> I need to run the same worker's method twice per day with different
>  >>  arguments.  Unfortunately, only the second entry in the schedule
>  >> is firing.
>  >>
>  >>  I created an experimental worker to verify this:
>  >>
>  >>  Worker:
>  >>  class ExperimentWorker < BackgrounDRb::MetaWorker
>  >>   set_worker_name :experiment_worker
>  >>   def create(args = nil)
>  >>     # this method is called, when worker is loaded for the first time
>  >>   end
>  >>
>  >>   def experiment(args = {})
>  >>     logger.info "#{Time.now} - Experiment fired at #{args
>  >> [:repeat_second]}
>  >>  second mark."
>  >>   end
>  >>  end
>  >>
>  >>
>  >>  In backgroundrb.yml:
>  >>  :schedules:
>  >>   :experiment_worker:
>  >>     :experiment:
>  >>       :trigger_args: "0 * * * * * *"
>  >>       :data:
>  >>         :repeat_second: zero
>  >>     :experiment:
>  >>       :trigger_args: "30 * * * * * *"
>  >>       :data:
>  >>         :repeat_second: thirty
>  >>
>  >>
>  >>  In the log, I only see "Experiment fired at thirty second mark."
>  >> Why isn't
>  >>  the first entry for experiment worker firing?  Perhaps, I have
>  >> this set up
>  >>  wrong?
>  >
>  >
>  > Because it needs different method names for such a firing. As you
>  > might have noticed, currently you have two values with same key in
>  > configuration file.
>  >
>
>  You can try using alias_method to get a different method name:
>
>    alias_method :experiment2, :experiment
>
>  Note: the alias_method call should come _after_ the actual
>  "experiment" method definition.
>
>
>  Then your backgroundrb.yml becomes:
>
>
>   :schedules:
>    :experiment_worker:
>      :experiment:
>        :trigger_args: "0 * * * * * *"
>        :data:
>          :repeat_second: zero
>      :experiment2:
>
>        :trigger_args: "30 * * * * * *"
>        :data:
>          :repeat_second: thirty
>

There is a patch by Andy, in trac for enabling this kind of behaviour,
we can think of applying that.

From gethemant at gmail.com  Mon Jan 28 23:05:26 2008
From: gethemant at gmail.com (hemant)
Date: Tue, 29 Jan 2008 09:35:26 +0530
Subject: [Backgroundrb-devel] Setting up release-1.0.1
In-Reply-To: <479E9221.3090409@myavalaunch.com>
References: <d76343c4338d06e21ee507263bcc9085@localhost>
	<fb283d5e0801281815q15205debyd4fa6ca96c1cc44d@mail.gmail.com>
	<479E9221.3090409@myavalaunch.com>
Message-ID: <fb283d5e0801282005n4ae9e755sba7f4fe1f6741f85@mail.gmail.com>

On Tue, Jan 29, 2008 at 8:10 AM, andre <andre at myavalaunch.com> wrote:
>
>  hemant wrote:
>
>  On Tue, Jan 29, 2008 at 5:43 AM, Andre Borrelly <andre at myavalaunch.com>
> wrote:
>
>
>  Hello,
>
>  I am unable to get the backgroundrb server to run.
>
>  Initially I got:
>
>
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:263:in
> `load_missing_constant': uninitialized constant BackgrounDRb::MasterProxy
> (NameError)
>  from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:453:in
> `const_missing'
>  from ./script/backgroundrb:41
>
>  Then after doing this:
>
>  Index: script/backgroundrb
>  ===================================================================
>  --- script/backgroundrb (revision 313)
>  +++ script/backgroundrb (working copy)
>  @@ -6,8 +6,10 @@
>  WORKER_ROOT = rails_root + "/lib/workers"
>  SERVER_LOGGER = rails_root + "/log/backgroundrb_server.log"
>
>  -["server","framework","lib"].each { |x| $LOAD_PATH.unshift(PACKET_APP +
> "/#{x}")}
>  -$LOAD_PATH.unshift(WORKER_ROOT)
>  +#["server","framework","lib"].each { |x| $LOAD_PATH.unshift(PACKET_APP +
> "/#{x}")}
>  +["server","framework","lib"].each { |x| $:.push(PACKET_APP + "/#{x}")}
>  +#$LOAD_PATH.unshift(WORKER_ROOT)
>  +$:.push(WORKER_ROOT)
>
>  require RAILS_HOME + '/config/boot.rb'
>  require "active_record"
>
>
>  and adding in "environment.rb"
>
>  config.load_paths +=
> Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/server"]
>  config.load_paths +=
> Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/framework"]
>
>  I get this:
>
>
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:249:in
> `load_missing_constant': Expected
> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb to define
> MasterWorker (LoadError)
>  from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:453:in
> `const_missing'
>  from
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:465:in
> `const_missing'
>  from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:96:in
> `const_get'
>  from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:96:in
> `load_workers'
>  from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:93:in
> `each'
>  from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:93:in
> `load_workers'
>  from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in
> `run'
>  from
> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:163:in
> `initialize'
>  from ./script/backgroundrb:62:in `new'
>  from ./script/backgroundrb:62
>  RAILS_HOME/vendor/plugins/backgroundrb/lib/../framework/nbio.rb:23:in
> `read_data': Packet::DisconnectError (Packet::DisconnectError)
>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/worker.rb:47:in
> `handle_internal_messages'
>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:158:in
> `start_reactor'
>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:156:in
> `each'
>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:156:in
> `start_reactor'
>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:147:in
> `loop'
>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:147:in
> `start_reactor'
>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/worker.rb:21:in
> `start_worker'
>  from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:134:in
> `fork_and_load'
>  from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:115:in
> `start_worker'
>  from
> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:165:in
> `initialize'
>  from
> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:18:in
> `run'
>  from
> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:163:in
> `initialize'
>  from ./script/backgroundrb:62:in `new'
>  from ./script/backgroundrb:62
>
>
>  Did you actually delete your old, script/backgroundrb file and ran
>
> rake backgroundrb:setup
>
> Often you get start problems with new releases, because your plugin is
> upgraded, but start/stop script is not.
> Which OS you are on?
>
>  Yes I have just followed those steps, but still no go.
>  I get:
>
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:266:in
> `load_missing_constant': uninitialized constant MasterWorker (NameError)
>
>  If I add this in the environment.rb
>
>    config.load_paths +=
> Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/server"]
>    config.load_paths +=
> Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/framework"]
>  I get
>
> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:249:in
> `load_missing_constant': Expected
> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb to define
> MasterWorker (LoadError)

I am unable to reproduce this at my end. Can you try creating a fresh
rails app and check out the plugin. Send me the tarball of your sample
app, if all else fails.
-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From normelton at gmail.com  Tue Jan 29 11:34:06 2008
From: normelton at gmail.com (Norman Elton)
Date: Tue, 29 Jan 2008 11:34:06 -0500
Subject: [Backgroundrb-devel] Authoritative Documentation
In-Reply-To: <fb283d5e0801281812x7c66368fw88a4dc43cfc938f8@mail.gmail.com>
References: <9DD5416C-5A15-4D35-849B-4692A9570683@gmail.com>
	<fb283d5e0801281812x7c66368fw88a4dc43cfc938f8@mail.gmail.com>
Message-ID: <BDE5B727-B540-416F-A14A-160A7DEB8B41@gmail.com>

>
> I am quite aware of Documentation problem and I intend to fix it soon
> ( any help is more than welcome! )
>
> What you are looking for is: MiddleMan.all_worker_info()
>

Thanks for pointing me there. I also realized that part of my problem  
was that I was not running the latest 1.0.1 code. Now that things  
appear to be upgraded, I'm moving along. It's great to have an active  
mailing list to consult when I hit a roadblock.

I've created a very simple worker, much like the FooWorker outlined in  
the documentation. The create() method registers a status, and I'm  
trying to use send_request() to run another method on the worker. When  
I try to do this via ./script/console, I'm getting the following error:

# Logfile created on Tue Jan 29 05:10:16 -0500 2008 by /
00000008{	:
run_dump:       type:get_resulter_worker:	data"	1234:worker_method:
{:worker 
= 
 > 
:nf_server_worker 
, :data=>"1234", :type=>:get_result, :worker_method=>:run_dump}
You have a nil object when you didn't expect it!
The error occurred while evaluating nil.send_request
/home/normelton/nfsensei/vendor/plugins/backgroundrb/framework/packet/ 
packet_master.rb:43:in `ask_worker'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/server/ 
master_worker.rb:130:in `process_request'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/server/ 
master_worker.rb:39:in `receive_data'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/lib/../framework/ 
packet/bin_parser.rb:29:in `call'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/lib/../framework/ 
packet/bin_parser.rb:29:in `extract'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/server/ 
master_worker.rb:33:in `receive_data'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/framework/packet/ 
core.rb:199:in `read_external_socket'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/framework/packet/ 
core.rb:191:in `handle_external_messages'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/framework/packet/ 
core.rb:160:in `start_reactor'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/framework/packet/ 
core.rb:156:in `each'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/framework/packet/ 
core.rb:156:in `start_reactor'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/framework/packet/ 
core.rb:147:in `loop'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/framework/packet/ 
core.rb:147:in `start_reactor'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/framework/packet/ 
packet_master.rb:20:in `run'
/home/normelton/nfsensei/vendor/plugins/backgroundrb/server/ 
master_worker.rb:163:in `initialize'
./script/backgroundrb:41:in `new'
./script/backgroundrb:41

I've pasted my worker online:

http://pastie.caboo.se/144892

Any thoughts?

Thanks again for your help,

Norman Elton

From scott at shefield.com  Tue Jan 29 11:52:00 2008
From: scott at shefield.com (Scott Ward)
Date: Tue, 29 Jan 2008 08:52:00 -0800
Subject: [Backgroundrb-devel] Scheduling same worker/method at
	differenttimes with different args
In-Reply-To: <9771E750-7B6C-4FE6-86AD-2AAD27E2888B@gmail.com>
Message-ID: <20080129165059.C15A21858691@rubyforge.org>

Cool.  That worked like a charm. 

-----Original Message-----
From: backgroundrb-devel-bounces at rubyforge.org
[mailto:backgroundrb-devel-bounces at rubyforge.org] On Behalf Of Meng Kuan
Sent: Monday, January 28, 2008 6:34 PM
To: backgroundrb-devel at rubyforge.org
Subject: Re: [Backgroundrb-devel] Scheduling same worker/method at
differenttimes with different args


On 29 Jan 2008, at 4:15 AM, hemant wrote:

> On Tue, Jan 29, 2008 at 1:02 AM, Scott Ward <scott at shefield.com>
> wrote:
>> I need to run the same worker's method twice per day with different  
>> arguments.  Unfortunately, only the second entry in the schedule is 
>> firing.
>>
>>  I created an experimental worker to verify this:
>>
>>  Worker:
>>  class ExperimentWorker < BackgrounDRb::MetaWorker
>>   set_worker_name :experiment_worker
>>   def create(args = nil)
>>     # this method is called, when worker is loaded for the first time
>>   end
>>
>>   def experiment(args = {})
>>     logger.info "#{Time.now} - Experiment fired at #{args 
>> [:repeat_second]}  second mark."
>>   end
>>  end
>>
>>
>>  In backgroundrb.yml:
>>  :schedules:
>>   :experiment_worker:
>>     :experiment:
>>       :trigger_args: "0 * * * * * *"
>>       :data:
>>         :repeat_second: zero
>>     :experiment:
>>       :trigger_args: "30 * * * * * *"
>>       :data:
>>         :repeat_second: thirty
>>
>>
>>  In the log, I only see "Experiment fired at thirty second mark."   
>> Why isn't
>>  the first entry for experiment worker firing?  Perhaps, I have  
>> this set up
>>  wrong?
>
>
> Because it needs different method names for such a firing. As you
> might have noticed, currently you have two values with same key in
> configuration file.
>

You can try using alias_method to get a different method name:

   alias_method :experiment2, :experiment

Note: the alias_method call should come _after_ the actual  
"experiment" method definition.


Then your backgroundrb.yml becomes:

  :schedules:
   :experiment_worker:
     :experiment:
       :trigger_args: "0 * * * * * *"
       :data:
         :repeat_second: zero
     :experiment2:
       :trigger_args: "30 * * * * * *"
       :data:
         :repeat_second: thirty


_______________________________________________
Backgroundrb-devel mailing list
Backgroundrb-devel at rubyforge.org
http://rubyforge.org/mailman/listinfo/backgroundrb-devel


From normelton at gmail.com  Tue Jan 29 11:52:46 2008
From: normelton at gmail.com (Norman Elton)
Date: Tue, 29 Jan 2008 11:52:46 -0500
Subject: [Backgroundrb-devel] Authoritative Documentation
In-Reply-To: <fb283d5e0801281812x7c66368fw88a4dc43cfc938f8@mail.gmail.com>
References: <9DD5416C-5A15-4D35-849B-4692A9570683@gmail.com>
	<fb283d5e0801281812x7c66368fw88a4dc43cfc938f8@mail.gmail.com>
Message-ID: <B0049ADC-747B-473C-A89C-08503979CE21@gmail.com>

>>
>
> I am quite aware of Documentation problem and I intend to fix it soon
> ( any help is more than welcome! )
>
> What you are looking for is: MiddleMan.all_worker_info()
>

It appears that my problem came from configuring the worker not to  
start automatically. Removing that line causes everything to work fine.

Sorry!

Norman Elton

From rupert at adverplex.com  Tue Jan 29 16:19:33 2008
From: rupert at adverplex.com (Rupert Galea)
Date: Tue, 29 Jan 2008 16:19:33 -0500
Subject: [Backgroundrb-devel] Does status disappear after worker exits?
Message-ID: <F3DB94E0-350C-4CFB-89FC-18B07240F686@adverplex.com>

Hemant,

From
http://backgroundrb.rubyforge.org/

"Currently, when a worker is deleted/exits, its result/status is also  
gone (i.e. you can?t query the status of a worker which is not  
running). This behaviour is expected to change in future releases."

Is this still true? I ask because I am checking for a status after a  
worker exits and the code seems to work.
Thanks
Rupert


From rupert at adverplex.com  Tue Jan 29 15:59:00 2008
From: rupert at adverplex.com (Rupert Galea)
Date: Tue, 29 Jan 2008 15:59:00 -0500
Subject: [Backgroundrb-devel] Workers without rails environment
Message-ID: <D89B9169-666A-4D0E-B0D7-713C464415E3@adverplex.com>

Hemant,
Thank you for your work on backgrounrb.

In the old version there was the option of creating workers that do  
not load the rails framework. Since I was starting these processes as  
needed I decided to make use of this so they would use less memory. Is  
there a way to do this in the new version? This is not important to me  
but just wanted to make sure I was not missing something.

Rupert

From gethemant at gmail.com  Wed Jan 30 00:19:16 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 30 Jan 2008 10:49:16 +0530
Subject: [Backgroundrb-devel] Does status disappear after worker exits?
In-Reply-To: <F3DB94E0-350C-4CFB-89FC-18B07240F686@adverplex.com>
References: <F3DB94E0-350C-4CFB-89FC-18B07240F686@adverplex.com>
Message-ID: <fb283d5e0801292119p19c0fcdj205669adb1138e05@mail.gmail.com>

On Wed, Jan 30, 2008 at 2:49 AM, Rupert Galea <rupert at adverplex.com> wrote:
> Hemant,
>
>  From
>  http://backgroundrb.rubyforge.org/
>
>  "Currently, when a worker is deleted/exits, its result/status is also
>  gone (i.e. you can't query the status of a worker which is not
>  running). This behaviour is expected to change in future releases."
>
>  Is this still true? I ask because I am checking for a status after a
>  worker exits and the code seems to work.

No that has been changed and status/result objects remain even after a
worker has died
-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From gethemant at gmail.com  Wed Jan 30 00:42:43 2008
From: gethemant at gmail.com (hemant)
Date: Wed, 30 Jan 2008 11:12:43 +0530
Subject: [Backgroundrb-devel] Workers without rails environment
In-Reply-To: <D89B9169-666A-4D0E-B0D7-713C464415E3@adverplex.com>
References: <D89B9169-666A-4D0E-B0D7-713C464415E3@adverplex.com>
Message-ID: <fb283d5e0801292142x4a9e5707q81b38d83d12b2320@mail.gmail.com>

On Wed, Jan 30, 2008 at 2:29 AM, Rupert Galea <rupert at adverplex.com> wrote:
> Hemant,
>  Thank you for your work on backgrounrb.
>
>  In the old version there was the option of creating workers that do
>  not load the rails framework. Since I was starting these processes as
>  needed I decided to make use of this so they would use less memory. Is
>  there a way to do this in the new version? This is not important to me
>  but just wanted to make sure I was not missing something.
>

Currently you can't, but you can file a ticket on trac page, and we
can get it done.

From mengkuan at gmail.com  Wed Jan 30 12:59:19 2008
From: mengkuan at gmail.com (Meng Kuan)
Date: Thu, 31 Jan 2008 01:59:19 +0800
Subject: [Backgroundrb-devel] Schedule parser
Message-ID: <D6411E1F-DC02-4347-AA1D-7C32B3358833@gmail.com>

Hi,

I was trying to schedule a worker to run during the midnight hour  
like this:

:schedules:
   :collector:
     :process_something:
       :trigger_args: 30 * 0 * * 2,3,4,5,6


When I try to start backgroundrb, the "collector" worker failed to  
start up and I find this in backgroundrb_server.log:


/home/mengkuan/apps/testrailsapp/current/vendor/plugins/backgroundrb/ 
server/cron_trigger.rb:160:in `local': argument out o
f range (ArgumentError)
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/ 
backgroundrb/server/cron_trigger.rb:160:in `fire_time
_after'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/ 
backgroundrb/server/meta_worker.rb:266:in `new_load_s
chedule'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/ 
backgroundrb/server/meta_worker.rb:258:in `each'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/ 
backgroundrb/server/meta_worker.rb:258:in `new_load_s
chedule'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/ 
backgroundrb/server/meta_worker.rb:183:in `worker_ini
t'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/ 
backgroundrb/framework/packet/worker.rb:20:in `start_
worker'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/ 
backgroundrb/framework/packet/packet_master.rb:139:in
  `fork_and_load'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/ 
backgroundrb/framework/packet/packet_master.rb:98:in
`load_workers'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/ 
backgroundrb/framework/packet/packet_master.rb:93:in
`each'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/ 
backgroundrb/framework/packet/packet_master.rb:93:in
`load_workers'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/ 
backgroundrb/framework/packet/packet_master.rb:19:in
`run'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/ 
backgroundrb/server/master_worker.rb:163:in `initiali
ze'
         from script/backgroundrb:41:in `new'
         from script/backgroundrb:41


However, if I modify the trigger_args line like the following, the  
worker runs fine:

:schedules:
   :collector:
     :process_something:
       :trigger_args: 30 * 0,1 * * 2,3,4,5,6


cheers,
mengkuan

From scott at shefield.com  Wed Jan 30 13:12:25 2008
From: scott at shefield.com (Scott Ward)
Date: Wed, 30 Jan 2008 10:12:25 -0800
Subject: [Backgroundrb-devel] Schedule parser
In-Reply-To: <D6411E1F-DC02-4347-AA1D-7C32B3358833@gmail.com>
Message-ID: <20080130181117.EBDA718585FE@rubyforge.org>

I wonder if its because you need 7 entries for trigger_args?

>From the docs:

The fields (which can be an asterisk, meaning all valid patterns) are:
    sec[0,59] min[0,59], hour[0,23], day[1,31], month[1,12], weekday[0,6],
year

Looks like you're missing "year".  Of course, that may not be right since
your second example works...

Scotty


-----Original Message-----
From: backgroundrb-devel-bounces at rubyforge.org
[mailto:backgroundrb-devel-bounces at rubyforge.org] On Behalf Of Meng Kuan
Sent: Wednesday, January 30, 2008 9:59 AM
To: backgroundrb-devel at rubyforge.org
Subject: [Backgroundrb-devel] Schedule parser

Hi,

I was trying to schedule a worker to run during the midnight hour like this:

:schedules:
   :collector:
     :process_something:
       :trigger_args: 30 * 0 * * 2,3,4,5,6


When I try to start backgroundrb, the "collector" worker failed to start up
and I find this in backgroundrb_server.log:


/home/mengkuan/apps/testrailsapp/current/vendor/plugins/backgroundrb/
server/cron_trigger.rb:160:in `local': argument out o f range
(ArgumentError)
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/
backgroundrb/server/cron_trigger.rb:160:in `fire_time _after'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/
backgroundrb/server/meta_worker.rb:266:in `new_load_s chedule'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/
backgroundrb/server/meta_worker.rb:258:in `each'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/
backgroundrb/server/meta_worker.rb:258:in `new_load_s chedule'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/
backgroundrb/server/meta_worker.rb:183:in `worker_ini t'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/
backgroundrb/framework/packet/worker.rb:20:in `start_ worker'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/
backgroundrb/framework/packet/packet_master.rb:139:in
  `fork_and_load'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/
backgroundrb/framework/packet/packet_master.rb:98:in
`load_workers'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/
backgroundrb/framework/packet/packet_master.rb:93:in
`each'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/
backgroundrb/framework/packet/packet_master.rb:93:in
`load_workers'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/
backgroundrb/framework/packet/packet_master.rb:19:in
`run'
         from /home/mengkuan/apps/testrailsapp/current/vendor/plugins/
backgroundrb/server/master_worker.rb:163:in `initiali ze'
         from script/backgroundrb:41:in `new'
         from script/backgroundrb:41


However, if I modify the trigger_args line like the following, the worker
runs fine:

:schedules:
   :collector:
     :process_something:
       :trigger_args: 30 * 0,1 * * 2,3,4,5,6


cheers,
mengkuan
_______________________________________________
Backgroundrb-devel mailing list
Backgroundrb-devel at rubyforge.org
http://rubyforge.org/mailman/listinfo/backgroundrb-devel


From mengkuan at gmail.com  Wed Jan 30 13:19:16 2008
From: mengkuan at gmail.com (Meng Kuan)
Date: Thu, 31 Jan 2008 02:19:16 +0800
Subject: [Backgroundrb-devel] Schedule parser
In-Reply-To: <20080130181117.EBDA718585FE@rubyforge.org>
References: <20080130181117.EBDA718585FE@rubyforge.org>
Message-ID: <38BC7A31-5E72-4579-B739-A1D4B794EE38@gmail.com>


On 31 Jan 2008, at 2:12 AM, Scott Ward wrote:

> I wonder if its because you need 7 entries for trigger_args?
>
>> From the docs:
>
> The fields (which can be an asterisk, meaning all valid patterns) are:
>     sec[0,59] min[0,59], hour[0,23], day[1,31], month[1,12], weekday 
> [0,6],
> year
>
> Looks like you're missing "year".  Of course, that may not be right  
> since
> your second example works...
>

I just added a '*' to the end for the 'year' argument as you  
mentioned but same results. The year argument is probably optional.


Btw, the trigger_args that actually worked for me is

   :trigger_args: 30 * 0,1,2 * * 2,3,4,5,6


This one that I posted in my first mail gave the same errors:

   :trigger_args: 30 * 0,1 * * 2,3,4,5,6

From mengkuan at gmail.com  Thu Jan 31 01:34:51 2008
From: mengkuan at gmail.com (Meng Kuan)
Date: Thu, 31 Jan 2008 14:34:51 +0800
Subject: [Backgroundrb-devel] Schedule parser
In-Reply-To: <38BC7A31-5E72-4579-B739-A1D4B794EE38@gmail.com>
References: <20080130181117.EBDA718585FE@rubyforge.org>
	<38BC7A31-5E72-4579-B739-A1D4B794EE38@gmail.com>
Message-ID: <74C80CDB-9F63-45AD-8C81-E46D33B02F90@gmail.com>


On 31 Jan 2008, at 2:19 AM, Meng Kuan wrote:

>
> On 31 Jan 2008, at 2:12 AM, Scott Ward wrote:
>
>> I wonder if its because you need 7 entries for trigger_args?
>>
>>> From the docs:
>>
>> The fields (which can be an asterisk, meaning all valid patterns)  
>> are:
>>     sec[0,59] min[0,59], hour[0,23], day[1,31], month[1,12],  
>> weekday[0,6],
>> year
>>
>> Looks like you're missing "year".  Of course, that may not be  
>> right since
>> your second example works...
>>
>
> I just added a '*' to the end for the 'year' argument as you  
> mentioned but same results. The year argument is probably optional.
>
>
> Btw, the trigger_args that actually worked for me is
>
>   :trigger_args: 30 * 0,1,2 * * 2,3,4,5,6
>
>
> This one that I posted in my first mail gave the same errors:
>
>   :trigger_args: 30 * 0,1 * * 2,3,4,5,6


The above still did not solve the problem. So I dug deeper and found  
a possible logic bug in cron_trigger.rb file. It appears that  
cron_trigger incremented the day value beyond the number of days of  
the current month (from 31 to 32) and this caused the Time.local call  
to abort.

Here's my patch to adjust for this.

$ diff -c3 cron_trigger.rb cron_trigger.rb.new
*** cron_trigger.rb     Thu Dec 20 17:27:47 2007
--- cron_trigger.rb.new Thu Jan 31 14:25:57 2008
***************
*** 126,131 ****
--- 126,135 ----
             if next_hour < hour
               hour = next_hour
               day += 1
+             if day > month_days(year, month)
+               day -= month_days(year, month)
+               month += 1
+             end
               retry
             end
             hour = next_hour


This seems to finally fix the problem for me.

cheers,
mengkuan

From leavengood at gmail.com  Thu Jan 31 09:34:09 2008
From: leavengood at gmail.com (Ryan Leavengood)
Date: Thu, 31 Jan 2008 09:34:09 -0500
Subject: [Backgroundrb-devel] Authoritative Documentation
In-Reply-To: <BDE5B727-B540-416F-A14A-160A7DEB8B41@gmail.com>
References: <9DD5416C-5A15-4D35-849B-4692A9570683@gmail.com>
	<fb283d5e0801281812x7c66368fw88a4dc43cfc938f8@mail.gmail.com>
	<BDE5B727-B540-416F-A14A-160A7DEB8B41@gmail.com>
Message-ID: <a4855ac50801310634o6123d25ag4a398e1adb8be7fb@mail.gmail.com>

On Jan 29, 2008 11:34 AM, Norman Elton <normelton at gmail.com> wrote:
>
> I've pasted my worker online:
>
> http://pastie.caboo.se/144892
>
> Any thoughts?

The problem is you set the worker to auto_load but don't start it in
the console before trying to send a request. You either need to remove
the "set_no_auto_load true" from the worker, or make the following
call in the console before sending the request:

MiddleMan.new_worker(:worker => :nf_server_worker)

Regards,
Ryan

From leavengood at gmail.com  Thu Jan 31 09:35:23 2008
From: leavengood at gmail.com (Ryan Leavengood)
Date: Thu, 31 Jan 2008 09:35:23 -0500
Subject: [Backgroundrb-devel] Authoritative Documentation
In-Reply-To: <a4855ac50801310634o6123d25ag4a398e1adb8be7fb@mail.gmail.com>
References: <9DD5416C-5A15-4D35-849B-4692A9570683@gmail.com>
	<fb283d5e0801281812x7c66368fw88a4dc43cfc938f8@mail.gmail.com>
	<BDE5B727-B540-416F-A14A-160A7DEB8B41@gmail.com>
	<a4855ac50801310634o6123d25ag4a398e1adb8be7fb@mail.gmail.com>
Message-ID: <a4855ac50801310635y1d622a73tecd461927094162f@mail.gmail.com>

On Jan 31, 2008 9:34 AM, Ryan Leavengood <leavengood at gmail.com> wrote:
>
> The problem is you set the worker to auto_load but don't start it in

Rather you set it to NOT auto_load...the rest of my message is accurate :)

Ryan

From thomas.a.wood at uconn.edu  Thu Jan 31 11:48:25 2008
From: thomas.a.wood at uconn.edu (Tom Wood)
Date: Thu, 31 Jan 2008 11:48:25 -0500
Subject: [Backgroundrb-devel] cron scheduling problem
Message-ID: <8C81AA7D3B12F4408C6B3359AEB001CC040B2B76@LIB-EMarks.library.lib.uconn.edu>

I'm running a fresh checkout of release 1.0.1 ...

 

It's now 11:32 am local time on January 31.

 

I've configured a single worker for a test.

 

If I use the following schedule in my background.yaml file:

 

:schedules:

  :debugger_worker:

    :ping:

      :trigger_args: 0 35 11 * * * *

 

Backgroundrb starts up without error.

 

If I make the following change:

 

      :trigger_args: 0 30 11 * * * *

 

Backgroundrb complains:

 

brb-test/vendor/plugins/backgroundrb/server/cron_trigger.rb:161:in
`local': argument out of range (ArgumentError)

        from
brb-test/vendor/plugins/backgroundrb/server/cron_trigger.rb:161:in
`fire_time_after'

        from
brb-test/vendor/plugins/backgroundrb/server/meta_worker.rb:266:in
`new_load_schedule'

        from
brb-test/vendor/plugins/backgroundrb/server/meta_worker.rb:258:in `each'

        from
brb-test/vendor/plugins/backgroundrb/server/meta_worker.rb:258:in
`new_load_schedule'

        from
brb-test/vendor/plugins/backgroundrb/server/meta_worker.rb:183:in
`worker_init'

        from
brb-test/vendor/plugins/backgroundrb/framework/worker.rb:20:in
`start_worker'

        from
brb-test/vendor/plugins/backgroundrb/framework/packet_master.rb:134:in
`fork_and_load'

        from
brb-test/vendor/plugins/backgroundrb/framework/packet_master.rb:98:in
`load_workers'

        from
brb-test/vendor/plugins/backgroundrb/framework/packet_master.rb:93:in
`each'

        from
brb-test/vendor/plugins/backgroundrb/framework/packet_master.rb:93:in
`load_workers'

        from
brb-test/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in
`run'

        from
brb-test/vendor/plugins/backgroundrb/server/master_worker.rb:163:in
`initialize'

        from script/backgroundrb:60:in `new'

        from script/backgroundrb:60

 

A puts added to cron_trigger.rb reveals why Time.local is failing.  When
the worker is scheduled to run at 11:35, Time.local is called ala:

 

Calling Time.local 0, 35, 11, 31, 1, 2008, 4, 31, false, EST

 

When the worker is scheduled to run at 11:30, Time.local fails and is
called:

 

Calling Time.local 0, 30, 11, 32, 1, 2008, 4, 31, false, EST

 

"32" is not a valid day of the month.

 

I think the error is dependent on the time that backgroundrb starts.
When I waited until 11:40, backgroundrb started to generate the same
exception for the 11:35 worker.

 

-Tom

 

Tom Wood

thomas.a.wood at uconn.edu <mailto:thomas.a.wood at uconn.edu> 

ITS Applications Developer

University of Connecticut Libraries

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080131/ab43b380/attachment.html 

From mengkuan at gmail.com  Thu Jan 31 12:02:23 2008
From: mengkuan at gmail.com (Meng Kuan)
Date: Fri, 1 Feb 2008 01:02:23 +0800
Subject: [Backgroundrb-devel] cron scheduling problem
In-Reply-To: <8C81AA7D3B12F4408C6B3359AEB001CC040B2B76@LIB-EMarks.library.lib.uconn.edu>
References: <8C81AA7D3B12F4408C6B3359AEB001CC040B2B76@LIB-EMarks.library.lib.uconn.edu>
Message-ID: <0179DE8F-933E-4C88-B508-40C101824F24@gmail.com>

On 01 Feb 2008, at 12:48 AM, Tom Wood wrote:

> I?m running a fresh checkout of release 1.0.1 ?
>
>
>
> It?s now 11:32 am local time on January 31.
>
>
>
> I?ve configured a single worker for a test.
>
>
>
> If I use the following schedule in my background.yaml file:
>
>
>
> :schedules:
>
>   :debugger_worker:
>
>     :ping:
>
>       :trigger_args: 0 35 11 * * * *
>
>
>
> Backgroundrb starts up without error.
>
>
>
> If I make the following change:
>
>
>
>       :trigger_args: 0 30 11 * * * *
>
>
>
> Backgroundrb complains:
>
>
>
> brb-test/vendor/plugins/backgroundrb/server/cron_trigger.rb:161:in  
> `local': argument out of range (ArgumentError)
>
>         from brb-test/vendor/plugins/backgroundrb/server/ 
> cron_trigger.rb:161:in `fire_time_after'
>
>         from brb-test/vendor/plugins/backgroundrb/server/ 
> meta_worker.rb:266:in `new_load_schedule'
>
>         from brb-test/vendor/plugins/backgroundrb/server/ 
> meta_worker.rb:258:in `each'
>
>         from brb-test/vendor/plugins/backgroundrb/server/ 
> meta_worker.rb:258:in `new_load_schedule'
>
>         from brb-test/vendor/plugins/backgroundrb/server/ 
> meta_worker.rb:183:in `worker_init'
>
>         from brb-test/vendor/plugins/backgroundrb/framework/ 
> worker.rb:20:in `start_worker'
>
>         from brb-test/vendor/plugins/backgroundrb/framework/ 
> packet_master.rb:134:in `fork_and_load'
>
>         from brb-test/vendor/plugins/backgroundrb/framework/ 
> packet_master.rb:98:in `load_workers'
>
>         from brb-test/vendor/plugins/backgroundrb/framework/ 
> packet_master.rb:93:in `each'
>
>         from brb-test/vendor/plugins/backgroundrb/framework/ 
> packet_master.rb:93:in `load_workers'
>
>         from brb-test/vendor/plugins/backgroundrb/framework/ 
> packet_master.rb:19:in `run'
>
>         from brb-test/vendor/plugins/backgroundrb/server/ 
> master_worker.rb:163:in `initialize'
>
>         from script/backgroundrb:60:in `new'
>
>         from script/backgroundrb:60
>
>
>
> A puts added to cron_trigger.rb reveals why Time.local is failing.   
> When the worker is scheduled to run at 11:35, Time.local is called  
> ala:
>
>
>
> Calling Time.local 0, 35, 11, 31, 1, 2008, 4, 31, false, EST
>
>
>
> When the worker is scheduled to run at 11:30, Time.local fails and  
> is called:
>
>
>
> Calling Time.local 0, 30, 11, 32, 1, 2008, 4, 31, false, EST
>
>
>
> ?32? is not a valid day of the month.
>
>
>
> I think the error is dependent on the time that backgroundrb  
> starts.  When I waited until 11:40, backgroundrb started to  
> generate the same exception for the 11:35 worker.
>
>
>

Hi Tom,

The following is from an email I sent to the list earlier today. You  
can give it a try:

It appears that cron_trigger incremented the day value beyond the  
number of days of the current month (from 31 to 32) and this caused  
the Time.local call to abort.

Here's my patch to adjust for this.

$ diff -c3 cron_trigger.rb cron_trigger.rb.new
*** cron_trigger.rb     Thu Dec 20 17:27:47 2007
--- cron_trigger.rb.new Thu Jan 31 14:25:57 2008
***************
*** 126,131 ****
--- 126,135 ----
             if next_hour < hour
               hour = next_hour
               day += 1
+             if day > month_days(year, month)
+               day -= month_days(year, month)
+               month += 1
+             end
               retry
             end
             hour = next_hour




From gethemant at gmail.com  Thu Jan 31 12:28:31 2008
From: gethemant at gmail.com (hemant)
Date: Thu, 31 Jan 2008 22:58:31 +0530
Subject: [Backgroundrb-devel] Schedule parser
In-Reply-To: <74C80CDB-9F63-45AD-8C81-E46D33B02F90@gmail.com>
References: <20080130181117.EBDA718585FE@rubyforge.org>
	<38BC7A31-5E72-4579-B739-A1D4B794EE38@gmail.com>
	<74C80CDB-9F63-45AD-8C81-E46D33B02F90@gmail.com>
Message-ID: <fb283d5e0801310928o4731ae4ekba5ea2cd21e2eaaf@mail.gmail.com>

On Thu, Jan 31, 2008 at 12:04 PM, Meng Kuan <mengkuan at gmail.com> wrote:
>
>  On 31 Jan 2008, at 2:19 AM, Meng Kuan wrote:
>
>  >
>  > On 31 Jan 2008, at 2:12 AM, Scott Ward wrote:
>  >
>  >> I wonder if its because you need 7 entries for trigger_args?
>  >>
>  >>> From the docs:
>  >>
>  >> The fields (which can be an asterisk, meaning all valid patterns)
>  >> are:
>  >>     sec[0,59] min[0,59], hour[0,23], day[1,31], month[1,12],
>  >> weekday[0,6],
>  >> year
>  >>
>  >> Looks like you're missing "year".  Of course, that may not be
>  >> right since
>  >> your second example works...
>  >>
>  >
>  > I just added a '*' to the end for the 'year' argument as you
>  > mentioned but same results. The year argument is probably optional.
>  >
>  >
>  > Btw, the trigger_args that actually worked for me is
>  >
>  >   :trigger_args: 30 * 0,1,2 * * 2,3,4,5,6
>  >
>  >
>  > This one that I posted in my first mail gave the same errors:
>  >
>  >   :trigger_args: 30 * 0,1 * * 2,3,4,5,6
>
>
>  The above still did not solve the problem. So I dug deeper and found
>  a possible logic bug in cron_trigger.rb file. It appears that
>  cron_trigger incremented the day value beyond the number of days of
>  the current month (from 31 to 32) and this caused the Time.local call
>  to abort.
>
>  Here's my patch to adjust for this.
>
>  $ diff -c3 cron_trigger.rb cron_trigger.rb.new
>  *** cron_trigger.rb     Thu Dec 20 17:27:47 2007
>  --- cron_trigger.rb.new Thu Jan 31 14:25:57 2008
>  ***************
>  *** 126,131 ****
>  --- 126,135 ----
>              if next_hour < hour
>                hour = next_hour
>                day += 1
>  +             if day > month_days(year, month)
>  +               day -= month_days(year, month)
>  +               month += 1
>  +             end
>                retry
>              end
>              hour = next_hour
>
>
>  This seems to finally fix the problem for me.

Thanks Meng, We will have this patch in.

-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

From andre at myavalaunch.com  Thu Jan 31 21:55:11 2008
From: andre at myavalaunch.com (andre)
Date: Thu, 31 Jan 2008 18:55:11 -0800
Subject: [Backgroundrb-devel] Setting up release-1.0.1
In-Reply-To: <fb283d5e0801282005n4ae9e755sba7f4fe1f6741f85@mail.gmail.com>
References: <d76343c4338d06e21ee507263bcc9085@localhost>	
	<fb283d5e0801281815q15205debyd4fa6ca96c1cc44d@mail.gmail.com>	
	<479E9221.3090409@myavalaunch.com>
	<fb283d5e0801282005n4ae9e755sba7f4fe1f6741f85@mail.gmail.com>
Message-ID: <47A28A0F.9030702@myavalaunch.com>

hemant wrote:
> On Tue, Jan 29, 2008 at 8:10 AM, andre <andre at myavalaunch.com> wrote:
>   
>>  hemant wrote:
>>
>>  On Tue, Jan 29, 2008 at 5:43 AM, Andre Borrelly <andre at myavalaunch.com>
>> wrote:
>>
>>
>>  Hello,
>>
>>  I am unable to get the backgroundrb server to run.
>>
>>  Initially I got:
>>
>>
>> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:263:in
>> `load_missing_constant': uninitialized constant BackgrounDRb::MasterProxy
>> (NameError)
>>  from
>> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:453:in
>> `const_missing'
>>  from ./script/backgroundrb:41
>>
>>  Then after doing this:
>>
>>  Index: script/backgroundrb
>>  ===================================================================
>>  --- script/backgroundrb (revision 313)
>>  +++ script/backgroundrb (working copy)
>>  @@ -6,8 +6,10 @@
>>  WORKER_ROOT = rails_root + "/lib/workers"
>>  SERVER_LOGGER = rails_root + "/log/backgroundrb_server.log"
>>
>>  -["server","framework","lib"].each { |x| $LOAD_PATH.unshift(PACKET_APP +
>> "/#{x}")}
>>  -$LOAD_PATH.unshift(WORKER_ROOT)
>>  +#["server","framework","lib"].each { |x| $LOAD_PATH.unshift(PACKET_APP +
>> "/#{x}")}
>>  +["server","framework","lib"].each { |x| $:.push(PACKET_APP + "/#{x}")}
>>  +#$LOAD_PATH.unshift(WORKER_ROOT)
>>  +$:.push(WORKER_ROOT)
>>
>>  require RAILS_HOME + '/config/boot.rb'
>>  require "active_record"
>>
>>
>>  and adding in "environment.rb"
>>
>>  config.load_paths +=
>> Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/server"]
>>  config.load_paths +=
>> Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/framework"]
>>
>>  I get this:
>>
>>
>> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:249:in
>> `load_missing_constant': Expected
>> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb to define
>> MasterWorker (LoadError)
>>  from
>> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:453:in
>> `const_missing'
>>  from
>> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:465:in
>> `const_missing'
>>  from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:96:in
>> `const_get'
>>  from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:96:in
>> `load_workers'
>>  from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:93:in
>> `each'
>>  from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:93:in
>> `load_workers'
>>  from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in
>> `run'
>>  from
>> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:163:in
>> `initialize'
>>  from ./script/backgroundrb:62:in `new'
>>  from ./script/backgroundrb:62
>>  RAILS_HOME/vendor/plugins/backgroundrb/lib/../framework/nbio.rb:23:in
>> `read_data': Packet::DisconnectError (Packet::DisconnectError)
>>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/worker.rb:47:in
>> `handle_internal_messages'
>>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:158:in
>> `start_reactor'
>>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:156:in
>> `each'
>>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:156:in
>> `start_reactor'
>>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:147:in
>> `loop'
>>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/core.rb:147:in
>> `start_reactor'
>>  from RAILS_HOME/vendor/plugins/backgroundrb/framework/worker.rb:21:in
>> `start_worker'
>>  from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:134:in
>> `fork_and_load'
>>  from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:115:in
>> `start_worker'
>>  from
>> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:165:in
>> `initialize'
>>  from
>> RAILS_HOME/vendor/plugins/backgroundrb/framework/packet_master.rb:18:in
>> `run'
>>  from
>> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb:163:in
>> `initialize'
>>  from ./script/backgroundrb:62:in `new'
>>  from ./script/backgroundrb:62
>>
>>
>>  Did you actually delete your old, script/backgroundrb file and ran
>>
>> rake backgroundrb:setup
>>
>> Often you get start problems with new releases, because your plugin is
>> upgraded, but start/stop script is not.
>> Which OS you are on?
>>
>>  Yes I have just followed those steps, but still no go.
>>  I get:
>>
>> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:266:in
>> `load_missing_constant': uninitialized constant MasterWorker (NameError)
>>
>>  If I add this in the environment.rb
>>
>>    config.load_paths +=
>> Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/server"]
>>    config.load_paths +=
>> Dir["#{RAILS_ROOT}/vendor/plugins/backgroundrb/framework"]
>>  I get
>>
>> /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.1/lib/active_support/dependencies.rb:249:in
>> `load_missing_constant': Expected
>> RAILS_HOME/vendor/plugins/backgroundrb/server/master_worker.rb to define
>> MasterWorker (LoadError)
>>     
>
> I am unable to reproduce this at my end. Can you try creating a fresh
> rails app and check out the plugin. Send me the tarball of your sample
> app, if all else fails.
>   
For future reference I found the cause of this problem.  Apparently I 
generated a worker called Master which created a MasterWorker class.  
This appears to be a namespace conflict with a backgroundrb system class.

I hope someone doesn't have to dig as far as I did to find this problem.

Hemant, is my evaluation correct?

Thank you for all the time you have spent working on this tool.

Let me know if I can be of any help on the persistence feature request.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080131/331ba4af/attachment.html