[Backgroundrb-devel] worker process hanging around, spiking memory

Greg Campbell gtcampbell at gmail.com
Wed Feb 27 19:07:12 EST 2008


Hi Clint,

On Wed, Feb 27, 2008 at 3:26 PM, Clint Troxel <clint at ctro.net> wrote:

> Hi. Posted a general question earlier today.  I have a specific
> question this time -- hoping someone can help out.  I think I'm
> probably doing something incorrectly.  Here goes:
>
> I have a Worker method that sends a lot of emails.  The method looks like
> this:
>
>
> def send_participant_reminders(args=nil)
>    args[:emails].each do |email|
>
>      unless email.notified?
>        Emailer.deliver_survey_reminder(email)
>      end
>
>    end
>  end
>
>
> I call that worker from a controller like so:
>
>    MiddleMan.new_worker(:worker => :participant_worker)
>    MiddleMan.ask_work(:worker => :participant_worker,
>      :worker_method => :send_participant_reminders,
>      :data => {:emails => @emails})
>
> ----------------------------
>
> My emails are being sent, but I'm seeing a new process created *each
> time I invoke the code in the controller*.  That process starts out at
> about 17m of memory then, once all work appears to have been
> completed, jumps to about 50m.  The process stays around until
> "script/backgroundrb stop".
>
> Maybe I've been reading the examples incorrectly?
>
> Any help would be appreciated.  Seeing this same problem (I think)
> suck up 4G of ram on a production server.  yuck.
>
> Thanks again,
> clint


Calling new_worker does indeed create a new backgroundrb process (which may
or may not be necessary in your case, see below).  If you do in fact need to
create a new process for each task, you'll need to call either exit (from
within the worker) or MiddleMan.delete_worker (from the controller) once the
task is done. If you need to keep track of these worker tasks while they're
running (to call delete_worker from the controller, or to use ask_status),
you should also use a job_key when you create it.

You may not actually need the new process, however.  If the likely use case
is that there will only be one of these tasks running at a time, then you
can just omit the new_worker call entirely and direct all calls to a single
running instance of the worker process.  You may also want to investigate
using thread_pool.defer inside the worker to run the individual tasks in
threads (rather than spawning new worker processes for each one) if it seems
likely that there will be multiple requests running simultaneously.

Thanks,
Greg
p.s. If I've misrepresented anything, I'd like to invite Hemant or anyone
else more knowledgeable than I am to correct me, as I've only started
working on backgroundrb in the last few weeks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080227/aa903a9b/attachment.html 


More information about the Backgroundrb-devel mailing list